In recent years, new developments in science education have been witnessed. The concept of engineering is now found in teaching curricula – just one example of these new developments. Engineering concepts are also in harmony with the nature of science education. In particular, integrating engineering in science education and linking disciplines in science, technology, engineering and mathematic (STEM) education are reform efforts aimed at meeting 21st– century needs in education (National Research Council [NRC], 2009).
One of the new subjects in the school setting in US K-12 is engineering (NRC, 2005). The NRC (2005) covers the relationship between the foundations of science standards and technological design processes, with students determining the problem, developing a solution or designing a product, applying the design, and finally evaluating the design. Similarly, the concept of engineering was included in middle school science curricula (Grades 5-8) in Turkey.
In this way, a new skill area has been added to the 2018 middle school science curriculum under the name of “engineering and design skills.” This skill area was aimed at students being able to integrate science, mathematics, technology and engineering subjects and solving problems with a cross-disciplinary approach (The Ministry of Education, 2018).
In parallel with these developments, Turkish middle school science teachers must incorporate engineering designs for their students in their science courses. Science teachers and prospective science teachers must first have an awareness with regard to engineering design. Thus, in Turkey, it was noted that science teachers’ and prospective science teachers’ experiences regarding STEM education should be improved as part of preservice and in-service training (Akgündüz et al., 2015).
STEM education is expressed as an interdisciplinary teaching system, consisting of practical approaches aimed at integrating the four disciplines (Bybee, 2010). In recent years, STEM education has been seen as one of the most notable innovations in engineering design. In this sense, it has been pointed out that STEM education and the engineering design process, which is one of the dimensions of STEM, may have important outcomes for students. For example, in the STEM education process, an interdisciplinary perspective becomes dominant, and students are involved in an inquiry-based learning process (Bell, 2010; Eron & Rachlin, 2015; Milaturrahmah, Mardiyana, & Pramudya, 2017).
Engineering design activities are a powerful strategy for the integration of science, mathematics and technology (Cantrell, Pekcan, Itani, & Velasquez-Bryant, 2006). Moreover, science inquiry and engineering design offer learning opportunities to embody K-12 STEM education (NRC, 2011). Thus, most governments have introduced strong initiatives to promote STEM awareness and motivation, as STEM is one of the competitive areas that will determine a nation’s future status (Bahar & Adıguzel, 2016).
In this regard, STEM awareness is seen as a prerequisite for individual interaction, self-efficacy, and self-development (Kovarik et al., 2013). The awareness to be created with regard to STEM will, therefore, both increase the four different disciplines’ importance and increase the number of individuals that the present era needs. Thanks to STEM education, more importance has been given to the design process (the engineering dimension) in terms of science courses (Bequette & Bequette, 2012). Thus, it is emphasized that science, or STEM, must be associated with other subjects such as philosophy, language, history, and the various disciplines at all levels of education (European Commission, 2015).
Rube Goldberg Machines
Rube Goldberg was not only an engineer, he was a popular cartoonist at the beginning of the 20th century. Moreover, although he is known for his drawings, he also designed machines involving a series of complex steps to perform simple tasks (Howard, Williams, & Yao, 2010). In this sense, Rube Goldberg was a man who became synonymous with the use of convoluted, complicated machines to carry out simple tasks (Pierson & Suchora, 2002).
For example, if the goal is to turn on a light switch, a bowling ball that descends from a ramp hits an arm that triggers the fall of a line of dominoes, creating a series of waves. This wave strikes the button, causing a mechanism to be sprung that causes the light to come on as intended (Quigley, Herro, & Jamil, 2017).
Rube Goldberg machines involving a chain reaction have been used in science education because they are also science-focussed and particularly suitable for science. Various studies have concentrated on concepts related to physics and mathematics (Brush, 2017; Davis, Chlebowski, & Ellert, 2017; Ganesh & Thieken, 2010; O’Connor, 2003; Selvi & Soto-Caban, 2016; Yanik, Ferguson, Kaul, & Yan, 2017).
Ganesh and Thieken (2010), for example, gave various tools to seventh-grade students for creating a simple circuit (a battery pack, power cables, buzzers, a light-emitting diode LED, switches, milk/juice cartons, coat hangers, aluminum foil, and cardboard). Then the students explored different combinations with regard to building electrical circuits and formed various circuits with chain reactions.
Brush (2017) showed how Rube Goldberg machines can be used for Grade 6-8 students in teaching force and motion. Similarly, O’Connor (2003) stated that Rube Goldberg machines could be used for teaching metric measurement (mathematics) and simple machines (physics) to fifth-grade students.
Kim and Park (2012) pointed out that Rube Goldberg machines have also helped to develop positive attitudes on the part of students toward science. Thus, teaching science concepts and creating awareness about engineering to students may be possible using Rube Goldberg machines.
Additionally, Rube Goldberg machines can be used to create a STEM experience in the form of an interdisciplinary activity integrating science, technology, and engineering as part of an authentic problem-solving project (Ambrose & Sternberg, 2016). Similarly, it was pointed out that Rube Goldberg machines not only integrate STEM concepts, but also require individuals to craft the design of the machine creatively (O’Byrne et al., 2018).
Thus, it can be said that Rube Goldberg machines could be important in terms of creating engineering awareness. In this sense, Marklin (2018) stated that Rube Goldberg machines are not only drawings, but also innovative engineering designs, while Acharya and Sirinterlikci (2010) noted that Rube Goldberg machines have been used for engineering design. In this way, students who are exposed to design-oriented processes such as a Rube Goldberg machines may become aware of what is involved and understand what STEM education means.
In this current study, a design cycle was needed to create Rube Goldberg machines, and it was decided that the steps of the engineering design process comprised the most appropriate design cycle. In the process of creating STEM designs, students can be inspired by Rube Goldberg machines (Marklin, 2018). Consequently, the engineering design process cycle shown in Figure 1 was taken into consideration when designing Rube Goldberg machines.
Steps followed by this engineering design process are as follows:
- Students determine a purpose or problem (for example, the need to blow out a candle).
- Students research the problem by using the internet, the library, or experts in order to decide which science concepts to use.
- Students decide which simple machines are most suitable, and they develop many simple machine ideas to be used (for example, a wheel and axle, a lever, a wedge, a pulley, compound machines, an inclined plane, and a screw).
- Students choose the most suitable simple machine or science concepts to solve the problem (for example, using the lever for the relevant stage).
- Students draw the draft they first imagined on paper. They then make prototypes with tools that they use in daily life and which do not cost money (for example, a bottle, cardboard, or wood).
- Students test the Rube Goldberg machines which consisting of two, three, five or more stages, (depending on the number of steps per week), and as part of this process, they test whether the machine has deteriorated or does not work.
- Students present the Rube Goldberg machines in the classroom environment, by explaining the tools and the science concept they use. Other groups express their opinions about the Rube Goldberg machines during the presentation, and so the Rube Goldberg machines can be improved (at this stage, one group builds their Rube Goldberg machine in the classroom environment, and other groups introduce their machines with pictures and video).
- Students prepare for the following week by redesigning their machines, taking into consideration the suggestions and opinions expressed by their peers in the previous step. This cycle should be repeated for 10 weeks, and a machine consisting of at least 10 stages should be prepared by the end of the application.
Studies of Rube Goldberg machines are generally found to be in two categories. The first category includes studies related to engineering students at the university level. The second category includes studies related to middle school and high school students.
In the first of these categories, engineering students designed Rube Goldberg machines in a 6-14 week process. The research results showed that Rube Goldberg machines make engineering students feel happy because they are working toward a goal. They also find the design process to be fun, it develops their imagination, their teamworking skills, and their time management abilities, it encourages cooperation, extraordinary thinking, and social networking, and it develops communication skills, engineering skills, and leadership skills (Berg, 2015; Davis et al., 2017; DeMontigny, Smithson, & Wright, 2011; Mahinroosta & Lindsay, 2016; Selvi & Soto-Caban, 2016; Yanik et al., 2017).
Berg (2015) carried out a study with students on an engineering dynamics course who developed Rube Goldberg machines. Berg found that the engineering students were satisfied with the fact that the machines worked in the end. Davis et al. (2017) carried out a study with freshman engineering design course students and found that this process contributed to teamwork, communication and engineering skills. Similarly, Selvi and Soto-Caban (2016) carried out a study with junior level engineering students on a design course and found that they were excited when it came to display their Rube Goldberg machines, resultimg in enthusiastic teamwork.
DeMontigny and Smithson (2010) examined shortcomings and student feedback with regard to the Rube Goldberg machines applied in previous pieces of research. They found that students felt that Rube Goldberg machines were a good method for teaching engineering design in the first year of their studies. Similarly, DeMontigny et al. (2011) performed a study with engineering students in an Engineering Communications and Design course. They found that these students found the process to be fun. They also observed that the imagination of these students developed. As shown in the relevant literature, engineering students generally have a positive view of the design process involving Rube Goldberg machines. Moreover, it is possible to identify research results that emphasize that Rube Goldberg machines make teaching effective (DeMontigny, Smithson, & Wright, 2011; Mahinroosta & Lindsay, 2016; Selvi & Soto-Caban, 2016). These results indicated that Rube Goldberg machines could give effective results with regard to university level students.
In the second category, when studies related to middle school and high school students were examined, it was seen that Rube Goldberg machines attracted students’ attention, positively affected academic achievement related to science concepts, gave students the pleasure of achieving their goals, allowed students to take positive risks, and developed students’ creativity, critical thinking, and personal responsibility (Ganesh & Thieken, 2010; Jordan & Pereira, 2009; Matty, 2017; O’Connor, 2003; Sheriff, Sadan, Keats, & Zuckerman, 2017).
O’Connor (2003) found that the Rube Goldberg machines maintained the attention of fifth-grade students, introduced the students to new methods of instruction, and offered an opportunity for them to communicate with experts. Ganesh and Thieken (2010) proved that the Rube Goldberg machines had a positive effect on seventh-grade students’ knowledge of electrical circuits. Jordan and Pereira (2009) reported that the Rube Goldberg machines encouraged sufficient maturity on the part of fifth- and sixth-grade students to allow them to manage their own schedule toward an abstract goal.
Sheriff et al. (2017) found that Rube Goldberg machines promoted positive risk-taking on the part of students (aged 8-12), and the authors drew attention to the fact that most children expressed a preference toward using Rube Goldberg machines. Additionally, Matty (2017) proved that Rube Goldberg machines positively reflected core competencies such as communication, creativity, critical thinking, and personal responsibility of 11th- and 12th-grade students. According to Lei et al. (2012), Rube Goldberg machines can be used to trigger the motivation of students with regard to the engineering design process. As shown in the relevant literature, although positive results were achieved at middle school level about Rube Goldberg machines, any research on prospective science teachers who are due to educate middle school student has not been undertaken.
When the STEM studies conducted with regard to prospective science teachers are examined, it is possible to see some pieces of research regarding STEM education applications in terms of prospective science teachers. In this sense, the research results showed that design-oriented STEM training processes had a significant positive impact on prospective science teachers (Altan, Yamak, & Kırıkkaya, 2016; Yıldırım & Altun, 2015). For example, Altan et al. (2016) implemented “design based science education” in STEM education for prospective science teachers. The results of this research showed that prospective science teachers stated that this process enabled learning by doing, and also stated that the design task was motivating. Moreover, Yıldırım and Altun (2015) enabled prospective science teachers to create STEM designs by considering the 5E (Engage, Explore, Explain, Elaborate, Evaluate) learning model, in order increase their interest in engineering, and the authors found that STEM designs have a positive effect on prospective teachers’ academic achievements.
Although positive results regarding STEM education have been achieved, there are also negative results. For instance, in a survey conducted extensively in Turkey, Çolakoğlu and Gökben (2017) have found faculty members serving in faculties of education to have an awareness and interest in STEM. However, it has been determined that there are inufficient concrete applications in the faculties of education that they considered (Çolakoğlu & Gökben, 2017). Therefore, it can be said that some prospective teachers want to see concrete implementation examples regarding STEM. For example, Aslan-Tutak, Akaygün, and Tezsezen (2017) in their study, determined that prospective mathematics and chemistry teachers want to see sample projects among the priority subjects about STEM education.
Furthermore, since prospective teachers do not encounter too many practices related to STEM education, they face some difficulties when they attempt to introduce these practices for the first time. For example, Tarkın-Çelikkıran and Aydın-Günbatar (2017) stated that prospective chemistry teachers were challenged when it came to deciding on materials to be used in the STEM design process, deciding how to design the product, and investigating the necessary information. In fact, the process of designing Rube Goldberg machines can reduce the difficulties experienced by prospective teachers and can contribute to STEM awareness.
If so, the following question arises: How does the design process of the Rube Goldberg machines reflect on prospective science teachers’ STEM awareness? Consequently, it is believed that this research has an important role to play in helping answer this question and contributing to the literature. In this context, the purpose of this study is to examine the reflections of the design process of the Rube Goldberg machines on the STEM awareness of the prospective science teachers. For this purpose, the subproblems of the research are as follows:
- Does the design process of the Rube Goldberg machines have a statistically significant effect on prospective science teachers’ STEM awareness?
- What kind of changes does design process of the Rube Goldberg machines create on prospective science teachers’ STEM awareness?
A mixed design approach was used in this research. Among the reasons why a mixed design approach was preferred were the need to clarify the research findings, to make the comments of the participants more detailed, and to investigate contradictions encountered in the research process (as in Greene, Caracelli, & Graham, 1989). Consequently, an embedded experimental model was used in this research, given that this model is known for allowing the embedding of qualitative data in a quantitative experimental design (Creswell & Clark, 2017).
In the quantitative dimension of the research, a single-case experimental design was used to determine the effects of the design process associated with Rube Goldberg machines on the STEM awareness of prospective science teachers (as in Smith, 2012). In the qualitative dimension of the research, the phenomenological research method was used to examine reflections with regard to the design process involving Rube Goldberg machines on the STEM awareness of prospective science teachers. The phenomenological research method focuses on explaining the experiences that participants have when they interact with their environment (Ary, Jacobs, Sorensen, & Razavieh, 2010).
The participants in this research consisted of 23 prospective science teachers in their sophomore year. Three were male and 20 were female. The participants’ age range was from 18 to 21. In this study the purposeful sampling method was used because the experience of the prospective science teachers in designing the Rube Goldberg machines and the reflections of the Rube Goldberg machines on the prospective science teachers were examined. Purposeful sampling is defined as the selection of participants who have specific knowledge or experience identified by the researcher as a potential area for research (Sandelowski, 1995). Thus, these prospective science teachers were deemed to be the most appropriate sample in terms of the aim of this research.
The most appropriate elective course was Applied Science and Technology Teaching, which is given in the spring semester in the second year. There was no elective course that enables students to teach design in the first and third years. Senior prospective science teachers were busy preparing for the public personnel selection examination to become new teachers. Consequently, it was decided that it would be most appropriate to work with sophomore prospective science teachers.
Data Collection Tools
STEM Awareness Scale (SAS). The SAS was developed by Buyruk and Korkmaz (2017a) and was used to determine the STEM awareness of prospective teachers. The SAS was prepared using a five-item Likert-type scale which consisted of 17 items. The sample consisted of 256 prospective teachers, including prospective science teachers. The Cronbach’s alpha reliability coefficient was found to be 0.92. The original form of the scale was developed for prospective teachers, including prospective science teachers. In this regard, it can be said that this scale is valid for the sample group in the current study. In addition, research conducted on STEM education indicate that the scale is valid (Çevik, 2017; Pekbay, 2017).
In the present study, in terms of reliability, the Cronbach’s alpha for the pretest was .82, as was the case for the posttest. Therefore, this scale seems to be valid and reliable for the current research. For example, some of the items are as follows:
- “STEM contributes to the development of creativity in the field of engineering by using the basic knowledge and skills of individuals.”
- “STEM education encourages students to learn.”
- “STEM education improves the problem solving skills of the students.”
- “STEM education develops cooperative work in students.”
- “The aim of STEM education is to develop by a holistic approach by establishing a relationship between disciplines.”
The learning diary. The learning diary is a document kept on a regular daily basis and is of personal value in that it is used to record ongoing activities (Alaszewski, 2006). Diaries often contain special experiences that are not readily accessible in interviews. Moreover, diaries enable the participants to record their expectations and predictions. Also, diaries do not require the researcher to be present. It seems that diary users have the opportunity to write down details of an event or experience, and they can write it whenever they want to (Bytheway, 2012).
In addition, learning diaries that are kept for learning purposes seem to give effective feedback on student performance in the process of applying the teaching curriculum (Ruiz-Primo, 2004). Among the methods and techniques that can be used in the engineering design process, learning diaries should be used (Jaakma, Kiviluoma, & Kuosmanen, 2015; Mielke, Grünewald, & Brück, 2016). For these reasons, learning diaries were used to collect data in the current research. The learning diaries were collected weekly from each student. No special information was requested from the students when using the learning diaries in order to allow them to express their feelings and thoughts about the design process freely.
This research was conducted in an elective course, Applied Science and Technology Teaching, in the spring semester of 2016-2017 at a state university in Turkey. As can be seen in Table 1, a pretest and posttest were applied to the prospective science teachers before and after the implementation process.
Summary of the Application Process
|STEM Awareness Scale||1-Learning Dairies
2-Rube Goldberg machines
3-Introduction of designs
4- Weekly presentations
|STEM Awareness Scale|
In this section, information on the implementation process is covered in more detail. First, I gave the participants information about Rube Goldberg machines. As an example, I showed Rube Goldberg machines that had been previously completed. I also mentioned the features of these machines. For example, it is only necessary to intervene once from the outside in order to start the design. Also, at least one simple machine must be used in each step.
The name of the simple machine used must be written on the design and must be a triggering factor when passing from one stage to the next. In this way, one stage must be completed each week, and the designs must consist of at least 10 stages. At the end of the semester, the designs should consist of between 10 and 14 steps. Priority should be given to materials that are used at home, such as recycled materials, water bottles, cola boxes, and wooden or broken toys, so that an effort is made to avoid costly designs.
Finally, the students needed to have a goal at the end of their design process, when all the stages were complete. This goal may be a simple task or a joke (such as blowing out a candle or adding water to a bowl). In this process, groups of two, three, or four participants were created, and the participants created their designs in these groups. The distribution of the working groups in the design process is given in Table 2.
Working Groups in the Design Process
|1st group||P3, P7, P16||2nd group||P10, P17, P20||3rd group||P18, P19|
|4th group||P4, P6, P22||5th group||P12, P13||6th group||P8, P14|
|7th group||P9, P11||8th group||P15, P21||9th group||P1, P2, P5, P23|
In the process of exhibiting designs, only one group was to bring their developed design to the classroom each week, while the other groups were to bring pictures and videos of their designs and present them in this way. Bringing in all the designs every week would lead to confusion and loss of time for the participants. In this way, each group would bring their design once to the classroom. At the beginning of the course, the participants watched and listened to the presentation of the group that brought in their design. During the rest of the course, other groups presented their designs in the form of pictures and videos (Table 3).
Weekly Presentation Process
|Week||Presenting Group||Stages Involved in Design|
|1||–||Introduction of Rube goldberg designs, information about the implementation process|
|2||1st group||1 stage|
|3||2nd group||2 stages|
|4||3rd group||3 stages|
|5||4th group||4 stages|
|6||5th group||5 stages|
|7||6th group||6 stages|
|8||7th group||7 stages|
|9||8th group||8 stages|
|10||9th group||9 stages|
|11||Elimination of deficiencies||10 stages|
|12-13||Presentation of all groups, exhibition of designs|
|Note. Those who wanted to do more stages were not prevented from doing so.|
When presentations were made, the participants first explained the science concepts they used in their designs. Then, the simple machines used in the design were explained, and examples of places where they could be used in daily life were given. This process was repeated every week. In addition, the learning diaries written in this process were collected weekly. The sample designs are shown in the appendix.
Analysis of the Data
Quantitative data. Skewness and kurtosis values for normal distribution were found to lie between -2 and +2, and the distribution is normal in terms of STEM awareness. If skewness and kurtosis values are between -2 and +2, the data are considered to be normally distributed (according to George & Mallery, 2003). As seen in Table 4, the paired samples test was used when STEM awareness scores were analyzed.
Skewness and Kurtosis Values for Normal Distribution
|Variable: STEM Awareness||Skewness||Kurtosis|
Analysis of the qualitative data. The appropriate technique for the analysis of the learning diary data, which is converted into written text, is content analysis (Alaszewski, 2006). Consequently, a conventional content analysis technique was used. In conventional content analysis, the research is based on the emergence of new understandings (Kondracki, Wellman, & Amundson, 2002), while conventional content analysis is based on an inductive coding process (Mayring, 2000).
The process used in conventional content analysis was to combine similar data within the framework of specific concepts and themes and to edit them in such a way that the reader could understand them. First, the qualitative data obtained from the learning diaries was collected around similar semantic clauses. Codes representing similar expressions were created. Codes are tags that combine semantic similarities (Graneheim & Lundman, 2004). The analysis began with open-ended coding. That is, both the codes in the living (the actual terms used by the participants) as well as in the existing theories and terms in the literature, have been used (see Strauss & Corbin, 1998). I extracted from the analysis any expressions that did not relate to the purpose of the research before the encoding process began. This process is known as reduction. Reduction means to reduce the size, but not in such a way as to impact on the quality of the rest (Graneheim & Lundman, 2004). I carried out both the reduction and the coding process.
Moreover, to ensure reliability, a different expert researcher’s opinions were consulted to determine whether or not the codes explained the expressions. A different expert researcher was asked to classify the data according to the codes. The percentage agreement of the analyses on the part of the two investigators was calculated. For this, Cohen’s kappa coefficient was used for intercoder reliability and was found to be 0.76. In the presentation of the data, in terms of ensuring confidentiality and adhering to ethical rules, the prospective science teachers are identified as a code, e.g., P1 for the first participant. Moreover, while direct quotations from the diaries are given, information is given in brackets. For example, “(P1, 7.3.17)” means that the first participant’s diary was written on the March 7, 2017.
This research has some limitations. For example, the intervention process was carried out by the researcher/trainer himself. I gave notes to the students about setting written exams, which did not relate to the Rube Goldberg machines for the midterm and final exams. In this way, I tried to prevent students from worrying about the midterm and final exams during the design process. It takes a long time to create Rube Goldberg machines on the part of major students at the university because there should be a greater number of stages. Therefore, the intervention process was limited to 13 weeks. This process may require between 4 to 6 weeks if this research is carried out with young students.
Another limitation of the study is that there was no control group, because each grade level was composed of only one branch. Another limitation relates to the raw materials used to create the Rube Goldberg machines. I asked the students primarily to make use of materials that are used in everyday life in order to minimize cost. The vast majority of the materials used by the students consisted of recycled materials. Moreover, activities such as the release of harmful materials and harmful fumes was not allowed.
The qualitative findings of the study were limited by the participants’ characteristics. Thus, generalizations cannot be made to different participant groups and contexts.
Quantitative Results on STEM Awareness
Descriptive statistics of prospective science teachers regarding STEM awareness scores are given in Table 5. The mean score was higher in Round 2 (74.95), indicating that the prospective science teachers became more reflective in their reactions toward STEM awareness. The range between the minimum and maximum scores supports this finding. Statistically significant differences existed among the mean scores with regard to the STEM awareness of the prospective science teachers. In other words, the paired samples test showed that after the Rube Goldberg training the participants’ Round 2 reflection scores were statistically significantly higher than their Round 1 scores (t(22) = -3.201; p < .05).
Descriptive Statistics of Prospective Science Teachers Regarding STEM Awareness Scores
Qualitative Results on STEM Awareness
Qualitative findings were obtained from the diaries that prospective science teachers wrote each week during the design process associated with Rube Goldberg machines. The results obtained from the diaries related to the STEM awareness are included in Table 6.
Findings Related to STEM Awareness
|The difficulty of the design process||P1, P2, P3, P4, P6, P8, P9, P10, P11, P14||10|
|Happiness after completing each stage of the design||P1, P2, P3, P4, P9, P10, P11, P13, P14, P19||10|
|The psychology of uncertainty in the design process||P4, P6, P8, P12, P13, P18, P19||7|
|Learning science concepts||P6, P8, P9, P11, P12, P13, P14||7|
|Discussions on the improvement of the design||P1, P9, P11, P17, P18, P19,||6|
|To have fun in the design process||P2, P11, P12, P13, P17||5|
|Future reflections||P7, P12, P13, P14, P21||5|
|No findings were found in learning dairies||P5 [b]||1|
|[a] Frequency value of the relevant characteristic.
[b] Since the P5 coded prospective teacher did not attend class regularly, no findings could be obtained.
The difficulty of the design process. As can be seen in Table 6, a large majority of the prospective science teachers faced difficulties in the design process. In the following example, the need to be patient and the difficulties experienced when completing the stages were highlighted by a prospective science teacher:
We are in the fifth and sixth stages of the Rube Goldberg machine. Our work this week was even harder than in the other weeks. This is because the stages are really getting harder and harder. Sometimes it takes 6 hours of work in one day. Even if the pleasure in doing this job starts to diminish a little, we are trying very hard to make the device ready. Every once in a while, my friends and I end our work by saying that we should be more patient. (7.3.17- learning dairy- P1)
Another participant said that Rube Goldberg machines became more difficult to make as the stages increased, and she/he was thinking more about what a new stage would be. The following excerpts from the interview reflect this tendency:
We thought too much about what additional steps we could add this week. As the stages increase, maintaining a balance becomes more difficult and challenging. We have to maintain a balance, maintain continuity in sequence, add stages, and try again, especially when shooting a video. Constantly repeating the steps requires patience. If one stage works, the other does not. It is a design process that requires patience and hard work, but we enjoy it. The design process is very difficult for us, but the feeling of victory after we have done everything is very good. (25.4.17- learning dairy- P2)
Happiness after completing each stage of the design. Some of the prospective science teachers were happy when each stage of the design had been completed. They also expressed their satisfaction with the appreciation of the product by their peers, and they felt that they were happy when the design was finished, no matter how difficult the design had been. Moreover, they stated that even developing a concrete design made them happy. Example statements include the following:
We brought our design to school this week. It was very exhausting and troublesome. Despite a lot of mishaps, it was still a successful presentation. But there were setbacks. The fan was broken, the balloon did not blow up, the steps were missing. But our friends looked at our presentation with admiration. It was a proud moment for me. (18.4.17- learning dairy- P2)
We completed the first two phases of the design of the Rube Goldber machine. I was exchanging ideas with my friends while preparing the exercise. We first designed our setup with our group of friends and then we plotted. The anxiety continued as to whether it would be on the one hand when doing these things. Now it’s time for something to happen. We were successful. Of course, after a few tests, we sat down to celebrate our achievement and drink some tea. But when the result was so good we worked harder than ever before. We committed ourselves to this. (21.2.17- learning dairy- P1)
The psychology of uncertainty in the design process. Some of the prospective science teachers particularly mentioned that were annoyed during the design process because the stages were not working as intended, and they that indicated that the designs became more complex. Examples of their comments follow:
Our instructor told us what we needed to do during the semester. At first, I was very frightened of this task, but I began to like it in the end. The designs were fun the first week. Later, the steps became complicated. The design needed some external intervention in some phases. In the process, my friend got very angry and occasionally disrupted the design. Of course, the design was later reorganized. When we were designing, we were very bored with some steps but we enjoyed some of the other steps. (23.5.17- learning dairy- P12)
Today is a very exciting day. I did a nice piece of homework with my friends. We were so panicky and stressed out that we did not know what to do. I made a cube. My friend made an oblique plane. Then the stones were knotted. Finally, we came to the merger stage. We could not do it anyway. It broke every time after I did it. We had a nervous breakdown. (7.3.17- learning dairy- P4)
Learning science concepts. Some of the prospective science teachers stated that Rube Golgberg machines were effective in learning science concepts, and they expressed the view that the design process provided a better understanding of science concepts such as inclined plane, friction force, push and pull forces, fixed pulleys, levers, equal arm scales, momentum, and torc. In addition, several prospective science teachers said that they had the opportunity to transfer science concepts into practice through using Rube Goldberg machines. Examples of the prospective science teachers’ opinions follow:
In this lesson, I learned what science concepts such as inclined plane, friction force, push and pull forces mean. This was because we always use these concepts in our designs (-07.03.17- learning dairy- P8)
I have learned concepts such as fixed pulleys, levers, inclined plane, equal arm scales, momentum, torc. I also learned better by applying these concepts that have been encountered in our daily lives. (28.02.17- learning dairy- P9)
In this week’s course, we started to find better ideas that we have improved further. We used equal arm scales for the next stage of our design. In this way we have used and continue to use many science concepts. (28.02.17- learning dairy- P11)
Discussions on the improvement of the design. Some of the prospective science teachers thought that their designs could be improved when they saw the other designs, which generated discussions on the improvement of their design. In this sense, the prospective science teachers especially emphasized the views expressed by the groups about the other designs during the design of the Rube Goldberg machines and pointed out suggestions for improvement with regard to the designs. The prospective teachers stated that they cared about the opinions of others with regard to their machines. Some of the prospective science teachers’ opinions follow:
Today, the first group brought the design they had created. Other groups brought their videos. Seeing other designs, I understood that our design was a bit simple. When we watched the videos of the other groups, we understood better. We understood that we needed to improve the design. We got opinions from other friends. Although it was the first session, it was quite fruitful for us. (21.2.17- learning dairy- P18)
When designing, we prepared our draft first and discussed how things would be done on the materials. We calculated our materials and placed them on the plate. We tried several times to see if it worked. In the first week, the first group of friends brought their design. We discussed their design. We opened our video to show our own design. We got the views of our classmates. (21.2.17- learning dairy- P11)
To have fun in the design process. Some of the preservice science teachers stated that they had a lot of fun in the design process with regard to Rube Goldberg machines, despite all the difficulties. Also, few prospective science teachers stated that they were using too many trial-and-error approaches, and they were very happy when the designs worked. Examples of the prospective science teachers’ opinions follow:
Anyway, the process of making, testing and correcting the design was included and, finally, the design was completed. But we’re done. We had a lot of fun and we were very happy, even though we had a lot of problems. So we had a nice day. (7.3.17- learning dairy- P4)
It’s fun to do the Rube Goldberg stages, but it’s a job that needs attention. If one stage does not work, the other stages do not achieve their goal and the result is frustrating. It is work that requires patience and hard work because it is done by going over it again and again. (21.2.17- learning dairy- P2)
Before we did our setup this week, we decided what materials we needed at this stage…. Magnets stuck in the cup hit the car which will move away from each other and the glass on the other side would fall down. We had a lot of fun doing this design. When the design was not working we tried it over and over again. (7.3.17- learning dairy- P11)
Future reflections. A prospective science teacher said that she/he will use Rube Goldberg machines when she/he works as a teacher. Another prospective teacher learned that entrepreneurship, innovation, and creativity skills of the students can be developed with engineering design. Generally, some of the prospective science teachers had become aware that design processes should be included in middle school science courses, and they clearly stated that they would include these designs in the future.
We are very pleased with our design that we have completed with determination, patience and effort. Through this lesson, I understood how to include entrepreneurship, creativity, innovative thinking and classroom practices in engineering. I am proud to think that we have developed a very successful design. (6.6.17- learning dairy- P14)
When I got angry, I broke the design we made, then I rebuilt it. But I never gave up because it was not good. I believed that I could accomplish when I did not give up. I think this design process will be very useful in our professional lives. That’s why the way to teach simple machine-related concepts is different. I saw that the engineering design process can be transferred to the students in a fun way. I intend to introduce this lesson to my students. (6.6.17- learning dairy- P13)
When the qualitative findings were generally examined, it was seen that the prospective science teachers had fun despite the difficulties in the design process, they experienced uncertainty in the design process, they learned some science concepts, they discussed the stages, and they felt that the designs had a positive personal effect.
In this study, it was found that the process of designing Rube Goldberg machines led to different reflections on the part of those prospective science teachers who participated in the research. The results with regard to the quantitative findings were examined, and it was found that Rube Goldberg machines have a positive effect on the STEM awareness of prospective science teachers. Since the design process is entirely design oriented, the prospective science teachers first dreamt, than drew up, and finally, tried to create their designs.
In fact, prospective science teachers are familiar with creating designs. In teacher education, prospective science teachers are continuously given design tasks, such as designing cell models, simple machines, and models of the universe. Moreover, these designs are effective in terms of embodying science concepts.
Rube Goldberg machines have been effective in embodying prospective science teachers’ concepts related to simple machines. Thus, design and engineering-focused events, such as the design and creation of Rube Goldberg machines, seem to be particularly effective in creating an awareness of STEM. Rube Goldberg machines, which are derived from STEM disciplines, can help middle-school students become more interested in these subjects (Shanker, 2017). Chain reaction machines, such as Rube Goldberg machines, offer children the opportunity to personally design meaningful spaces without actively using intelligent technologies (Sheriff et al., 2017). In this sense, Pierson and Suchora (2002) pointed out that, although the aim of Rube Goldberg machines and the goal of the engineer’s designs do seem different, it is possible for them to come together as a design project. The findings supporting this claim can be seen in the literature.
The mean pretest score of the students was 66.65, a value that can be seen as being initially high. When looked at other research findings using the same measurement tool, it can be seen that their values are close. Hebebci and Usta (2017) found the average score of university students (first and second grades) to be 62.12 in a survey conducted using the same measurement tool at a different university. On the other hand, Buyruk and Korkmaz (2017b) found the mean score of prospective science teachers to be 79.10 in a survey study conducted using the same measurement tool.
The literature indicates that the mean scores of prospective science teachers in terms of STEM awareness are high (Buyruk & Korkmaz, 2017b). For example, Yenilmez and Balbağ (2016) revealed that the attitudes of prospective science teachers toward STEM are positive, and Marulcu and Sungur (2012) showed that some of prospective science teachers thought that engineering was important for science education.
Kızılay (2016) found that prospective science teachers had positive opinions about STEM education. In the current study, it can be said that the mean values found in the pretest scores can be considered to be normal. What is important here is the size of the increase caused by the intervention process, which is statistically significant. It may be the case that lower average values might be obtained if researchers work with prospective teachers from different branches (primary, math, music, art, etc.). Depending on these results, it can be said that prospective science teachers generally approach the engineering design process positively. This result show that Rube Goldberg machines are capable of creating awareness with regard to STEM.
The quantitative research findings can be said to be supported by the qualitative findings. Throughout the design process, some of prospective science teachers were happy after the design had been completed, and they had exchanged ideas to improve their design. These processes showed that there was an awareness of engineering design on the part of some of prospective science teachers.
On the other hand, one of the most striking findings in the qualitative research findings were the difficulties experienced in the design process with regard to the Rube Goldberg machines. These findings are compatible with those in the literature. For example, Jordan and Pereira (2009) expressed the view that students faced difficulties such as working in teams, putting the design into practice, focusing on the design, and managing their own programs during the design of the Rube Goldberg machines. Similarly, Mahinroosta and Lindsay (2016) stated that students were also challenged in terms of time management, risk management, team leadership, and participation in the process of creating the Rube Goldberg machines.
In the current research, some of prospective science teachers stated that they faced difficulties in designing the Rube Goldberg machines, although the same prospective science teachers also said that they were very happy when the design was completed. In fact, although people encounter many difficulties and problems in daily life, these difficulties and problems are important in terms of creating awareness. One of the components that affects prospective science teachers’ STEM awareness may be the difficulties t experience in the design process. As a result, Rube Goldberg machines can be said to be a motivating feature when designs emerge.
In the design of the Rube Goldberg machines, it was seen that some of prospective science teachers fell into periods of uncertainty when the design step did not happen as expected. In fact, these findings are quite normal. This is because Rube Goldberg machines are extremely “brittle,” and if a step in the design does not work properly, the whole design would break down (Boudry & Pigliucci, 2013). In the current study, the prospective science teachers were constantly confronted with these problems, and they had uncertainty about what they would do. This uncertainty forced them to solve the problems. Therefore, the uncertainties experienced by some of the prospective science teachers may have created a STEM awareness about how they perform design.
Another point that emerges is that all of the prospective science teachers have been constantly engaged in discussions with regard to their Rube Goldberg machines. The prospective science teachers debated what was required of a new stage and what could be done about failing design stages. In this way, the prospective science teachers constantly exchanged ideas about how to design. These processes can also be important for awareness-raising with regard to STEM.
In addition, some of the prospective science teachers reflected in their learning diaries that Rube Goldberg machines helped them to learn science concepts, although I did not aim to determine whether or not Rube Goldberg machines are effective when it comes to teaching science concepts. These science concepts are simple machines such as inclined plane, friction force, push and pull forces, fixed pulleys, lever, equal arm scales, momentum, and torc. These findings show that Rube Goldberg machines can be effective in teaching science concepts. Thus, some prospective science teachers intended to use the Rube Goldberg machines for teaching science concepts in their professional lives.
Finally, the Rube Goldberg machines had a positive influence on the STEM awareness of the prospective science teachers. Rube Goldberg machines offer many opportunities for students in terms of STEM awareness. For instance, Rube Goldberg machines allow students to create a design, to explore science concepts, and to establish links between engineering and science. The prospective science teachers expressed the view that they repeatedly experimented as part of the process of adding a simple step to their Rube Goldberg machines.
Moreover, the prospective science teachers constantly talked about simple machines, and they used such machines in their designs. It can be said that the awareness of engineering in science education has begun to emerge. As a result, Rube Goldberg machines have an important role to play in terms of increasing the STEM awareness of the prospective teachers who participated in the research.
Two main recommendations are made based on the notable results found in the research. These suggestions are directed to the teaching of engineering design and science concepts. First, in this study, it was determined that Rube Goldberg machines led to positive reflections on the part of prospective science teachers. In this sense, it can be seen that Rube Goldberg machines can be used to increase the STEM awareness of prospective science teachers who share similar characteristics. Thus, Rube Goldberg machines seem to be a good option when it comes to teaching methods courses in engineering design for.
Second, the prospective science teachers said that Rube Goldberg machines are particularly effective in learning scientific concepts related to simple machines, although this was not among the purposes of this research. Future research could be conducted to examine the effectiveness of Rube Goldberg machines in learning basic science concepts related to simple machines. Moreover, Rube Goldberg machines could be used as a method of teaching that prospective science teachers could use in the future courses they teach.
Acharya, S., & Sirinterlikci, A. (2010). Introducing engineering design through an intelligent Rube Goldberg implementation. Journal of Technology Studies, 36(2), 63-72. doi: 10.21061/jots.v36i2.a.7
Akgündüz, D., Aydeniz, M., Çakmakçı, G., Çavaş, B., Çorlu, M.S., Öner, T., & Özdemir, S. (2015). STEM education Turkey report / STEM eğitimi türkiye raporu. (Devrim Akgündüz & Hamide Ertepınarm, Eds.,). İstanbul: Scala Edition.
Alaszewski, A. (2006). Using diaries for social research. New Delhi, India: SAGE Publications Inc. doi: 10.4135/9780857020215
Altan, E.B., Yamak, H., & Kırıkkaya, E.B. (2016). A proposal of the STEM education for teacher training: Design based science education. Trakya University Journal of Education, 2, 212-232.
Ambrose, D., & Sternberg, R.J. (2016). Creative intelligence in the 21st century: Grappling with enormous problems and huge opportunities. Rotterdam, NE: Sense Publisher. doi: 10.1007/978-94-6300-506-7
Ary, D., Jacobs, L.C., Sorensen, C., & Razavieh. A. (2010). Introduction to research in education. Cengage Learning, 2013. Introduction to research in education (8th ed.). Boston, MA: Wadsworth Cengage Learning.
Aslan-Tutak, F., Akaygun, S., & Tezsezen, S. (2017). Collaboratively learning to teach STEM: Change in participating pre-service teachers’ awareness of STEM. Hacettepe University Journal of Education, 32(4), 794-816.
Bahar, A., & Adıguzel, T. (2016). Analysis of factors influencing interest in STEM career: Comparison between American and Turkish high school students with high ability. Journal of STEM Education: Innovations and Research, 17(3), 64-69.
Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House, 83(2), 39–43. doi: 10.1080/00098650903505415
Bequette, J. W., & Bequette, M. B. (2012). A place for art and design education in the STEM conversation. Art Education, 65(2), 40-47. doi: 10.1080/00043125.2012.11519167.
Berg, D.R. (2015, June). Use of a Rube Goldberg design project for engineering dynamics. In Proceedings of the 122nd ASEE Annual Conference Expo, Seattle, WA. doi: 10.18260/p.24976
Boudry, M., & Pigliucci, M. (2013). The mismeasure of machine: Synthetic biology and the trouble with engineering metaphors. Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences, 44(4), 660-668. doi: 10.1016/j.shpsc.2013.05.013
Brush, J.L. (2017). Forces and motion (Rube Goldberg PBL), Understanding by design: Complete collection, 388. Retrieved From http://digitalcommons.trinity.edu/educ understandings /388
Buyruk, B., & Korkmaz, Ö. (2016a). STEM awareness scale (SAS): Validity and reliability study. Part B: Türk Fen Eğitimi Dergisi, 13(2), 61-76.
Buyruk, B., & Korkmaz, Ö. (2016b). Teacher Candidates’ STEM Awareness Levels. Participatory Educational Research (PER), Special Issue 2016-III, 272-279.
Bybee, R.W. (2010). What is STEM education? Science, 329(5995), 996. doi: 10.1126/science.1194998
Bytheway, B. (2012). The use of diaries in qualitative longitudinal research purposively utilizing a criterion sampling. Timescapes methods guides series 2012, Guide No. 7, ISSN 2049-9248.
Cantrell, P., Pekcan, G., Itani, A., & Velasquez-Bryant, N. (2006). The effects of engineering modules on student learning in middle school science classrooms. Journal of Engineering Education, 95(4), 301-309. doi: 10.1002/j.2168-9830.2006.tb00905.x
Çevik, M. (2017). A study of STEM Awareness Scale development for high school teachers. Journal of Human Sciences, 14(3), 2436-2452. doi: 10.14687/jhs.v14i3.4673
Çolakoğlu, M.H., & Gökben, A.G. (2017). STEM studies in Turkish faculties of education. Journal of Research in Informal Environments, 2(2), 46-69.
Creswell, J.W., & Clark, V.L.P. (2017). Designing and conducting mixed methods research.
Thousand Oaks, CA: Sage publications.
Davis, J.L., Chlebowski, A.L., & Ellert, D.J. (2017, June). Work in progress: Oh… The irony (A six-section Rube Goldberg machine for freshman engineering design). 2017 ASEE Annual Conference & Exposition. doi: 10.18260/1-2–29173
DeMontigny, D., & Smithson, H. (2010, July). The Rube Goldberg experiment: Year 2. Proceedings of the Canadian Engineering Education Association. doi: 10.24908/pceea.v0i0.3153
DeMontigny, D., Smithson, H., & Wright, C. (2011). Teaching engineering design and communication in first year using Rube Goldberg projects. Proceedings of the Canadian Engineering Education Association. doi: 10.24908/pceea.v0i0.3684
Department of Education (2006). Massachusetts Science and Technology/ engineering curriculum framework. Malden: MA: Massachusetts Department of Education.
Eron, M.B., & Rachlin, S.L. (Eds.). (2015). Middle math: Improving the undergraduate preparation of teachers of middle grades mathemathics. San Diego, CA: Association of Matematics Teacher educators.
European Commission. (2015). Science education for responsible citizenship. Report to the european commıssıon of the expert group on scıence education. Luxembourg, GE: Publications Office of the European Union. doi: 10.2777/12626
Ganesh, T.G., & Thieken, J. (2010). Designing and implementing chain reactions: A study of seventh-grade students’ knowledge of electrical circuits. Washington, DC: American Society for Engineering Education.
George, D., & Mallery, P. (2003). SPSS for windows step by step: A simple guide and reference (4th ed.). Boston, MA: Allyn & Bacon.
Graneheim, U.H., & Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105-112. doi: 10.1016/j.nedt.2003.10.001
Greene, J., Caracelli, V., & Graham, W. (1989). Towards a conceptual framework for mixed method evaluation designs. Education Evaluation and Policy Analysis, 11(3), 255–274. doi: 10.2307/1163620
Hebebci, M.T. & Usta, E. (2017, May). Üniversite öğrencilerinin FeTeMM farkındalık durumlarının incelenmesi. Türk Bilgisayar ve Matematik Eğitimi Sempozyumu-3, Afyonkarahisar.
Howard, W., Williams, R., & Yao, J. (2010). Simulations of carnival rides and Rube Goldberg machines for the visualization of concepts of statics and dynamics. Engineering Design Graphics Journal, 74(2), 1-11.
Jaakma, K., Kiviluoma, P., & Kuosmanen, P. (2015). CAD and CAE in mechanical engineering education at aalto university. faculty of power engineering. Estonia: Tallinn University of Technology.
Jordan, S., & Pereira, N. (2009). Rube Goldberg engineering: Lessons in teaching engineering design to future engineers (School of Engineering Education Graduate Student Series. Paper 5). Retrieved from http://docs.lib.purdue.edu/enegs/5
Kim, Y., & Park, N. (2012). Development and application of STEAM teaching model based on the Rube Goldberg’s invention. In S. Yeo, Y. S. Lee, & H. B. Chang (Eds.), Computer science and its applications (pp. 693-698). The Netherlands: Springer. doi: 10.1007/978-94-007-5699-1_70
Kızılay, E. (2016). Pre-service science teachers’ opinions about STEM disciplines and education. The Journal of Academic Social Science Studies, 47, 403-417.
Kondracki, N. L., Wellman, N. S., & Amundson, D. R. (2002). Content analysis: review of methods and their applications in nutrition education. Journal of Nutrition Education and Behavior, 34(4), 224-230. doi: 10.1016/s1499-4046(06)60097-3
Kovarik, D.N., Patterson, D.G., Cohen, C., Sanders, E.A., Peterson, K.A., Porter, S.G., & Chowning J.T. (2013). Bioinformatics education in high school: Implications for promoting science, technology, engineering, and mathematics Careers. CBE-Life Sciences Education, 12(3), 441-459. doi: 10.1187/cbe.12-11-0193
Lei, C.U., So, H.K.H., Lam, E.Y., Wong, K.K.Y., Kwok, R.Y.K., & Chan, C.K. (2012, August). Teaching introductory electrical engineering: Project-based learning experience. Paper presented at the Teaching, Assessment and Learning for Engineering (TALE), 2012 IEEE International Conference on (pp. H1B-1). Hong Kong. doi: 10.1109/tale.2012.6360320
Mahinroosta, R., & Lindsay, E.D. (2016). Rube Goldberg Machines as a transition to university tool. In 27th annual conference of the Australasian Association for Engineering Education: AAEE 2016 (p. 484). Southern Cross University.
Marklin, B. (2018, February). How STEM inspired Rube Goldberg. Learning Liftoff. Retrieved from https://www.learningliftoff.com/how-stem-inspired-rube-goldberg/
Marulcu, I., & Sungur, K. (2012). Investigating pre-service science teachers’ perspectives on engineers, engineering and engineering design as context. Afyon Kocatepe Üniversitesi Fen Bilimleri Dergisi, 12, 13-23.
Matty, A.C. (2017). Large scale PBL activity develops core competencies in students (Unpublished master’s thesis). City University of Seattle, Washington.
Mayring, P. (2000). Qualitative content analysis [28 paragraphs]. Forum qualitative sozialforschung/Forum: Qualitative Social Research, 1(2), Art. 20, Retrieved from http://nbnresolving.de/urn:nbn:de:0114-fqs0002204.
Mielke, M., Grünewald, A., & Brück, R. (2016). Motivating students through problem-based learning and chip-fabrication in a microelectronics design laboratory. Proceedings of the Global Engineering Education Conference (EDUCON), 2016 IEEE (pp. 163-170). doi: 10.1109/educon.2016.7474548
Milaturrahmah, N., Mardiyana, I., & Pramudya, I. (2017). Science, technology, engineering, mathematics (STEM) as mathematics learning approach in 21st century. In AIP Conference Proceedings (Vol. 1868, No. 1, p. 050024). AIP Publishing. doi: 10.1063/1.4995151
The Ministry of Education. (2018). Primary and middle school science curriculum (3-8 grades). Ankara. Turkey: Head Council of Education and Morality.
National Research Council. (2005). National science education standards. Washington DC: National Academy Press.
National Research Council. (2009). Engineering in K-12 education: Understanding the status and improving the prospects. Washington, DC: The National Academies. doi: 10.17226/12635
National Research Council. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: The National Academies Press.
O’Connor, D. (2003). Application sharing in K-12 education: Teaching and learning with Rube Goldberg. TechTrends 47(5), 6-13. doi: 10.1007/bf02763198
O’Byrne, I., Radakovic, N., Hunter-Doniger, T., Fox, M., Kern, R., & Parnell, S. (2018). Designing spaces for creativity and divergent thinking: Pre-service teachers creating stop motion animation on tablets. International Journal of Education in Mathematics, Science and Technology, 6(2), 182-199. doi: 10.18404/ijemst.408942
Pekbay, C. (2017). Effects of science technology engineering and mathematics activities on middle school students (Unpublished doctoral dissertation). Institute of Educational Sciences, Hacettepe University, Ankara.
Pierson, H.M., & Suchora, D.H. (2002). The Rube Goldberg three-minute timer: A design based learning tool for engineering freshman. Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition.
Quigley, C.F., Herro, D., & Jamil, F.M. (2017). Developing a conceptual model of STEAM teaching practices. School Science and Mathematics, 117(1-2), 1-12. doi: 10. 1111/ssm.12201
Ruiz-Primo, M.A. (2004). Examining concept maps as an assessment tool. In A. J. Cañas, J. D. Novak, & F. M. González (Eds.), Concept maps: Theory, methodology, technology (pp. 555–562). Proceedings of the First International Conference on Concept.
Sandelowski, M. (1995) Sample size in qualitative research. Research in Nursing and Health, 18, 179-183. doi: 10.1002/nur.4770180211
Selvi, E., & Soto-Caban, S. (2016, June). Experiences in teaching writing unit design courses to engineering students with advanced Rube Goldberg projects. In Proceedings of the 2016 ASEE Annual Conference & Exposition, 26-29, New Orleans, LA. doi: 10.18260/p.26815
Shanker, V. (2017). Rube Goldberg projects spread STEM interest at Dos Palos middle school. MERCED SAN STAR, Retrieved from http://www.mercedsunstar.com/news/local/education/article142044959.html
Sheriff, A., Sadan, R., Keats, Y., & Zuckerman, O. (2017, June). From smart homes to smart kids: Design research for catakit. Proceedings of the 2017 Interaction Design and Children Conference, Stanford, CA.
Smith, J. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17(4), 510-550. doi: 10.1037/a0029312
Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks: Sage.
Tarkın-Çelikkıran, A., & Aydın-Günbatar, S. (2017). Investigation of pre-service chemistry teachers’ opinions about activities based on STEM approach. Yüzüncü Yıl Üniversitesi Eğitim Fakültesi Dergisi, 14(1), 1624-1656.
Yanik, P.M., Ferguson, C.W., Kaul, S., & Yan, Y.A. (2017, June). Pilot program in opem-ended problem solving and project management. T. B. Natascha & J. P. James (Eds.), ASEE Annual Conference & Exposition. doi: 10.18260/1-2–27494
Yenilmez, Z., & Balbağ, M.Z. (2016). The STEM attitudes of prospective science and middle school mathematics teachers. Journal of Research in Education and Teaching 5(4), 301-307.
Yıldırım, B., & Altun, Y. (2015). Investigating the effect of STEM education and engineering applications on science laboratory lectures. El-Cezeri Journal of Science and Engineering, 2(2), 28-40.