Flipped learning has grown in popularity as an instructional practice over the past decade. Studies have suggested that flipped learning provides potential opportunities to improve students’ mathematical achievement (e.g., Dove & Dove, 2015a), as well as students’ attitudes toward mathematics (McGivney-Burrelle & Xue, 2013). However, the research is limited comparing various forms of flipped learning and ways they may influence students’ mathematical anxieties. This study aims to examine how differences in flipped instruction within a mathematics course may influence elementary preservice teachers’ math anxieties and perceptions of learning.

## The Influence of Mathematics Anxiety on Elementary Educators

The existence of mathematics anxiety has been well documented, with research examining its root causes and methods for mitigating its impact on the lives of students. A recent study estimated that over 25% of 4-year college students suffer from moderate to high levels of mathematics anxiety (Beilock & Willingham, 2014). Elementary preservice teachers are no exception, although mathematics anxiety among this group is particularly worrisome since many will go on to teach mathematics to young children (Bursal & Paznokas, 2006; Hembree, 1990; Kelly & Tomhave, 1985).

High levels of mathematics anxiety and low levels of mathematics teaching efficacy can inhibit preservice teachers from achieving higher levels of mathematics knowledge for teaching, thus limiting their preparation for teaching elementary mathematics (Aslan, 2013; Bursal & Paznokas, 2006; Gresham, 2008). As these teachers enter the classroom, their high anxiety and low teaching efficacy can have a far-reaching impact on their students. For example, female teachers with high levels of mathematics anxiety can negatively impact elementary girls’ mathematics achievement and attitudes toward mathematics (Beilock, Gunderson, Ramirez, & Levine, 2010).

Negative experiences created in the mathematics classroom also increase the likelihood that students will develop mathematics anxiety, thus creating a new cycle of highly anxious elementary teachers with low mathematics teaching efficacy (Bekdemir, 2010). As Ball, Lubienski, and Mewborn (2001) suggested, this leads “the school mathematics experience of most Americans [to be] uninspiring at best, and intellectually and emotionally crushing at worst” (p. 434).

### Decreasing Students’ Mathematics Anxiety

With the negative impact that mathematics anxiety can have on an individual’s ability to teach elementary mathematics effectively, addressing this anxiety is of paramount importance for teacher preparation programs. One effective approach in decreasing preservice teachers’ mathematics anxiety and improving their confidence has been the integration of mathematics methods courses into the elementary preservice teachers’ curriculum (Huinker & Madison, 1997; Tooke & Lindstrom, 1998). These courses typically focus on creating environments that are student centered and emphasize conceptual understanding, often utilizing mathematical manipulatives (Sloan, 2010), as well as placing an emphasis on sharing multiple processes and methods to help decrease anxiety (Furner & Berman, 2003).

Engaging in active learning processes in methods courses is helpful for preservice elementary teachers, but they also need to experience such practices in mathematics content courses. Research indicates that mathematics classes that utilize student-centered instructional practices improve achievement (e.g., Borko, Stecher, Alonzo, Moncure, & McClam, 2005; Lawson, Benford, Bloom, & Carlson, 2002), student engagement (e.g., Peterson & Miller, 2004; Shernoff, Csikszentmihalyi, Schneider, & Shernoff, 2003), and students’ attitudes and dispositions about mathematics (e.g., Zakaria, Chin, & Daud, 2010).

Unfortunately, the integration of high levels of student-centered, inquiry-based learning into mathematics classes has been a tenuous process at best. K-12 and university instructors are often pressed for time and see direct instruction as the most efficient method for covering material (Hannafin, Burruss, & Little, 2001; Kor & Lim, 2009). In addition, many parents (Sam & Ernest, 2000; Ujifusa, 2014), as well as mathematics instructors (Stipek, Givvin, Salmon, & MacGyvers, 2001), have strong feelings about how math should be taught, which can lead to resistance when implementing alternative forms of instruction in the mathematics classroom.

### Flipped Learning’s Potential Influence

Flipped learning is an instructional practice that encourages the use of more active learning practices in the mathematics classroom. As access to online video becomes more readily available via websites like YouTube, flipped learning has become a more viable instructional strategy.

Bishop and Verleger (2013) defined flipped learning as “an educational technique that consists of two parts: interactive group learning activities inside the classroom, and direct computer-based individual instruction outside the classroom” (p. 6). While the opportunity to provide some form of direct instruction can placate instructors’ concerns regarding content coverage, moving in-class lectures offline offers teachers the opportunity to create more flexible learning environments, thus shifting the learning process toward more student-centered approaches (Hamdan, McKnight, McKnight, & Arfstrom, 2013).

Recent research suggests that flipped learning can assist with the creation of more student-centered, active-learning environments, providing more opportunities for students to collaborate (Dove, 2014; Johnson, 2013). Additionally, studies suggest flipped learning may help improve students’ self-efficacy (McLaughlin et al., 2013), engagement (Butt, 2014), and achievement (Dove & Dove, 2015a; Fulton, 2012).

The motivation for implementing flipped learning is not the removal of lectures per se, but rather the addition of more student-centered opportunities during class, which are the hallmark of methods courses. Specifically, Wallace, Walker, Braseby, and Sweet (2014) suggested that flipped learning provides instructors with better opportunities to implement constructivist teaching approaches that allow students to tackle more challenging problems while constructing stronger understandings of the concepts being learned.

Foldness (2016) found that when flipped classes did not incorporate cooperative learning opportunities, students’ academic success was no better than in lecture-based classrooms. By emphasizing more student-centered learning approaches, flipped learning may help decrease mathematics anxiety in mathematics content courses in a manner similar to that observed in methods courses. To examine this possibility specifically, our study considered the following questions:

- What commonalities and differences in instructional practices occur between a flipped classroom using teacher created videos, a flipped classroom using third-party videos, and a nonflipped class?
- What impact do different instructional approaches have on students’ mathematics anxiety and anxiety about teaching mathematics?
- What are the students’ perceptions of learning mathematics within different instructional approaches?

## Methods

This study was conducted with three sections of a mathematics content course for elementary education preservice teachers at a midsized public university. This course was the first mathematics course of a three-course series required by elementary education students. The course emphasized fundamental concepts in number and operations, algebra, and data analysis. Due to the complexity of scheduling, students chose their section. However, no prior knowledge related to the instructional methods would be used within a given section; only course time, location, and the instructor’s name were provided during course registration.

Additionally, first author Anthony Dove taught all three classes.** ** He was chosen based on his experience teaching the course and his experience with flipped learning. Using the same instructor also ensured course requirements, expectations, and assessments were identical across all sections. To limit researcher bias, Dr. Dove did not have access to any of the research data until after the completion of the course.

### Classroom Settings

The instructor kept the three sections as similar as possible to limit potential confounding variables. Each class met for 50 minutes, 3 days per week. The three sections of the course were taught back to back on the same days in the morning (9 a.m., 10 a.m., and 11 a.m.) in the same classroom. The classroom included a SMART Podium, two large projectors in the front of the room that projected above a full-length whiteboard, and mathematical manipulatives. All three courses used the same textbook, *Mathematics for Elementary Teachers* (4th ed.; Beckmann, 2014).

All course sections were assigned the same homework problems from the textbook or instructor-created problems, took the same assessments and final exams, and completed the same two long-term projects. All course materials were provided to the students using the university learning management system, Desire2Learn (D2L)*.*

The primary difference between the three sections was the manner in which the fundamental content was delivered. This deliver, in turn, influenced the learning opportunities that occurred during class time. For the duration of the semester, each section was taught using a different instructional practice: the Teacher In-Class Lecture method, the Teacher Flipped method, and the Khan Academy Flipped method.

### The Teacher In-Class Lecture Method

The Teacher In-Class Lecture (TL) method served as the control group since it closely resembled the way the mathematics course would typically be taught at the university. In this section, multiple instructional methods were often employed during class, including direct instruction, collaborative learning, and inquiry-based activities. Students were asked to read upcoming sections from the textbook, but no formal preparation was expected prior to class.

Class typically began with a warm-up to review the previous in-class lesson while the instructor checked homework and was followed by a homework review. Next, the instructor typically provided a 10- to 15-minute electronic slideshow accompanied lecture related to the content from one to two sections of the textbook. The electronic slideshow was created by the instructor and was made available for students to download from D2L prior to class, thus eliminating the need for students to copy notes from slides.

During the direct instruction, the instructor would ask questions to engage the students. Students were also encouraged to ask questions during the lecture. The remaining class time was spent completing practice problems and activities that provided depth and conceptual understanding of the mathematical concept of the lesson. Students were assigned homework problems to help them continue building their understanding of the concept. If the problems were created by the instructor, they would be made available on D2L.

### The Teacher Flipped Method

In comparison to the TL, the Teacher Flipped Class (TF) method required classroom preparation both by the instructor and the students. Prior to the beginning of the semester, the instructor created a lecture video for each major mathematical concept to be covered in the class. The videos were created to align with the sections of the textbook. Videos were recorded using Screencast-O-Matic (http://screencast-o-matic.com) and PowerPoint presentation software. The PowerPoint presentation used to create videos was often an abbreviated version of the one used during class for the TL. All videos were uploaded to the instructor’s YouTube account and sorted into playlists that matched the textbook chapters (https://goo.gl/1f0I6Y). In total, 42 videos were created, with the average video lasting 4:31 (*SD* = 1:21).

Prior to class, TF students were required to watch one to two lecture videos and bring notes to the upcoming class. The notes were checked as part of a homework requirement. Class began similarly to the TL. Students completed a warm-up review while the instructor checked homework and notes. Afterwards, students could ask questions about the homework or concepts from the lecture video. Once the review was completed, students worked with a partner or partners to complete practice problems or activities. The instructor walked around the room listening to group discussions, providing one-on-one/small group assistance as needed. Whole group discussions were utilized to share strategies and to discuss any questions that may have arisen from a given problem or activity.

The emphasis of the instructional methods with the TF was to limit the amount of direct instruction that occurred in class and, instead, increase active learning opportunities by focusing on activities and group discussions to allow students to build their own conceptual understanding.

For example, when learning about different interpretations of division, the TF students explored the different interpretations using physical manipulatives, examined children’s work to determine their understanding and misconceptions related to the interpretations, and watched and discussed multiple video clips of children completing tasks related to the different interpretations. In comparison, TL students examined only one example of student work and watched one video related to interpretations of division due to time needed to complete the in-class lecture. The TF students also completed the same assigned homework problems as the TL. * *In addition, the TF students were required to watch the lecture videos for the upcoming class and take notes.

### The Khan Academy Flipped Method

Preparation for the Khan Academy Flipped (KF) method was similar to the TF method. Khan Academy (www.khanacademy.org) is an online website that provides lecture videos covering early childhood to college mathematics topics. Khan Academy was chosen because of its growing national popularity and its breadth of topics. It also provided a consistent and standardized lecture format compared to finding different videos by different authors from sites like YouTube.

All students were required to join the instructor’s course on Khan Academy, which allowed the instructor to assign videos directly and determine who had watched them. The KF students were then assigned Khan Academy videos related to the course content. Because the topics did not align directly with the book, students would often be given two to three videos that together met the fundamental requirements for the upcoming class. Instead of turning in notes like the TF students did, the KF students completed challenge questions that were built into Khan Academy topics. To make sure students acquired basic knowledge from the videos, they were required to answer 80% of items within the built-in challenge questions correctly to receive homework credit for the assigned videos.

The time during class was set up the same as the TF method, thus instructional methods were comparable. Class began with a warm-up review related to the previous in-class lesson, followed by homework review and a question/answer session from the lecture videos. The rest of the class time was spent with students working with a partner or in groups to complete various practice problems and extension activities directly related to the lecture videos from the previous night. These problems were identical to those used in the TF method. The KF students were assigned the same homework problems as the TF and TL students. In addition, they watched the necessary Khan Academy videos and completed the corresponding challenge questions for the upcoming class.

### Data Collection and Analysis

Throughout the course of the semester, multiple forms of data were collected so we could examine classroom instruction, students’ mathematics anxieties, and students’ perceptions of teaching and learning within their given course. To compare classroom instruction, each section was video recorded twice. The recordings were analyzed by two observers using the Reform Teaching Observation Protocol (RTOP; Piburn et al., 2000).

Students took the mathematics anxieties precourse online survey during the second week of the course. The survey collected basic background information about mathematics coursework and academic standing. It also included two surveys, the Math Anxiety Rating Scale – Revised (MARS-R; Hopko, 2003) and the Anxiety About Teaching Mathematics Scale (ATMS; Hadley & Doward, 2011). An example of the precourse survey can be found at https://goo.gl/lihusV.

During the final week of the course, students took the mathematics anxieties postcourse online survey to examine how their mathematics anxieties had changed over the course of the semester, again including the MARS-R and ATMS, as well as four open-ended questions and two Likert responses. An example of the postcourse survey can be found https://goo.gl/yDmOU3. Finally, a whole class semistructured interview was conducted with each class by a researcher other than the instructor to obtain data regarding the students’ perceptions of teaching and learning in the given class (see Appendix).

* Precourse Group Comparisons.* As part of the precourse survey, students were asked to describe the mathematics courses they had taken in high school or at college and their current standing at the university. Students in all three sections reported taking Algebra I, Algebra II, and Geometry in high school. Table 1 provides details on additional mathematics courses taken. Also, the TL course included 21 first-year students, 11 sophomores, five juniors, and two seniors. The TF course included 22 first-year students, 10 sophomores, four juniors, and two seniors. The KF course included 31 first-year students, three sophomores, one junior, and two seniors.

**Table 1**

Number of Student Reported Mathematics Courses Completed

Section |
Statistics |
Precalculus |
Calculus |
College Algebra |

Teacher Flipped (TF) | 10 (26%) | 14 (37%) | 3 (8%) | 3 (8%) |

Khan Academy Flipped (KF) | 15 (41%) | 20 (54%) | 1 (3%) | 2 (5%) |

Teacher In-Class Lecture (TL) | 11 (28%) | 13 (33%) | 5 (13%) | 1 (3%) |

Additionally, students were required to take a 25-question in-class simulated Praxis Core Math test during the second week of the course. The Praxis Core Math test is a national standardized test required for entrance into most education programs. A one-way analysis of variance (ANOVA) was conducted to determine any significant differences in mathematics background among the three groups. Results suggested no significant academic differences among the three classes, *F*(2, 114) = 0.52, *p* = 0.60.

* Classroom Analysis. *To determine commonalities and differences among the sections, two video recordings of each class were evaluated by two observers using the RTOP (Piburn et al., 2000). RTOP is an observation tool created from the National Science Foundation funded project, ACEPT, which compares levels of reform teaching that occur during an observed class. It is comprised of 25 Likert statements ranging from 0 (

*never occurred*) to 4 (

*very descriptive*) related to lesson design and implementation, content, and classroom culture. RTOP scores over 50 suggest that substantial active learning occurred during the observed class (MacIsaac & Falconer, 2002). Additionally, concepts related to high RTOP scores align with what are considered the four pillars of flipped learning (Hamdan et al., 2013).

Prior to performing any analyses with RTOP, the observers completed an online training together to make sure they shared an understanding of how to use the instrument (http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/index.htm). Afterward, the observers watched videos of the TL, TF, and KF classrooms together. For each recorded observation, each observer took notes and completed the RTOP individually. At the end of each recording, the observers compared ratings for each RTOP question and came to a consensus rating for each question, as well as for the overall RTOP score.

Interrater reliability between the two observers was 80% across all videos. To provide reasoning and justification for the RTOP scores, the observers collected data regarding the number of activities completed and time spent completing different instructional practices (e.g., review, direct instruction, and activities). Additionally, the observers constructed qualitative descriptions of teacher and student participation for the duration of the observation.

* Mathematics Anxieties Analysis. *Two forms of anxiety, general mathematics anxiety and anxiety about teaching mathematics, were analyzed to examine the influence of the different instructional practices. To analyze these anxieties, students completed the MARS-R (Hopko, 2003) and ATMS (Hadley & Doward, 2011) as part of the precourse survey during the second week of the course and the postcourse survey the last week of the course. These instruments provided data associated with each form of anxiety and were chosen for their brevity and reliability.

We considered using reliable short surveys to be advantageous to improve survey completion rates, especially with students who may have high mathematics anxiety. The MARS-R consists of 12 questions constructed through multiple factor analyses of mathematics anxiety surveys (Hopko, 2003). Similarly, the ATMS consists of 12 questions focused on teaching mathematics and was constructed based on Hopko’s MARS-R (Hadley & Doward, 2011).

To improve completion rates of the precourse and postcourse surveys, students were provided time during class to complete the surveys under the supervision of a researcher other than the instructor. In addition, students were assured that completion of the surveys was optional and would not be examined until the semester was completed. Pre- and postcourse surveys were paired for each student to measure changes in mathematics anxiety and anxiety about teaching mathematics (Table 2).

**Table 2
**Number of Complete Paired Samples for Each Survey

Section |
MARS-R |
ATMS |

Teacher Flipped | 32 (82%) | 29 (74%) |

Khan Academy Flipped | 29 (74%) | 27 (69%) |

Teacher In-Class Lecture | 20 (50%) | 22 (55%) |

Analyses were completed both within each section and across the three sections in our examination of each form of anxiety. For each section, a paired sample *t*-test was conducted for the MARS-R and the ATMS to determine if there were significant changes in students’ anxiety levels by the end of the semester. In addition, a one-way multivariate analysis of variance (MANOVA) was conducted to examine whether there were significant differences between the instructional practices and the changes in students’ mathematics anxieties.

* Students’ Perceptions Analysis. *Finally, students’ perceptions of learning mathematics were examined through multiple data sources. First, the postcourse survey included four open-ended questions. These questions asked students if the course had influenced their mathematics anxiety or their ability to teach elementary mathematics, suggestions for improving the course, and any additional comments about teaching and learning in the course. The postcourse survey also included two Likert questions asking students whether they would prefer to take future mathematics courses that used lecture videos and whether they would recommend such courses to friends. Due to the limited sample size, validity and reliability were not established for these questions.

Finally, noting that group dynamics can “be a stimulus to elaboration and expression” (Frey & Fontana, 1991, p. 184), a semistructured, whole class interview was conducted by a researcher other than the instructor to assist with triangulation of the data. While the interview protocol included seven questions that were asked to all three classes, the interviewer asked follow-up questions to better ascertain student perceptions of the course and of learning mathematics (Appendix A).

A case study analysis approach (Creswell, 2007) was used to examine student perceptions. Data from the postcourse survey and whole class semistructured interviews were examined by two researchers. Four common themes were considered based upon the questions asked and responses received both in the open-ended questions and class interview. These themes focused on the role of the instructor, the methods of the lecture, the in-class instructional methods, and students’ perceptions of their mathematics anxieties. Each set of data was examined three times for any statements related to each theme. These statements were further reviewed to determine commonalities and differences among students in the different groups.

## Results

### Classroom Comparisons

The three sections were examined to determine similarities and differences during in-class instruction. RTOP scores were relatively similar between the TF and KF classes, while the TL class was approximately 10 points lower (Table 3). Although the TL scores were substantially lower than either flipped section, the RTOP scores (MacIsaac & Falconer, 2002) still suggested high levels of reform instruction were observed in the TL classroom. Observations found that the instructor provided all three sections with opportunities during class meetings to participate in small group activities, discuss key concepts, and ask questions to peers and the instructor. This result suggests that the instructor successfully incorporated active learning processes in all sections when possible, which was one the instructor’s goals for this course.

**Table 3**

Consensus RTOP Score for Each Observation

Section |
Observation 1 |
Observation 2 |

Teacher Flipped | 81 | 87 |

Khan Academy Flipped | 80 | 89 |

In-Class Lecture | 73 | 74 |

Subscales of the RTOP were also examined (Table 4). Differences were prominent within the subcategory Classroom Culture, which suggests that a primary difference between the flipped sections and the lecture-based section was enhanced opportunities for student communication and collaboration. In both flipped sections, the instructor engaged students individually and in small groups, utilizing student comments to drive conversation about different problem solving methods. While these skills were also observed in the TL classroom, they occurred less frequently due to the 10 to 15-minute lecture at the beginning of each class meeting.

**Table 4
**RTOP Subscale Scores

Subscale |
Observation 1 |
Observation 2 |
||||

Subscale | TF | KF | TL | TF | KF | TL |

Lesson Design & Implementation | 14 | 15 | 14 | 16 | 17 | 11 |

Content: Propositional Knowledge | 16 | 15 | 15 | 19 | 18 | 19 |

Content: Procedural Knowledge | 13 | 15 | 14 | 16 | 16 | 15 |

Classroom Culture: Communicative Interactions | 17 | 17 | 13 | 16 | 18 | 15 |

Classroom Culture: Student/Teacher Relationships | 19 | 18 | 17 | 19 | 20 | 14 |

Additional analysis by the observers provided a more robust understanding of the structure of each class. Activities and student-teacher interactions in the TL section were formatted similarly to the flipped sections, with students collaborating in small groups. Across all sections the instructor facilitated student problem solving both individually and in small groups. Although the small group activities utilized with the TL method were identical to those used with the flipped sections, the time dedicated to direct instruction during class meetings resulted in the TL class completing one less small group task during each observation. This result equated to approximately 40 fewer tasks completed by the TL section over the semester compared to the flipped sections.

### Anxieties of Mathematics

A one-way MANOVA was conducted to determine if any differences existed among the three sections’ mean anxiety scores on the precourse MARS-R and ATMS. First, test assumptions were verified. A Pearson correlation found MARS-R and ATMS precourse scores to be moderately correlated, *r*(77) = 0.675, *p* < 0.001. Additionally, the Box’s M value of 4.597 was nonsignificant, *p* = 0.62, thus the covariance matrices between the groups were assumed to be equal. With assumptions verified, the one-way MANOVA found a nonsignificant multivariate main effect for the sections, *F*(4, 146) = 1.76, *p* = 0.14, *η*^{2 }= 0.05. This finding suggests no significant differences between the three groups’ mathematics anxiety or anxiety related to the teaching of mathematics at the beginning of the semester.

To examine changes in students’ general mathematics anxiety, paired sample *t*-tests were conducted to compare precourse and postcourse MARS-R scores within each section (Table 5). Results indicated that students’ levels of general mathematics anxiety significantly decreased in all three sections. In addition, paired sample *t*-tests were conducted to compare precourse and postcourse anxiety about teaching mathematics within each class using the ATMS scores (Table 6). Results indicated that students’ levels of anxiety of teaching mathematics significantly decreased in the TF and TL sections, but not the KF. These results suggest that the course may alleviate students’ general anxiety of mathematics regardless of instructional method used within the course; however, flipped learning with third-party videos may not be effective at alleviating anxieties about teaching mathematics.

**Table 5
**Pre/Post Course Scores on MARS-R

Section |
Precourse Scores |
Postcourse Scores |
t |
df |

Teacher Flipped | 36.6 (11.0) | 27.2 (8.8) | -4.7** | 28 |

Khan Academy Flipped | 30.4 (12.6) | 29.2 (11.8) | -0.74 | 26 |

In-Class Lecture | 36.6 (10.4) | 30.6 (11.3) | -2.3* | 21 |

Note: Standard deviations appear in parentheses.*p < 0.05, **p<0.01 |

** Table 6
**Pre/Post Course Scores on ATMS

Section |
Precourse Scores |
Postcourse Scores |
t |
df |

Teacher Flipped | 36.6 (11.0) | 27.2 (8.8) | -4.7** | 28 |

Khan Academy Flipped | 30.4 (12.6) | 29.2 (11.8) | -0.74 | 26 |

In-Class Lecture | 36.6 (10.4) | 30.6 (11.3) | -2.3* | 21 |

Note: Standard deviations appear in parentheses.*p < 0.05, **p < 0.01 |

We also sought to determine whether any of the instructional methods were more effective in alleviating anxieties. To examine changes in the anxieties between the three class sections, we conducted a one-way MANOVA. To create the mean change for each measure of anxiety, student data were paired. The precourse score was subtracted from the postcourse score, with a negative change score suggesting a decrease in anxiety on each survey (Table 7).

**Table 7**

Means and Standard Deviations for the Change in Anxiety Scores

Anxiety Measure |
Instructional Method |
Mean Change |
Standard Deviation |

MARS-R | Teacher Flipped | -12.0 | 1.5 |

Khan Academy Flipped | -3.6 | 1.5 | |

Teacher Lecture | -6.5 | 1.7 | |

ATMS | Teacher Flipped | -9.4 | 2.0 |

Khan Academy Flipped | -1.2 | 2.1 | |

Teacher Lecture | -6.0 | 2.3 |

Prior to conducting the one-way MANOVA on the change scores, test assumptions were verified. A Pearson correlation found MARS-R and ATMS scores to be moderately correlated, *r*(77) = 0.524, *p* < 0.001. Additionally, the Box’s M value of 4.934 was nonsignificant, *p* = 0.58; thus, the covariance matrices between the groups were assumed to be equal. With assumptions verified, the one-way MANOVA found a significant multivariate main effect for the instructional method used, *F*(4, 146) = 4.26, *p* = 0.003, *η*^{2 }= 0.10. The power to detect the effect was 0.92. This suggested there were significant differences between the three groups’ change scores on the MARS-R and ATMS.

To determine what these differences were, we conducted follow-up ANOVAs. Prior to conducting the follow-up ANOVAs, Levene’s Test was examined. Results found the assumption of homogeneity of variance was satisfied for both anxiety measures (Mathematics Anxiety, *p* > 0.05; and Anxiety About Teaching Mathematics, *p* > 0.05). Follow-up ANOVAs revealed significant differences for both Mathematics Anxiety, *F*(2, 77) = 8.2, *p* = 0.001, *η*^{2 }= 0.18, power = 0.95; and Anxiety About Teaching Mathematics, *F*(2, 77) = 4.13, *p* = 0.02, *η*^{2 }= 0.10, power = 0.71.

Tukey’s HSD post hoc tests found that the decrease in Mathematics Anxiety was significantly greater in the TF course than both the TL and KF sections, while t no significant difference was found between the TL and KF sections. Additionally, Tukey’s HSD post hoc tests found that the decrease in Anxiety About Teaching Mathematics was significantly greater in the TF than the KF section, while no significant difference was found between the TF and TL nor the TL and KF sections. These results suggest that participation in a flipped mathematics class using instructor’s videos was significantly better at decreasing students’ general math anxiety than were the other two methods and was significantly better at decreasing students’ anxiety about teaching mathematics than was a flipped mathematics class using third-party videos.

## Students’ Perceptions of Learning Mathematics

To examine students’ perceptions of learning mathematics, we analyzed the data around the themes of the role of the instructor, the in-class instructional methods, the methods used for the lecture, and students’ perceptions of their mathematics anxieties.

** The Influence of the Instructor.** Both on the postcourse survey and during the interview, students were overwhelmingly positive about the role of the instructor in their learning. While the TF and KF participants each had seven specific statements about the positive influence of the instructor in their survey questions, the TL survey responses included 15 statements. These statements included that the professor had a positive influence on learning by providing detailed and coherent explanations, assisting them with examining different perspectives, and pushing them to find multiple methods for solving problems. One student in the KF course included that the instructor “lowered my anxiety because of the way [he] teaches his lessons.”

Interview responses supported findings from the postcourse survey responses. In both the TF and KF interviews, four students discussed the instructor’s positive influence; seven students from the TL section made similar comments. For example, a student in the TL section explained, “He is just very personal. If you have an issue, he goes out of his way to help you out and make sure you understand.” Another student in the KF said, “He cares. I feel like when you mirror him with another professor, he’s gonna be the one to come and sit down with you like for hours to get you to learn a concept.”

One possible reason for more frequent positive responses in the TL section in both the postcourse survey and interview may be the differing role of the instructor in the TL class. In a more traditional class, the instructor is often viewed as the center of the learning and dispenser of knowledge; thus, having a strong mathematics teacher is arguably more important and more readily recognized by students. The TL section comments often mentioned the words “teach” and “explain.” In contrast, the TF and KF section comments more often included the word “help.” While the TF and KF section comments suggest an importance of the instructor, the limited statements may also be indicative to the shift of a more student-centered teaching approach throughout the entire class.

** The Influence of the Classroom Instructional Methods. **Instructional methods and the role of the instructor were closely related. Student responses both on the postcourse survey and in the interview supported the RTOP results, in that students in all three sections recognized the significant use of reform teaching practices during class time. In contrast to the role of the instructor, the TL survey responses mentioned class methods six times; the KF section mentioned methods 11 times; and the TF did so 14 times. Again, responses were overwhelming positive about the course format.

Responses on the survey mentioned that the course required them to create multiple methods for solving problems, examine student perspectives of learning, and work in collaborative group settings throughout the semester. One TF student responded, “I feel being in this class has given me some new techniques on how elementary students learn and how I can teach them.” A TL student stated, “Because we were required to explain how we solve everything, it definitely helped with understanding how to explain things and that I will need to put a lot of effort into that when I’m a teacher.”

When combined with student responses about the instructor, this change in response rate may suggest how flipped learning modifies the classroom experience. The flipped sections encouraged students to take responsibility for their learning, and the students positively reacted to the increased opportunities for structured small group activities.

The number of interview responses were comparable, as the groups were specifically asked about the in-class activities and role of technology during class. Interestingly, during such questions, the TL students discussed the in-class activities but did not mention the lecture aspect of the class. Students in all groups suggested that manipulatives and various technologies were a regular part of the classroom instruction.

For example, a KF student stated, “We talk about how kids learn…and he’d show us videos of different ages and different kids and how they, like, process things.” A TF student said, “We’re always on the computers or playing with blocks and stuff like that.” A TL student also explained, “There is a lot of group work,” suggesting that even with an in-class lecture, students perceived the course as one that fostered a collaborative learning environment.

** The Methods Used for Delivery of the Content. **The greatest difference among the three classes occurred within the responses related to the delivery of lecture material. Within the postcourse survey, 10 TF section statements mentioned how the flipped classroom had provided a positive learning experience. One student responded, “I believe watching lecture videos at home and working through practice problems as a class truly benefited my success in what I thought would be a very difficult class.” Only one respondent would rather have taken notes in class.

TF students expanded upon their positive attitudes toward the flipped approach during the interview, with 10 positive statements related to the impact of lecture videos. Students mentioned the advantage of pausing and rewinding the lecture as they were taking notes, the ability to review videos at any time, and the ability of the videos to prepare them for class. As one TF student stated,

You kinda do it on your own speed. You can pause the video….Your notes are more developed because you can pause it when you want and go back and look at it. If he lectured in class, you can’t really be like, “Wait, stop, go back.” It’s a lot easier to do it on your own.

While the TL students discussed the lecture very little on the postcourse survey or interview, the limited comments were positive. However, students’ positive responses did not focus on the delivery of the material, but instead on having access to the PowerPoint slideshow. While only one student mentioned the positive use of lectures with electronic slides in the post-course survey, four students discussed the lectures during the interview as part of the instructor’s integration of technology. For example, all were appreciative that the electronic slideshow files were posted prior to class for students to print or download on their computers.

One TL student stated, “There are a lot of visuals. He has a lot of PowerPoints, and it brings you step by step about what is happening. Instead of just a teacher saying ‘Oh this happens,’ it is shown on a PowerPoint.” This comment suggests that what the students appreciated was not the style of the lecture or the use of lecture during class, but rather the access to the material used for the lecture. The electronic slides provided an artifact that could be used as students completed other activities or reviewed for upcoming assessments, similarly to the way students in the TF section used their notes to complete such activities.

In contrast, the KF students were mixed in their attitudes toward the use of Khan Academy videos as their method of direct instruction. Within the postcourse survey, four students responded negatively towards the use of Khan Academy videos, with only one student responding positively. As one KF student wrote, “I would take out the Khan Academy videos*. *I really did not enjoy them.” The whole class interview provided a fuller understanding of the potential differences and issues that occurred with the use of the Khan Academy videos in comparison to teacher created videos.

In general, students were positive about the concept of using lecture videos to allow for more active learning in class. When asked by the interviewer which students liked the broader idea of the flipped class, over three fourths of the class raised their hands. As one KF student suggested, “It’s only a 50-minute class. You can only do so much in that time period.” Another student followed, “Whenever you go in your homework and don’t understand something, you can always go back to the videos.”

The general displeasure of the students related to Khan Academy was not in watching the video, but instead in the requirement of completing the Khan Academy assessments associated with the lecture videos. One student commented, “I don’t like it because if you don’t get five right in a row, it takes you hours and then you just give up.” The challenge question requirement, unlike the notes requirement for the TF, created an additional unintended consequence with the KF students. As the interviewer continued to probe the class about Khan Academy*, *one KF student explained, “You are supposed to watch the videos and then do activities with that video…but I know most people just do the activities. And if they need help with the activities, then they go to the videos.”

The interviewer asked for a show of hands to see who typically followed this method, and approximately two thirds of the class raised their hands. Another KF student said, “Well, I try [the activity] and if I don’t understand it, I will go back to the video. So I try and then go back and then try it again.”

The instructor did not anticipate this method for how students would use the Khan Academy videos. His expectation was that students would review the material and use the assessments as a method of confirming their learning from the videos. This hyper-focus on the challenge questions as an assessment could potentially suggest why the KF had smaller decreases in mathematics anxieties than the other two courses, as they felt they were continually being tested. Such test exhaustion may have reinforced their anxieties in learning mathematics.

** Students’ Mathematics Anxieties. **Student perceptions of their change in mathematics anxiety and anxiety to teach math was of significant interest. To better understand these perceptions, the postcourse survey included the open-ended question, “Do you feel that completion of this course has influenced any anxiety you have toward math? If so, how?” As suggested in Table 8, no student who responded to the question perceived an increase in their mathematics anxieties.

**Table 8
**Survey Responses to the Item, “Do you feel that completion of this course has influenced any anxiety you have toward math? If so, how?”

Section |
Positive Impact |
No Impact |
No Response |
Total Responses |

Teacher Flipped | 24 | 7 | 3 | 34 |

Khan Academy Flipped | 22 | 5 | 6 | 33 |

Teacher In-Class Lecture | 18 | 3 | 4 | 25 |

The results of all three groups were comparable, with an overwhelming portion of respondents believing participation in the course helped decrease their anxieties. We further examined the positive responses to determine how students felt their anxieties had been impacted. First, several students in each group stated only that their anxiety had lessened (TL = 3, TF = 9, KF = 4) but provided no other information; thus, we do not know why these students perceived lower anxiety levels.

For those who did provide a more detailed response, three subthemes emerged. The most common response from respondents related to an improvement in confidence and comfort with mathematics (TL = 6, TF = 7, KF = 7). For instance, a TF student stated, “This course has actually given me confidence when it comes to math because I always thought I was bad at it.”

A second theme suggested that the instructor and his course methods positively influenced students’ mathematics anxieties (TL = 7, TF = 3, KF = 5). However, while respondents from the KF and TL sections focused more on the instructor, the three TF respondents emphasized the combination of the teacher, the use of lecture videos, and the activities completed in class. For example, a KF student stated, “It has lowered my anxiety because of the way [the instructor] teaches his lessons.” In contrast, one TF student suggested,

I feel like the course has helped me overcome any type of anxiety I had toward math tests, because I feel my professor’s lecture videos and the activities in class are helpful when it came to studying and taking any assessments.

Several students also believed their decrease in anxiety could be attributed to newly acquired skills related to mathematics and the teaching of elementary mathematics (TL = 2, TF = 5, KF = 6). For instance, a TF student stated,

I feel this course has definitely helped me with my anxiety towards teaching math. I now know how to do the problems I will have to teach in the future, and I have also learned many different ways to solve and teach them.

** Students’ Desire to Take Flipped Courses. **Finally, the postcourse survey asked students to rate on a scale of 1 (

*Very Unlikely*) to 4 (

*Very Likely*) their desire to take another flipped course or recommend such a course to friend. To examine the influence of actual participation in a flipped class on these two questions, we conducted independent samples

*t*-tests between the KF and TF sections. Results found the TF Likert scale averages were significantly higher on both questions (Table 9). This finding suggests that TF students were more likely to take and recommend flipped mathematics courses than were KF students; thus, the majority of students were more satisfied with the learning experience that included teacher-created videos compared to third-party videos. These lower averages align with KF students’ frustration with the Khan Academy requirements. Possibly, different lecture video expectations for the third-party videos will influence these responses with future classes.

**Table 9
**Flipped Course Likert Scale Comparisons

Survey Item |
Khan Academy Flipped |
Teacher Flipped |
t |
df |

If offered, how likely would you be to take a class that uses lecture videos instead of lectures during class? | 3.0 (0.8) | 3.4 (0.8) | -2.3* | 65 |

If offered, how likely would you be to recommend to a peer to take a class that uses lecture videos? | 3.1 (0.8) | 3.5 (0.8) | -2.4* | 65 |

Note: Standard deviations appear in parentheses.*p < 0.05 |

## Discussion

The purpose of this study was to examine whether different instructional practices could positively influence students’ anxieties and perceptions about mathematics. Results from this study suggest that when examining the multiple aspects of teaching and learning for a mathematics content course for elementary education preservice teachers, flipped learning with teacher-created videos has the potential to help improve students’ anxieties and confidence in mathematics more than do instruction that incorporates in-class lectures or third-party videos. For example, while all three sections incorporated some form of active learning as part of the class, the two flipped sections were able to incorporate more opportunities for interaction and communication within and between students and the instructor. This result supports previous findings that flipped learning can provide additional opportunities for instruction that are engaging and focused on building student understanding through collaboration (Hamdan et al., 2013).

The flipped section that used teacher-constructed videos (TF) had the greatest significant decreases in both forms of mathematics anxieties, in addition to demonstrating a positive increase in beliefs about teaching and learning mathematics. Students’ open-ended responses and interviews suggested that such a positive increase occurred due to the improved opportunities for collaboration, exploration of multiple methods, and opportunities to examine student work related to the mathematical concepts. Additionally, their postcourse survey results indicated a desire to take more flipped classes and recommend flipped classes to friends.

These findings also support previous research that (a) increased use of student-centered, inquiry-based instruction in courses for preservice elementary teachers can alleviate students’ anxieties about mathematics (Huinker & Madison, 1997; Sloan, 2010; Tooke & Lindstrom, 1998); (b) a decrease in mathematics anxiety corresponds to a perceived increase in mathematical confidence (Bursal & Paznokas, 2006), and (c) flipped learning can have a significant positive influence on student attitudes and beliefs (McGivney-Burelle & Xue, 2013; Johnson, 2013, Wilson, 2013).

The least impacted group in terms of anxiety was the flipped section using the Khan Academy videos (KF). While the KF utilized flipped learning, the third-party video component may have limitations for students’ anxieties and beliefs about mathematics for several reasons. One possible reason may be that instructor-created videos provided better alignment between the lecture videos and the in-class activities. Since the videos were created by the instructor, he was able to focus the videos on targeted goals, outcomes, and concepts that would allow for more focused and enhanced learning opportunities during in-class activities (National Council of Teachers of Mathematics, 2014).

By directly aligning videos to each lesson, students may have better understood the goals for the upcoming class and the purpose of the different in-class activities and may have been better able to monitor their progress of learning during class (Marzano, 2009). This may have also been true to a lesser degree for the TL, since the in-class lectures were directly aligned with the activities that followed.

In contrast, the use of the Khan Academy videos often required using multiple videos to cover the upcoming topic, so the mathematical goals for the videos may not have been as consistently clear to the students. A lack of focus may have recreated feelings from negative school experiences and, thus, diminished some students’ opportunity to alleviate their anxiety (Bekdemir, 2010; Bursal & Paznokas, 2006; Geist; 2010).

The class interview of the KF also suggested that the student focus when using the Khan Academy videos was not on learning but on passing the videos’ challenge questions. This extrinsic motivator appeared for most students to be powerful, thus they spent more time completing the review questions than watching videos to prepare for the upcoming class. As previous research has suggested, a heightened concern about these assessments may have offset the impact the active learning opportunities provided during class (Geist, 2010).

Also, since a significant portion of the KF students admitted that they did not watch the videos, the potential priming benefit may have been diminished for improving students’ anxieties and perceptions of mathematics at the same level as with the TF students (Bodie, Powers, & Fitch-Hauser, 2006). As one TF student commented,

When you watch the video outside of class, you already know what you’re going to be talking about inside of class, so you’re, like, pre-informed of what you need to know. It makes me feel smarter when I come to class.

Many KF students did not share that experience.

### Implications for Flipped Instruction

The results of this study suggest that various factors must be considered before flipping any course. For those examining the replacement of in-class lectures, this study suggests that the emphasis of the instruction must be on creating an active and engaging in-class experience. Yet, careful detail must also be placed on the direct instruction component that is used to prime students.

Although a more time-consuming option, this study suggests instructors may want to consider creating their own videos, which will assure that the content aligns with their goals and expectations for preparation before the upcoming lesson. If third-party videos are used, instructors should take time to find a single video that effectively covers the same content as a video that might have been created by the instructor.

This study examined only the utilization of flipped learning to replace in-class lectures. However, other strategies may be utilized and investigated related to flipped learning, such as guided directions to start a project-based learning activity or providing a real-world demonstration that can be used to start an exploratory activity in the upcoming class (as in Dove & Dove, 2015b). This strategy can help create cognitive dissonance prior to class so that students come motivated and ready to engage in the activity or lesson.

This study also suggests that instructors may want to consider alternative methods for increasing the expectations of students to watch the videos. If videos are not watched, the in-class activities will often be unsuccessful, as students do not have the requisite knowledge to begin. While notes checks were perceived as appropriate by the TF students, video challenge questions were not highly regarded by the KF students. If review questions are a desired component, instructors may consider using them as formative instead of as summative assessments to help determine where there may be weaknesses and challenges with the upcoming lesson.

Finally, the primary focus of flipped learning in the mathematics classroom is to provide the in-class time needed to utilize student-centered instructional practices often observed in methods courses. This study suggests that significant amounts of collaboration between students and with the instructor are possible with the additional time provided by flipped learning. While mathematics instruction has in the past focused primarily on procedural understanding, in-class activities can now consistently include activities that build conceptual understanding. Additionally, since the instructor’s role shifts to that of facilitator, instruction can be differentiated with additional assistance provided to the students who need it most (Dove & Dove, 2015b; Wallace et al., 2014).

### Future Research

Although this study supports the potential of flipped learning as an instructional practice, much is still to be learned about the practice’s strengths and limitations. For example, this study examined the use of third-party videos that did not fully align with the curriculum. However, many mathematics textbooks are beginning to include section-based instructional videos. Future research may examine whether utilization of these videos has a comparable impact on students’ mathematics anxieties, or if the impact of teacher-created videos may have deeper roots in the connections and relations with the class instructor. Future research may also examine optimal methods for maximizing the time students spend watching videos and assisting students with retaining information from lecture videos.

Much of the research on flipped learning is only for a limited time frame (one unit, one semester, or one yearlong class). Future research may examine more longitudinal aspects related to flipped learning, such as how elementary preservice teachers’ mathematics anxiety and anxiety about teaching changes as they take more mathematics courses, as well as whether taking multiple flipped courses has any additional impact on decreasing students’ mathematics anxieties. Future research may also examine the instructional practices that preservice teachers incorporate once they become in-service teachers and how learning opportunities in flipped classes may have influenced their current teaching practices.

Additionally, more large-scale studies of flipped learning are needed. Increasing class sizes and number of instructors will help determine what variations may truly influence the success or failure of a flipped learning experience. For example, during this study the KF section was at 9 a.m., the TF at 10 a.m., and the TL at 11 a.m. While having the same instructor limited teaching variation, future studies might ask how the time of day impacts the students’ success. They might also ask how might a single instructor teaching three sections back-to-back impact material delivery and personal relationships with students in the class? With additional instructors and multiple sections of the same courses, these types of questions could be addressed.

A final interesting aspect within this study was the lower survey response rate by the TL. While the flipped classes were relatively high for repeated-measures survey responses, only 50% of the TL class completed both the MARS-R and ATMS surveys. We do not know why this group specifically avoided completing the surveys, but similar low survey completion rates by the lecture class has occurred in previous flipped studies (Dove & Dove, 2015a). Future research should follow up with nonrespondents to better understand the reasons for not completing the surveys, whether there is a relationship with mathematics anxieties, and how completion rates may be improved.

## References

Aslan, D. (2013). A comparison of pre- and in-service preschool teachers’ mathematical anxiety and beliefs about mathematics for young children. *SAVAP International. 4*(2) 2-22.

Ball, D. L., Lubienski, S., & Mewborn, D. (2001). Research on teaching mathematics: The unsolved problem of teachers’ mathematical knowledge. In V. Richardson (Ed.), *Handbook of research on teaching *(4^{th} ed.; pp. 433-456). New York, NY: Macmillan.

Beckmann, S. (2014). Mathematics for elementary teachers with activities (4th ed.). Boston, MA: Pearson.

Beilock, S. L., Gunderson, E. A., Ramirez, G., & Levine, S. C. (2010). Female teachers’ mathematics anxiety affects girls’ mathematics achievement. *Proceedings of the National Academy of Sciences of the United States of America, 107*(5), 1860–1863*. *Retrieved from http://www.pnas.org/content/107/5/1860.full.pdf

Beilock, S. L., & Willingham, D. T. (2014). Ask the cognitive scientist – math anxiety: Can teachers help students reduce it? *American Educator, 38*, 28-33.

Bekdemir, M. (2010). The pre-service teachers’ mathematics anxiety related to depth of negative experiences in mathematics classroom while they were students. *Educational Studies of Mathematics, 75*(3)*,* 311-328.

Bishop, J. L., & Verleger, M. A. (2013, June). The flipped classroom: A survey of the research. In *Proceedings of the 120 ^{th} American Society for Engineering Education Annual Conference & Exposition*. Atlanta, GA: ASEE.

Bodie, G. D., Powers, W. G., & Fitch-Hauser, M. (2006). Chunking, priming, and active learning: Toward an innovative and blended approach to teaching communication-related skills. *Interactive Learning Environments, 14*(2), 119-135.

Borko, H., Stecher, B. M., Alonzo, A. C., Moncure, S., & McClam, S. (2005). Artifact packages for characterizing classroom practice: A pilot study. *Educational Assessment, 10*(2), 73-104.

Bursal, M., & Paznokas, L. (2006). Mathematics anxiety and preservice elementary teachers’ confidence to teach mathematics and science. *School Science and Mathematics, 104*(6), 173-180.

Butt, A. (2014). Student view on the use of the flipped classroom approach: Evidence from Australia. *Business Education & Accredidation, 6*(1), 33-41.

Creswell, J. W. (2007). *Qualitative inquiry and research design: Choosing among five approaches *(2^{nd} ed.). Thousand Oaks, CA: Sage.

Dove, A. (2014). Students’ perceptions of learning in a flipped statistics class. In G. Marks, (Ed.), *Education and Information Technology Annual – 2014: A Selection of AACE Award Papers *(pp. 35 – 42). Chesapeake, VA: Association for the Advancement of Computers in Education.

Dove, A., & Dove, E. (2015a). Examining the influence of a flipped mathematics course on preservice elementary teachers’ mathematics anxiety and achievement. *Electronic Journal of Mathematics & Technology, 9*(2), 166-179*. *Retrieved from: https://php.radford.edu/~ejmt/deliveryBoy.php?paper=eJMT_v9n2n2

Dove, A., & Dove E. (2015b). Flipped mathematics classroom: What? Why? & How? *Virginia Mathematics Teacher, 42*(1), 36-40.

Foldness, N. (2016). The flipped classroom and cooperative learning: Evidence from a randomised experiment. *Active Learning in Higher Education, 17*(1), 39-49.

Frey, J. H., & Fontana, A. (1991). The group interview in social research. *The Social Science Journal, 28*(2), 175-187.

Fulton, K. (2012, June). Upside down and inside out: Flip your classroom to improve student learning. *Learning & Leading with Technology, *13-17. Retrieved from http://files.eric.ed.gov/fulltext/EJ982840.pdf

Furner, J. M., & Berman, B. T. (2003). Math anxiety: Overcoming a major obstacle to the improvement of student math performance. *Childhood Education*, *79*(3), 170-174.

Geist, E. (2010). The anti-anxiety curriculum: Combating math anxiety in the classroom. *Journal of Instructional Psychology, 37*(1), 24-31.

Gresham, G. (2008). Mathematics anxiety and mathematics teacher efficacy in elementary pre-service teachers. *Teaching Education*, *19*(3), 171-184.

Hadley, K. M., & Dorward, J. (2011). The relationship among elementary teachers’ mathematics anxiety, mathematics instructional practices, and student mathematics achievement. *Journal of Curriculum and Instruction, 5*(2), 27-44.

Hamdan, N., McKnight, P., McKnight, K., & Arfstrom, K. M. (2013). *A review of flipped learning. *Retrieved from http://www.flippedlearning.org/cms/lib07/VA01923112/ Centricity/Domain/41/LitReview_FlippedLearning.pdf

Hannafin, R. D., Burruss, J. D., & Little, C. (2001). Learning with dynamic geometry programs: Perspectives of teachers and learners. *The Journal of Educational Research, 94*(3), 132-144.

Hembree, R. (1990). The nature, effects, and relief of mathematics anxiety. *Journal for Research in Mathematics Education*, *21*, 33-46.

Hopko, D. R. (2003). Confirmatory factor analysis of the mathematics anxiety rating scale – revised. *Educational and Psychological Measurement*, *63*(2), 336-351.

Huinker, D., & Madison, S. K. (1997). Preparing efficacious elementary teachers in science and mathematics: The influence of methods courses. *Journal of* *Science Teacher Education, **8*(2), 107-126.

Johnson, G. B. (2013). *Student perceptions of the flipped classroom *(Unpublished masters thesis). The University Of British Columbia, Okanagan. Retrieved from https://circle.ubc.ca/bitstream/handle/2429/44070/ubc_2013_spring_johnson_graham.pdf?sequence=1

Kelly, W. P., & Tomhave, W.K. (1985). A study of math anxiety/avoidance in preservice elementary teachers. *Arithmetic* *Teacher, 32*(5), 51-52.

Kor, L.K., & Lim, C.S. (2009). Lesson study: A potential driving force behind the innovative use of Geometer’s Sketchpad. *Journal of Mathematics Education, 2*(1), 69-82.

Lawson, A., Benford, R., Bloom, I., & Carlson, M. (2002). Evaluating college science and mathematics instruction. *Journal of College Science Teaching, **6*, 388-393.

MacIsaac, D., & Falconer, K. (2002). Reforming physics instruction via RTOP. *The Physics Teacher, 40, *479-485.

Marzano, R. J. (2009). *Designing and teaching learning goals and objectives*. Bloomington, IN: Marzano Research Laboratory.

McGivney-Burelle, J., & Xue, F. (2013). Flipping calculus. *PRIMUS, 23*(5), 477-486.

McLaughlin, J. E., Griffin, L. M., Esserman, D. A., Davidson, C. A., Glatt, D. M., Roth, M. T…, & Mumper, R. J. (2013). Pharmacy student engagement, performance, and perception in a flipped satellite classroom. *American Journal of Pharmaceutical Education, 77*(9), 196-204.

National Council of Teachers of Mathematics. (2014). *Principles to actions: Ensuring mathematical success for all*. Reston, VA: Author.

Peterson, S. E., & Miller, J. A. (2004). Comparing the quality of students’ experiences during cooperative learning and large-group instruction. *The Journal of Educational Researcher, 97*(3), 123-133.

Piburn, M., Sawada, D., Turley, J., Falconer, K., Benford, R., Bloom, I., & Judson, E. (2000). *Reformed* *teaching observation protocol (RTOP): Reference* *manual *(ACEPT Technical Report No. IN00-3). Tempe, AZ: ACEPT.

Sam, L. C., & Ernest, P. (2000). A survey of public images of mathematics. *Research in Mathematics Education, 2*(1), 193-206.

Shernoff, D. J., Csikszentmihalyi, M., Schneider, B., & Shernoff, E. (2003). Student engagement in high school classrooms from the perspective of flow theory. *School Psychology Quarterly, 18*(2), 158-176.

Sloan, T. R. (2010). A quantitative and qualitative study of math anxiety among preservice teachers. *The Educational Forum, 74*, 242-256.

Stipek, D. J., Givvin, K. B., Salmon, J. M., & MacGyvers, V. L. (2001). Teachers’ beliefs and practices related to mathematics instruction. *Teaching and Teacher Education, 17, *213-226.

Tooke, D. J., & Lindstrom, L. C. (1998). Effectiveness of a mathematics methods course in reducing mathematics anxiety of preservice elementary teachers. *School Science and Mathematics, 98*(3), 136-139.

Ujifusa, A. (2014). Resistance to the common core mounts. *Education Week, 33*(29). Retrieved from http://www.edweek.org/ew/articles/2014/04/23/29cc-backlash.h33.html

Wallace, M. L., Walker, J. D., Braseby, A. M., & Sweet, M. S. (2014). “Now, what happens during class?” Using team-based learning to optimize the role of expertise within the flipped classroom. *Journal on Excellence in College Teaching, 25*(3&4), 253-273.

Wilson, S. G. (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. *Teaching of Psychology, 40*, 193-199.

Zakaria, E., Chin, L. C., & Daud, M. Y. (2010). The effects of cooperative learning on students’ mathematics achievement and attitude towards mathematics. *Journal of Social Science, 6*(2), 272-275.

## Appendix

Whole Class Semi-Structured Interview Questions

1. Please describe what occurs during the typical class.

- Do you feel that the instruction in class is meeting your learning needs? Why or why not?

2. Please describe the typical expectations for homework and other work outside of class.

- Do you feel that the expectations outside of class are meeting your learning needs? Why or why not?

3. Describe your overall impression of technology use in the classroom?

4. How, if at all, do you think the technology has impacted your learning experiences?

5. Overall, do you feel your MATH 111 course is meeting your needs to learn the mathematics content? Why or why not?

6. Overall, do you feel your MATH 111 course is meeting your needs to begin thinking about how you might teach mathematics content in your future class? Why or why not?

7. Is there anything else you would like to share about your MATH 111 class or your learning in this class?