McGuire, P. (2013). Using online error analysis items to support preservice teachers’ pedagogical content knowledge in mathematics. Contemporary Issues in Technology and Teacher Education, 13(3). Retrieved from https://www.citejournal.org/volume-13/issue-3-13/mathematics/using-online-error-analysis-items-to-support-preservice-teachers-pedagogical-content-knowledge-in-mathematics

Using Online Error Analysis Items to Support Preservice Teachers’ Pedagogical Content Knowledge in Mathematics

by Patrick McGuire, University of Colorado, Colorado Springs

Abstract

This article describes how a free, web-based intelligent tutoring system, (ASSISTment), was used to create online error analysis items for preservice elementary and secondary mathematics teachers. The online error analysis items challenged preservice teachers to analyze, diagnose, and provide targeted instructional remediation intended to help mock students overcome common error patterns and misconceptions. A short description of how the ASSISTment system was used to support follow-up in-class discussions among preservice teachers is provided, as well as suggestions for producing similar online error analysis items in other content areas. Directions for accessing all of the mathematics error analysis problem sets currently available in the ASSISTment system, sample error analysis items and responses, and a rubric for implementing these assignments in mathematics methods classes to support preservice teachers are included at the conclusion of the article.

Errors are inevitable in the learning of mathematics, as the human brain is not genetically programmed to memorize multiplication facts, carry out multistep operations, or perform exact mathematical calculations (Sousa, 2008). The history of error analysis (also called error pattern analysis) in mathematics education dates back to the work of Radatz (1979). Since then, much of the literature has focused on two key areas of error analysis in mathematics: (a) the identification and interpretation of students’ common error patterns as a result of misconceptions and (b) best practices for instructional remediation. This article addresses both areas and, specifically, highlights the importance of exposing preservice elementary and secondary mathematics teachers to common student error patterns that are the result of underlying mathematical misconceptions.

The first component of error analysis, the ability to identify and interpret children’s common errors, implies that teachers must not only possess strong mathematics content knowledge, but also the ability to focus on students’ levels of understanding. Researchers have suggested that the ability to interpret students’ understanding is a necessary skill for good teaching (Carpenter & Lehrer, 1999; Davis, 1989; Graeber, 1999). This ability to interpret student understanding helps teachers to become more acutely aware of the process of learning and the aspects of mathematics that are difficult to grasp (Shulman, 1987). Identifying and interpreting students’ understanding also provides teachers with useful information about the underlying cognitive processes related to how students think and develop mathematical knowledge (Sousa, 2008), rather than simply focusing on the right or wrong answer, which does not necessarily provide a window into what students are learning (Ashlock, 2006).

An increased focus on student thinking and the problem solving process also serves as a powerful mechanism that helps to connect pedagogy, mathematics, and student learning (Franke & Kazemi, 2001) and provides teachers with the first step in providing targeted remedial instruction (Ketterlin-Geller & Yovanoff, 2009). Despite the general consensus that error analysis is a basic and important task needed for mathematics teaching, a 2009 study conducted by Morris, Hiebert, and Spitzer suggested that many preservice mathematics teachers lacked a complete ability to plan effectively for and evaluate students’ mathematical thinking. Additional research has suggested that preservice mathematics teachers share many of the same misconceptions as and make errors similar to those made by their students (Ryan & McCrae, 2005).

The second component of error analysis, the ability to diagnose and remediate common errors with targeted instruction, is perhaps the most important skill for mathematics teachers to possess. Peng and Lou (2009) considered error analysis to be an “inseparable part of the routine of mathematics teaching” (p. 25), one that can be used as a tool in organizing instruction. The ability to remediate misconceptions with developmentally appropriate and efficient instructional techniques underlies Shulman’s (1986) concept of pedagogical content knowledge (PCK). He described teachers who require knowledge of strategies that are most likely to reorganize the understanding of learners.

More recent research has supported Shulman’s idea and indicated that preservice teachers can demonstrate growth in PCK, especially with regard to the knowledge of student difficulties, by observing and discussing real classroom settings and remediation techniques (Akkoç & Yesildere, 2010). Finally, Hill, Ball, and Schilling (2008) cited the abilities not only to remediate errors, but rather to proactively anticipate student errors, interpret incomplete thinking, and predict how students will approach specific tasks, to be key components of an effective mathematics teacher.

Despite the fact that all teachers encounter students who make mathematical errors, on a daily basis, many preservice teacher courses do not include authentic opportunities for teacher candidates to analyze and discuss common errors.  This paper describes an online system that was used to give preservice teachers an opportunity to analyze and remediate student work.  It includes a brief overview of two mathematics education courses where the online items were implemented. Next, error analysis problem structure, predicated upon the technology-enhanced formative assessment (TEFA) framework, which was developed by Beatty and Gerace (2009) to support and assess teachers’ PCK, is discussed. This paper concludes with a discussion of the importance of engaging preservice teacher candidates in dialogical discourse and the implications of using similar online error analysis items in preservice teacher coursework.

Description of Preservice Teachers and Math Education Courses

Preservice teacher candidates from two separate mathematics courses enrolled at a mid-sized, suburban university in central Colorado completed the error analysis items described in this paper.  Error analysis items were developed and administered across two separate mathematics education courses: (a) Secondary Mathematics Methods (fall 2010, 2011) and (b) Mathematics and Cognition (spring 2011, 2012).  Secondary Mathematics Methods, a fall-only class, is specifically designed for undergraduate preservice teacher candidates enrolled in a secondary mathematics teacher licensure program.  The primary objectives of this course include developing preservice teachers’ PCK in secondary mathematics.  All students enrolled in the Secondary Mathematics Methods course were in their final semester of coursework prior to their student teaching experience (junior or senior class standing) and were secondary mathematics education majors in the College of Education.

Mathematics and Cognition is a graduate level class designed for preservice and in-service teacher candidates. This course explores mathematical development from birth to adulthood and makes explicit connections between teaching, thinking, and learning in mathematics. Unlike the Secondary Mathematics Methods course, Mathematics and Cognition spans a much wider age range and covers grades prekindergarten through high school.  Despite the fact that this is a graduate level course, all students enrolled over the past 2 years have been preservice teacher candidates seeking certification in the areas of mathematics education or special education.    

Error Analysis Problem Structure

The error analysis problem structure described in this article was developed based on the theoretically and empirically grounded TEFA framework outlined by Beatty and Gerace (2009) and included three separate but related levels (see Table 1). Each error analysis item was intentionally designed to be diagnostic in nature, but also to enjoin the three basic principles of the TEFA framework in that they provided (a) question-driven instruction, (b) opportunities for formative assessment, and (c) dialogical discourse. Table 1 provides an overview of each level of the error analysis problems and how these instructional design principles supported the development of preservice teacher candidates’ PCK in mathematics.

Table 1
Overview of the Error Analysis Three-Level Problem Structure

Level 
Description 
Answer Format
Scaffolding Provided?
TEFA Framework 
1. Identify students’ error pattern The preservice teacher analyzes a theoretical example of students’ work and is then responsible for identifying their error pattern or misconception

 

Open-ended essay (complete sentences) No Question-driven instruction
2. “Think like a student” The preservice teacher must answer one or two similar subproblems using the same error pattern that the student exhibits in level one of the problem.

 

Fill in the blank or multiple choice. Graded as correct vs. incorrect Yes. Two optional hint messages

Hint 1: description of error pattern

Hint 2 – Correct answer given

Formative assessment
3. Describe remediation strategies The preservice teacher completes the problem by providing developmentally appropriate instructional strategies that can be used to remediate the students’ error pattern. Open-ended essay (complete sentences) No Dialogical discourse

 

Appendix A describes a specific example that demonstrates the process that preservice teachers work through when analyzing an error problem.  This example displays an error pattern where a student (Daphne) has difficulties solving basic problems involving positive and negative integers.

At the first level preservice teacher candidates were responsible for identifying a student error pattern by analyzing several examples of their work and then responding to an open-ended question.  After determining the student error pattern, teacher candidates provided a description of the error pattern in complete sentences (see Appendix A, Level 1).  The next level of the problems, Level 2, required the teacher candidates to “think like the student” and required them to answer one or two related questions using the same error pattern (see Appendix A, Level 2).

Unlike Level 1, Level 2 items were presented in a multiple-choice or fill-in-the-blank format and, therefore, provided an opportunity to assess formatively teacher candidates’ understanding of the error pattern. Finally, each item was completed by asking the preservice teacher candidate to provide instructional remediation strategies specifically designed to support students’ conceptual understanding.  Level 3 responses served as the primary vehicle for facilitating dialogical discourse in the classroom among teacher candidates.

Development and Data Collection Using ASSISTment

The ASSISTment system (www.assistment.org) was used to create the error analysis problems described in this article.  Some items were created from scratch by the instructor, while other items were modified from Ashlock’s (2006) textbook, Error Patterns in Computation: Using Error Patterns to Improve Instruction, a text purchased by students for the two courses. ASSISTment is a free, web-based platform funded by the National Science Foundation. The primary goal of the system is to provide users with assistance while simultaneously providing instructors with assessment data.

In other words, ASSISTment is designed to blend tutoring and testing effectively, an outcome achieved by providing users with a combination of scaffolding questions, hints, and error messages.

Despite the fact that the ASSISTment system has been primarily funded to support middle and secondary level students in mathematics, it is free to anyone and can be used to create customized content and scaffolding in virtually any subject area and at any grade level. Researchers have demonstrated that the system content creators can build personalized content in a matter of minutes with little to no programming knowledge (Razzaq et al., 2008). The error analysis items described in this paper were all built using the ASSISTment system.  Like any technology, there is a bit of a learning curve; however, the system includes an extensive set of video and text-based support to help instructors build their own content. (For videos and supports describing the development process, see http://teacherwiki.assistment.org/wiki/Learn_ASSISTments_Now_Online. Preview a sample dynamic, that is, workable, error analysis problem created in the ASSISTment system at http://www.assistments.org/public_preview/link/dHlwZT1hc3Npc3RtZW50JmlkPTkxMjYw.)

Facilitating Dialogical Discourse

As preservice teachers completed the online error analysis problem sets, the ASSISTment system automatically tracked their responses at each level of the problem. Open-ended responses (associated with Level 1 and Level 3 of the error analysis problem structure) were captured online using the essay scoring feature of the ASSISTment platform (see Appendix B for an example). Using this system function, the instructor was able to review quickly all preservice teacher responses and then select a subset of responses (typically two to three) to display anonymously to the class.

Displaying preservice teachers’ answers in class provided an incentive for all students to submit high-quality answers, because they wanted their response to be selected as examples. It also provided a variety of remediation strategies for each problem, thus encouraging a higher level of dialogical discourse. This finding is consistent with research suggesting that teachers’ learning can be promoted through a common group analysis and discussion surrounding student’s work, which can have significantly positive effects on instructional practice (Kazemi & Franke, 2004).

At the conclusion of each semester in anonymous course surveys, the preservice teachers cited the sharing of remediation strategies as the most useful and relevant portion of the error analysis exercises, because these discussions directly supported the development of their own instructional strategies. An added bonus of the online data tracking and storage within ASSISTment is that the best preservice teacher responses from different semesters could be selected and displayed.  This practice is consistent with previous research suggesting that preservice teachers should be provided with opportunities to discuss instructional applications in the context of experiences that allow them to demonstrate and develop their own PCK (Guzel, 2010).

Through a careful analysis and rich discussion about different suggested remediation strategies, preservice teachers were exposed to a variety of techniques that could be used to help correct student errors (e.g., using concrete manipulatives, pictorial representations, real life connections, and graphic organizers). Finally, these rich discussions around common mathematical error patterns also directly informed preservice teachers’ subsequent lesson and activity designs for the courses.

Replication and Future Directions

Online error analysis items can be used to support dialogical discourse about students’ thinking in preservice mathematics methods courses. Similar error analysis items could be leveraged to support in-service mathematics teachers, as well, perhaps in professional development workshops or online learning communities to help current classroom teachers more effectively diagnose and remediate errors based on students’ understanding.

With the recent 2010 adoption of the new Common Core State Standards for Mathematics (CCSSM) across the United States, it would seem prudent to design error analysis items that specifically target CCSSM grade levels (e.g., K-12) or mathematical domains (e.g., Number and Operations in Base 10). Development of items based on the CCSSM would help to address key mathematical concepts that serve as the foundations on which students are expected to build their knowledge as they move through school. This strategy would also provide an opportunity for teachers to create and share specific lesson activities that are intended both to correct and prevent common mathematical errors.

Another natural extension of the online error analysis concept would be to replicate similar instructional models in other content areas with preservice or in-service teachers. For example, the system could be used to encourage dialogical discourse or support preservice teachers’ PCK in English (e.g., teachers could identify a common grammatical error pattern, “think” like the student, and then provide remediation strategies). In other words, instructors working with preservice teacher candidates or professional development workshop leaders could explore creative ways to use similar error analysis items to improve the PCK in their respective content areas.

Concluding Thoughts

 Even though the ASSISTment system is freely available and is currently being implemented in many middle and secondary classrooms throughout the United States, the potential advantages of using the system in higher education to support preservice teachers has yet to be fully realized.

Mathematics methods instructors working with preservice teachers at the university level or professional development coordinators working with in-service teachers at the district level may find these ideas useful. Error analysis sets are included in Appendix C and a sample assignment rubric is found in Appendix D).

Clearly, preservice mathematics teachers cannot be expected to learn all there is to know about student thinking in all areas of mathematics. It is also an unrealistic expectation that all preservice teachers will be exposed to every possible error pattern.  However, by exploring online error analysis problems and engaging in dialogical discourse about effective remediation strategies, preservice teachers can become equipped with a better understanding of why analyzing students’ thinking is important in key areas of mathematics and how they can effectively remediate. Through this process, preservice teachers will more clearly see a direct link between students’ understanding and the implications this insight has on the teaching and learning of mathematics.

 

References

Akkoç, H., & Yesildere, S. (2010). Investigating development of preservice elementary mathematics teachers’ pedagogical content knowledge through a school practicum course. Procedia Social and Behavioral Sciences, 2(2), 1410-1415.

Ashlock, R.B. (2006). Error patterns in computation: Using error patterns to improve instruction. Upper Saddle River, NJ: Pearson.

Beatty, I.D., & Gerace, W.J. (2009). Technology-enhanced assessment: A research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18, 146-162.

Carpenter, T., & Lehrer, R. (1999). Teaching and learning mathematics with understanding. In E. Fennema, & T. Romberg  (Eds.), Mathematics classrooms that promote understanding (pp. 19-32). Mahwah, NJ: Lawrence Erlbaum Associates.

Davis, R. (1989). Learning mathematics: The cognitive approach to mathematic education. London, England: Routledge.

Franke, M.L., & Kazemi, E. (2001). Learning to teach mathematics: Developing a focus on students’ thinking. Theory into Practice, 40, 102-109.

Graeber, A. O. (1999). Forms of knowing mathematics: What preservice teachers should learn. Educational Studies in Mathematics, 38, 189-208.

Guzel, E.B. (2010). An investigation of preservice mathematics teachers’ pedagogical content knowledge, using solid objects. Scientific Research and Essays, 5(14), 1872-1880.

Hill, H.C, Ball, D.L., & Schilling, S.G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4), 372-400.

Kazemi, E., & Franke, M.L. (2004). Teacher learning in mathematics: Using student work to promote collective inquiry. Journal of Mathematics Teacher Education, 7, 203-235.

Ketterlin-Geller, L.R., & Yovanoff, P. (2009). Diagnostic assessments in mathematics to support instructional decision making.  Practical Assessment, Research & Evaluation, 14(16).

Morris, A.K., Hiebert, J., & Spitzer, S.M. (2009). Mathematical knowledge for teaching in planning and evaluating instruction: What can preservice teachers learn? Journal for Research in Mathematics Education, 40(5), 491-529.

Peng, A., & Luo, Z. (2009). A framework for examining mathematics teacher knowledge as used in error analysis. For the Learning of Mathematics, 29(3), 22-25.

Radatz, H. (1979). Error analysis in mathematics education. Journal for Research in Mathematics Education, 10(3), 163-172.

Razzaq, L., Patvarczki, J., Almeida, S., Vartak, M., Feng, M., Heffernan, N.T., & Koedinger, K.R. (2008). The ASSISTment Builder: Supporting the life cycle of ITS content creation (WPI Tech Report No. WPI-CS-TR-08-06). Worcester, MA: Worcester Polytechnic Institute.

Ryan, J., & McCrae, B. (2005). Assessing preservice teachers’ mathematics subject knowledge. Mathematics Teacher Education and Development, 7, 72-89.

Russell, M., O’Dwyer, L.M., & Miranda, H. (2009). Diagnosing students’ misconceptions in algebra: Results from an experimental pilot study. Behavior Research Methods, 41(2), 414-424.

Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.

Shulman, L.S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1-22.

Sousa, D. A. (2008). How the brain learns mathematics. Thousand Oaks, CA: Corwin Press.

Author Notes

Patrick McGuire
University of Colorado Colorado Springs
Email: pmcguire@uccs.edu


Appendix A
Sample Error Analysis Problem
(With Student Responses) [a]

 

Level
Sample Problem
Sample Preservice Teacher Response
1. Identify students’ error pattern Daphne gets some correct sums.  Even so, many of her sums are incorrect. She seems to have constructed her own rule for adding integers.  Identify Daphne’s error pattern. Write your answer in complete sentences. 

image 1

It looks as though Daphne is summing the two numbers by use of a number line. Locating the first number and adding (going to the right on a number line) the second number with little or no regard to the sign.  
2. “Think like a student” Make sure you have identified Daphne’s error pattern by responding to the following problem using her incorrect procedure.  How would Daphne answer the following question?

10 + -6 =

16

3. Describe remediation strategies What instructional strategies would you use to assist this student with his/her difficulties? Answer in complete sentences

 

I would first have her look at the problem and circle the negative sign in front of all the negative numbers to draw attention to the negative sign. Then I would have her re-write the problem addressing the negatives signs. So, for example, given the problem 10 + -6, I would have her circle the negative sign in front of the 6 to draw attention to the negative and then explain that the problem can now be re-written as 10-6 and solved to be 10-6=4. I would do many of these problems with her showing how the negative changes the problem in different ways and then supervise her doing some on her own. Finally, providing a real world context (e.g., money, temperature, or sports – positive vs. negative rushing yards) may help Daphne to understand the concept more fully.
[a] Item used from Ashlock (2006) Error Patterns in Computation: Using Error Patterns to Improve Instruction. 

 

 

 


 

Appendix B
Screen Shots Taken From the ASSISTment System Showing the Automatically Generated Essay Reports for Level 1 and Level 3 of a Sample Error Analysis Item

Juan’s answers to a Pre-Algebra quiz are shown. Identify his error pattern. Write your answer in complete sentences.

 

image 1

Level 1: What is the error pattern displayed by Juan?

image 2

 

 

 

Level 3: What instructional strategies would you use to assist this student with his difficulties?

image 3


Appendix C
Problem Set ID Numbers

For those readers interested in using the error analysis problem sets the author has created to date, click on the problem set ID number in the far left column below. Note that these are the public preview versions of the problem sets, and you will not be able to assign/collect data from these problems.

Problem Set ID Number
Mathematical Content Coverage
Targeted Grade Level
10422 Fractions Elementary
10706 Whole Number Operations Elementary
10784 Geometry and Measurement Elementary
13640 Percentages, signed number rules, distributive property

 

Middle School
33323 Proportions, distributive property, graphing linear equations

 

Middle School
33324 Multiplying polynomials, simplifying radical expressions, Pythagorean Theorem Middle school & High School

 


 

Appendix D
Sample Assignment Rubric

 

Item

Unsatisfactory
(0-1 points)

Satisfactory
(2-3 points)

Exemplary
(4-5 points)

1. Identification of students error pattern

(5 points)

Error pattern is not identified correctly or inappropriate mathematical language is used to describe the student’s misconception. The error pattern described as mostly accurate, however, there parts of the error pattern description that are not clearly articulated AND/OR parts of the mathematical language is wrong. The error pattern described is accurate and correct mathematical language is used in the description.
2. Think like the student

(5 points)

None of the sub problems are completed using the student’s misconception. Some, but not all of the sub problems are completed using the student’s misconception. All sub problems are completed using the student’s misconception.

 

Item

Unsatisfactory
(0-4 points)

Satisfactory
(5-8 points)

Exemplary
(9-10 points)

3. Remediation strategies proposed

(10 points)

The remediation strategies are not developmentally appropriate and will not support the students’ conceptual understanding. There are no citations or references to course materials that support the remediation strategy. The proposed remediation strategies are developmentally appropriate and are designed to support students’ conceptual understanding. However, specific examples from course readings or course texts, in-class discussions, etc., are NOT cited. The proposed remediation strategies are developmentally appropriate and are designed to support students’ conceptual understanding. Specific examples from course readings or course texts, in-class discussions, etc. are cited.