Swan, K., & Locascio, D. (2008). Alignment of technology and primary source use within a history classroom<. Contemporary Issues in Technology and Teacher Education, 8(2). Retrieved from http://www.citejournal.org/vol8/iss2/currentpractice/article1.cfm

Evaluating Alignment of Technology and Primary Source Use Within a History Classroom

Kathleen Swan
University of Kentucky

David Locascio
Longwood University

Abstract

Many researchers in the social studies have supported the use of primary sources in history classrooms as a support for historical inquiry.  Although primary sources have become accessible via the Internet, simply using digital primary sources, does not automatically translate into historical thinking or technology best practice.  Consequently, an evaluation matrix was constructed for one study to gauge the fidelity of primary source use according to three domains, curriculum content, instructional processes, and student products or outcomes.  In this article, the researchers provide background information on the development of the evaluation matrix, present the instrument, and evaluate its effectiveness in categorizing both primary source and technology usage. 

The chief value of technology lies, therefore, in providing the leverage so urgently needed for moving social studies instruction away from passive, teacher-dominated approaches emphasizing recall and regurgitation toward active student centered forms of learning demanding critical and conceptual thinking from all students at all levels. (Crocco, 2001, p. 2)

Researchers in the teaching and learning of history advocate instructional approaches that engage students in the process of “doing” history, including building historical knowledge through the use of primary sources, conducting historical inquiry, and encouraging students to think historically (Kobrin, 1996; Levstik & Barton, 2001; van Hover & Yeager, 2002; Wineburg, 1991). This approach encourages students to raise questions and to marshal solid evidence in support of their answers; to go beyond the facts presented in their textbooks and examine the historical record for themselves; to consult documents, journals, diaries, artifacts, historic sites, works of art, quantitative data, and other evidence from the past, and to do so imaginatively―taking into account the historical context in which these records were created and comparing the multiple points of view of those on the scene at the time to build understandings of historical significance (Levstik, 1996; Seixas, 1996; Wineburg, 1991; Yeager & Davis, 1996). 

For history teachers wanting to embrace historical thinking processes in the social studies classroom, there is much promise. In response to the greater demand for primary and secondary resources, Web sites or archives of historic documents created by libraries, universities, and government agencies have proliferated. These sites allow teachers to access and download documents free of charge for use in the social studies classroom.  By allowing students to explore the raw materials of the past, digital history sites, as well as the use of complementary technologies, have the potential to engage students actively in the construction and interpretation of history (Ayers, 1999; Braun & Rissinger, 1999; Tally, 1996).

However, using primary sources does not automatically translate into historical thinking (Swan & Hicks, 2007).  Rather, it is the teacher who juxtaposes documents against one another, who asks critical thinking questions of a document, or who elicits the bias or perspective of the author of the document that allows students to practice historical inquiry skills.  As the quote in beginning of this article suggests, technology has the potential for facilitating these processes, but it is the teacher who leverages the technology to conduct historical inquiry in the classroom.

To date, little research has been done within this framework of intersection between historical thinking and technology in the history classroom (Swan & Hofer, 2008). As researchers wanting to explore this relationship, we constructed an evaluation matrix that would aid in categorizing observational data for one qualitative study of three secondary American history teachers and their uses of primary sources.  In this article, we provide background information on the development of the evaluation matrix, present the instrument, and evaluate its effectiveness in categorizing both primary source and technology usage.

Developing the Tool

Miles and Huberman (1994) stated,  “A conceptual framework explains, either graphically or in narrative form, the main things to be studied – the key factors, constructs or variables – and the presumed relationships among them” (p. 18). The development of the conceptual framework for the study, grounded in the literature on the efforts to bring technology and primary sources into history education, helped to provide a focus for the inquiry and means to display the data for analysis.  Embedded within the framework are foundational premises about the relationship between technology and the teaching of history.

Students’ technology skills need to be more than a distinct and often disconnected goal of the curriculum (International Society for Technology in Education, 2007; Mason et al., 2000), but also an embedded support for instructional designs that move beyond teacher-centered, textbook-driven approaches and toward models in which students are more actively involved in their learning (Doolittle & Hicks, 2003; Harris, 1995; Mason et al., 2000; van Hover, Berson, Bolick, & Swan, 2004). Ideally, the technology is employed not only to invite student engagement, but to broaden and deepen student understandings through the purposeful acquisition and assembly of materials to guide students’ learning and encourage independent inquiry.

According to Crocco (2001), technology-infused pedagogy is evident in “classrooms that foster questioning, challenging, and reflecting by all students” (p. 388). Incorporating the technology without framing it in sound pedagogy runs the risk of “investing a great deal of time, attention, and money to educationally marginal means” (Crocco, 2001, p. 387).  More recently, Crocco’s argument has been echoed in the development of Technological Pedagogical Content Knowledge (TPCK; Mishra & Koehler, 2006).

In the area of history instruction, the interpretive student stance advocated by Crocco is wholly consistent with the broader orientation toward disciplined inquiry and historical thinking advanced by Levstik and Barton (2001) and others (Kobrin 1996; van Hover & Yeager, 2002, Wineburg, 1991). The second premise holds that an orientation toward historical thinking is valid and desirable and can be uniquely supported by technology at several pedagogical stages (Brush & Saye, 2000; Hicks, Doolittle, & Lee, 2004; Hofer & Swan, 2006; Lee & Calandra, 2004; Saye & Brush, 1999; Swan & Hicks, 2007). The National Standards for History (National Center for History in Schools, NCHIS, 1996) characterized a set of five core skills under the broad concept of historical thinking; these include chronological thinking, historical comprehension, historical analysis and interpretation, historical research capabilities and historical issues-analysis and decisionmaking. From these historical habits of mind historiography, the writing of history, proceeds (Holt, 1995; Levstik & Barton, 2001; Van Sledright, 2001).

Students are exposed to the ways historians use text-based and nontext primary sources, relics, and artifacts as building blocks in the historiographic process. The goals of the history curriculum encompass the narrative explanation of historical events, as well as the consideration of broader structures and themes and the inclusion of historiographic processes referred to by Leinhardt (1993) as metasystems.  

When such metasystemic processes are appropriately scaled and applied to the classroom use of historical sources, students are expected to frame historical questions, look for and evaluate evidence, identify viewpoints, make connections across sources, assess relevance, draw inferences from text and nontext resources, and develop plausible historical narrative of their own (Barton, 2001). Technology can play several roles in this multistep process, serving as a repository from which sources can be acquired, a platform through which the sources can be delivered and evaluated, and a tool through which student understandings can be demonstrated and assessed. 

These premises were incorporated into developing our evaluative instrument in several ways. First, historical thinking can clearly be taught well without using electronic means of access, delivery, and product demonstration. Consequently, the evaluation of the overall fidelity of the instructional design must be separate from, and must effectively outweigh, the evaluation of technology use per se. Similarly, primary sources are a necessary but not sufficient condition for the practice of historical thinking in the history classroom.  For example, a teacher might use a primary source so that students could uncover author bias or to juxtapose it against another document of the same event to understand more fully multiple perspectives in history.  In doing so, teachers are building students’ understanding of historical comprehension as laid out by the National Standards for History (NCHS, 1996). 

Conversely, teachers could use primary sources as they would a textbook, not asking any questions of the authenticity or reliability of the document, but rather using primary sources as a content delivery mechanism.  Although primary sources provide an entry point into historical scholarship, simply using primary sources does not translate into historical thinking (Barton, 2005).  For those reasons, the evaluation matrix (see Appendix A [PDF]) was constructed to gauge the fidelity of primary source use according to three domains, curriculum content, instructional processes, and student products or outcomes (Tomlinson, 1995).         

The first domain, “Content,” consists of the ideas, concepts, descriptive information, and facts, rules, and principles presented to the learner (Tomlinson, 1995). Since we were concerned with measuring the content specific to historical thinking processes, we viewed primary sources as the foundation for the teaching of history.  In the evaluation matrix, primary sources used in the classroom were evaluated for their complexity, variety, and orientation. 

The second domain, “Instructional Process,” incorporated the presentation of content, including the design of learning activities for students, the framing of analytical questions, as well as the teaching methods and thinking skills used in the classroom (Tomlinson, 1995). Because the study was confined to measuring methods of historical thinking, instructional process was limited to the way in which primary sources were used in exercises promoting historical interpretation, teaching historical methodology, and assembling historical narratives. This component of the matrix was informed, in part, by the continuum of historical teaching purposes framed by Leinhardt (1993).

Finally, “Products” are the outcomes of instruction that consolidate learning and communicate ideas (Tomlinson, 1995).  The last domain of the evaluation matrix gauged the use of primary sources in assessment, looking at the autonomy given to students in constructing historical narratives.  Specifically, assessments were dissected to examine the level of independence given to students in historical inquiry, the degree to which students were supplied primary sources within the assessment, and the extent to which students documented the historical processes used within the assessment. 

The three domains (content, process, and product) were broken into four components, which assumed greater degrees of fidelity.  Because no similar evaluation tool existed at the time of this study, we called upon our own experiences as former high school history teachers and current teacher educators, as well as the amalgam of theoretical frameworks that presume levels of sophistication in the various components of instructional design, including content, process, and product, as well as technology integration (Crocco, 2001; Harris, 1997; International Society for Technology in Education, 2007; Kobrin, 1996; Levstik & Barton, 2001; Mishra & Koehler, 2006; Shulman, 1986).

Although the first three components of each domain related to use of primary sources, the fourth component addressed the use of technology within the three instructional domains.  For the technology component, the evaluation matrix confined the use of technology to a mechanism for teachers to acquire primary sources, for students or teachers to deliver primary sources for instruction, and finally, for students to construct a historical narrative using various software (e.g., Microsoft Powerpoint, or iMovie) and hardware (e.g., laptops, projectors, etc.). 

Special attention has been given to the relationship between the use of primary sources and technology, noting these two facets of instruction are potentially mutually exclusive. For example, a teacher may promote historical thinking in the classroom using a multitude of nondigitally acquired primary sources as a means of reconstructing a particular event.  The teacher could have students write historical narratives taking into account author bias and historical perspective, meanwhile documenting the metacognitive skills necessary in historical research.  Because we aimed to elucidate the role technology played in facilitating historical thinking, it was necessary to provide a mechanism for excluding technology as a factor in historical thinking.  The evaluation matrix was constructed with this in mind and provided a lens for examining the data collected.  A summary of the evaluation matrix is provided in Table 1.

Table 1
An Evaluation Matrix (Abridged) for the Use of Primary Sources and Technology in the Secondary Classroom

Using the Evaluation Matrix

The study entailed following three 11th-grade American history teachers during the 2003-2004 academic year.  These teachers attended a sequence of professional development workshops sponsored by the historians at a digital history center at a large Southern public university. These University faculty members had begun developing resources to address the issues of access and implementation of primary sources that teachers face. In 2002, the digital history center developed professional development workshops intended to train American history teachers in the use of digitized primary sources and the online multimedia guide. Initial surveys of these three teachers indicated instructional practices that included the frequent use of primary sources within their American history curriculum and a varied response in the use of technology in supporting historical thinking practices. 

The evaluation matrix was used in observations to validate the teachers’ self-reports, as well as to describe qualitatively the teaching methods used by each participant. Of particular interest were the ways in which these secondary history teachers used primary sources, whether their use constituted historical thinking (as defined by the study’s conceptual framework), and finally, the contextual factors influencing use of primary sources.   Additionally, this study sought to explicate the role of technology in supporting historical thinking practices, as well as the intrinsic and extrinsic influences that inhibited or prohibited the effective use of technology as it related to historical thinking.

Observations were conducted 12 to 15 times per participant using the evaluation matrix. Although all three participants consistently used primary sources in their American history classrooms, the three varied notably in the role primary sources played in the overall curricular design, as well as in the degree of instructional sophistication with which the sources were employed.  Their use was characterized by degrees of sophistication, prevalence within the curriculum, and level of student centeredness within each lesson.  Using the evaluation matrix, Table 2 offers both a characterization and summary of use in terms of historical content, instructional processes, and classroom assessment for each participant. 

Table 2
Characterization of Primary Source Use in Supporting Historical Thinking Practices

In almost all cases, the participants self-selected the primary sources used within the curriculum; however, there was a disparity in the types of documents the teachers chose, the way in which the teachers approached document analysis with their students, and the role of primary sources in the classroom assessment.  For example, Larry (pseudonym) consistently assembled text-based documents that featured male authors who influenced American political history. The variety of documents was minimal, but the sources he selected were complex and allowed students to engage with multiple perspectives of a historical event.  During many of the observations, we marked 1s or 2s for Variety on the Content evaluation matrix, but  3s and 4s for Complexity and Orientation.

In contrast, Jamie and Jason (pseudonyms) used a myriad of nondiscursive documents (e.g. , photographs, political cartoons, maps, diary entries, and video) that were easily read and interpreted by their students.  Although they often presented several primary sources per lesson, the sources did not contradict one another or the history textbook.   During these classroom observations, we marked the Content evaluation matrix with 1s and 2s for Complexity and Orientation, but 3s and 4s for Variety. 

Jamie and Jason had similar approaches to the types of primary sources they selected, yet they varied greatly in the ways they instructed their students to approach document analysis.    Jamie trained her students to use a four-step method to unpack sources.  This method began with determining the message of the source, the bias of the author, the purpose of the document, and the document’s effectiveness in achieving its purpose.  Often, the intention of this strategy outweighed its effectiveness in the classroom, but elements of scaffolding were evident within her instruction.

Students in Larry’s class effectively used the document reading strategy he called “APPARTS” to analyze the documents.  APPARTS stands for author, place, prior knowledge, audience, reason, the main idea, and significance.  In addition to utilizing the acronym as an analytical framework for his classes, Larry incorporated a more nonlinear trajectory to his instruction, developing tentative assertions about sources, then setting them aside in order to revisit them after other sources had been examined. The degree to which Larry’s students modeled their analyses after the competencies of professional historians led to these observations being generally coded higher under the Purpose criterion.

Jason, on the other hand, lacked any formal or informal strategy for document analysis, instead relying on an open ended, “What do you guys think?” We were able to use the Process evaluation matrix to characterize differences between the three teachers in terms of Fidelity, Purpose, and Activity.

The participants also varied in their use of primary sources in classroom assessment.  All three participants relied to varying degrees on traditional assessment, including multiple choice tests, objective quizzes, and chapter summaries.  However, Jason used only traditional assessment, while Jamie and Larry included document analysis in their assessments.  This use varied, and the Product evaluation matrix was used to uncover these differences.  Students in Larry’s AP American History course were regularly given Document Based Questions as a way of preparing for the end-of-year exam but, more importantly, as a good teaching tool for historical inquiry.  Jamie often used project-based assessment that required the inclusion of primary sources as a way of measuring historical understanding.  Additionally, using primary sources, Jamie created her own multiple-choice questions to prepare her students for the state history exam.   We often marked 0s for Jason, 1s and 2s for Jamie, and 3s and 4s on the Product evaluation matrix.

Participants’ use of technology in instruction did not include attention to all of the components of effective technology use, as defined by the study’s conceptual framework.  All of the teachers used technology to acquire and display artifacts, but use of technology in instruction varied according to frequency and level of student centeredness.   For example, Larry rarely used technology to display primary sources, whereas Jason and Jamie used technology almost exclusively. It is important to again note that Larry’s approach to teaching through these sources was less linear, with students returning to prior sources more frequently than in the other classroom settings, an instructional design that encouraged the use of hardcopy sources.   In terms of assessment, Jason and Larry rarely used technology, but Jamie regularly had students create Web pages, PowerPoint’s, etc., that required the use of primary sources.  Using the aggregated observations, the evaluation matrix (see Table 3) offers both a characterization and summary of technology use in supporting historical thinking practices.

Table 3
Characterization and Technology Use in Supporting Historical Thinking Practices

The evaluation matrix allowed us to parse the observations consistently across participants.  From the aggregated descriptors, we were able to make comparisons between participants and, ultimately, pursue contextual factors that might have influenced these characterizations.  More importantly, we were able to disaggregate the data in important ways.  All three participants used primary sources within their history classroom; however, we were able to show clearly that the teachers did not use the sources with the same degree of fidelity.  Moreover, we argued that, although primary sources can represent an important platform for historical thinking, the sources themselves are insufficient without sound pedagogical design.   Additionally, we also posited that digital acquisition of primary sources could be a first step in building technology enhanced curriculum, but it was far from the inquiry-based history instruction touted in the social studies literature.

 Last, embedded within the conceptual framework for this study is the assumption that technology in history classrooms should not be used solely to build technology facility in students but rather as a mechanism to facilitate document-based instruction.  As demonstrated in Table 3, although all three participants used technology, they were able to leverage technology to support historical thinking practices to varying degrees—a subtle but important point emphasized within the instrument. 

Conclusion

In a recent article by Mishra and Koehler (2006), the authors borrowed from Shulman (1986) to argue that effective technology integration requires developing sensitivity to the “dynamic, transactional relationship” between pedagogy, content, and technology (p. 1030).  The instrument described here was crafted before the framework of technological pedagogical content knowledge (TPCK). Yet, it certainly imbeds the spirit of Mishra and Koehler’s argument that in order to integrate technology and then to assess its effectiveness educators must take into account the complexity and contextuality of teaching.  As a result, a checklist that measured technological infusion outside of the context of the pedagogical objectives supported by the technology was clearly insufficient. Instead, the best of a priori evaluative instruments would contain a broad framework of instructional approach, embedded with applicable pedagogical sophistication and set alongside descriptions of how the technology is being employed.  

The Evaluation Matrix described in this article, while at times unwieldy, allowed us to capture the rich, thick description called for in qualitative research (Geertz, 1973) but, more importantly, to capture the nuances of teaching practice.   Our hope is that other researchers, program evaluators, or school administrators could use this matrix in their own attempts to develop more integrated, or cohesive, protocols for assessing the efficacy of history instruction.  

References

Ayers, E. (1999). History in hypertext.  Virginia Center for Digital History. Retrieved September 25, 2007, from, http://www.vcdh.virginia.edu/PastsFutures.html

Barton, K. C. (2001). A picture's worth: Analyzing historical photographs in the elementary grades. Social Education, 65(5), 278-283.

Barton, K. (2005).  Teaching history: Primary sources in history – Breaking through the myths.  Phi Delta Kappan, 86(10).

Braun, J., & Risinger, F. (1999). Surfing social studies. Washington, DC: National Council for the Social Studies.

Brush, T., & Saye, J.  (2000).  Implementation and evaluation of a student-centered learning unit: A case study.  Educational Technology, Research and Development, 48(3), 79-100.

Crocco, M.S. (2001). Leveraging constructivist learning in the social studies classroom: A response to Mason, Berson, Diem, Hicks, Lee, and Dralle. Contemporary Issues in Technology and Teacher Education, 1(3) . Retrieved from http://www.citejournal.org/vol1/iss3/currentissues/socialstudies/article2.htm

Doolittle, P., & Hicks, D. (2003).  Constructivism as a theoretical foundation for the use of technology in social studies.  Theory and Research in Social Education, 31(1), 72-104.

Geertz. C. (1973). The interpretation of cultures. New York: Basic Books.

Harris, J. (2005). Our agenda for technology integration: It's time to choose. Contemporary Issues in Technology and Teacher Education, 5(2), 116-122.

Harris, J. (1997). Wetware: Why use activity structures.Learning and Leading with Technology, 25(4), 13-17.

Hicks, D., Doolittle, P., & Lee, J. K. (2004). History and social studies teachers' use of classroom and Web-based historical primary sources. Theory and Research in Social Education, 32(2), 213-247.

Hofer, M., & Swan, K.O.  (2006). Standards, firewalls, and general classroom mayhem: Implementing student-centered technology projects in the elementary classroom.  Social Studies Research and Practice, 1(1).  Retrieved from http://www.socstrp.org/issues/viewarticle.cfm?volID=1&IssueID=1&ArticleID=13

Holt, T. (1995). Thinking historically: Narrative, imagination and understanding. New York: College Entrance Examination Board.

International Society for Technology in Teacher Education. (2007).  National educational standards and the states.  Retrieved from http://cnets.iste.org/

Kobrin, D. (1996).  Beyond the textbook: Teaching history using documents and primary sources. Portsmouth, NH: Heinemann.

Lee, J. K., & Calandra, B. (2004). Can embedded annotations help high school students perform problem solving tasks using a Web-based historical document? Journal of Research on Technology in Education, 36(4), 65-84.

Leinhardt, G. (1993). Instructional explanations in history and mathematics. In  Proceedings of the 15^th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Lawrence Erlbaum Associates.

Levstik, L. S. (1996).  Negotiating the historical landscape.  Theory and Research in Social Education, 24(4), 393-397.

Levstik, L.S., & Barton, K. (2001).  Doing history: Investigating with children in elementary and middle schools (2^nd ed.).  New Jersey: Lawrence Erlbaum Associates.

Mason, C., Berson, M., Diem, R., Hicks, D., Lee, J., & Dralle, T. (2000). Guidelines for using technology to prepare social studies teachers. Contemporary Issues in Technology and Teacher Education [Online serial], 1(1). Retrieved from http://www.citejournal.org/vol1/iss1/currentissues/socialstudies/article1.htm

Miles, M., & A. Huberman. (1994). An expanded sourcebook: Qualitative data analysis. Thousand Oaks, CA: SAGE Publications.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108(6), 1017-1054.

National Center for History in the Schools. (Ed.). (1996). National standards for history. Los Angeles: Author.

Saye, J., & Brush, T. (1999). Student engagement with social issues in a multimedia supported learning environment. Theory and Research in Social Education, 27(4), 472 -504.

Seixas, P. (1996). Conceptualizing growth in historical understanding. In D. Olson & N. Torrance (Eds.), Education and human development (pp. 765-783). London: Blackwell.

Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14.

Swan, K., & Hicks, D. (2007) Through the democratic lens: The role of purpose in leveraging technology to support historical thinking in the social studies classroom. The International Journal of Social Studies Education, 21(2), 142-168.

Swan, K., & Hofer, M. (2008). Information ecologies: Technology in the social studies.  In L. Levstik, & C. Tyson (Eds.), Handbook of research on social studies teaching and learning (pp. 307-326). Erlbaum Publishing.

Tally, B. (1996). Up against authentic history.  Electronic Learning, 16(2), 40-41.

Tomlinson, C. (1995). How to differentiate instruction in mixed-ability classrooms. Alexandria, VA: Association for Supervision and Curriculum Development.

van Hover, S., Berson, M., Bolick, C., & Swan, K.. (2004). Implications of ubiquitous computing for the social studies. Journal of Computing in Teacher Education, 20(3), 107-112.

van Hover, S.D., & Yeager, E.A. (2002).  "Making students better people?" A case study of a beginning teacher.  The International Studies Forum, 3(1), 219-232.

Van Sledright, B. (2001). From empathetic regard to self-understanding: Impositionality, empathy, and historical contextualization. In O.L. Davis Jr., E.A. Yeager, & S.J. Foster (Eds.), Historical empathy and perspective taking in the social studies (pp. 51-68). Lanham, MD: Rowan & Littlefield Publishers, Inc.

Wineburg, S. S.  (1991).  On the reading of historical texts: Notes on the breach between school and academy.  American Educational Research Journal, 28(3), 495-519.

Yeager, E.A., & Davis, O.L., Jr. (1996). Classroom teachers’ thinking about historical texts: An exploratory study.  Theory and Research in Social Education, 24(2), 146-66.

Author Note:

Kathleen Owings Swan
University of Kentucky
343 Dickey Hall
Lexington, KY 40506-017
Phone: 859-257-1893
kswan@uky.edu

David Locascio
Longwood University
201 High Street; Hull Room 215
Farmville, VA 23909
Phone: 434-395-2609
locasciod@longwood.edu