About Button Subscribe Button Archive Button Search Button Submissions Button
Social Studies [globe image]
Contemporary Issues in Technology and Social Studies Teacher Education
 Sponsored by the National Council for the
Social Studies College and University Faculty Assembly

Milman, N. B., & Bondie, R. (2012). An examination of teachers’ ratings of lesson plans using digital primary sources. Contemporary Issues in Technology and Teacher Education, 12(4). Retrieved from http://www.citejournal.org/vol12/iss4/socialstudies/article1.cfm

An Examination of Teachers’ Ratings of Lesson Plans Using Digital Primary Sources

Natalie B. Milman
George Washington University

Rhonda Bondie
Fordham University

Abstract

This mixed method study examined teachers’ ratings of 37 field-tested social studies lesson plans that incorporated digital primary sources through a grant from the Library of Congress Teaching with Primary Sources program for K-12 teachers. Each lesson, available in an online teaching materials collection, was field-tested and reviewed by at least three teachers other than the lesson’s author. The analysis illustrates that the majority of lessons (70%) utilized PowerPoint to show primary sources and pose questions to students about the primary sources examined. Qualitative analysis identified key factors that impacted lesson implementation and teachers’ analysis of student learning. This study raises several implications regarding the design and purpose of lesson plans available online for integrating digital primary sources into P-12 teaching, as well as the design and content of professional development whose purpose is to prepare teachers to integrate digital primary sources in their teaching. 


Social studies teachers today have thousands of digital primary source materials available to them for free on the World Wide Web (WWW), as well as an immeasurable number of lesson plans to help them utilize these resources in their classrooms. The proliferation of digital primary source materials and lesson plans available online, funded largely through government and private grants to establish and utilize digital collections and libraries (Bolick, Hicks, Lee, Molebash, & Doolittle, 2004), offers students the opportunity to “do history” (Levstik & Barton, 2001) in ways that could not be accomplished or even imagined before the advent of the World Wide Web.

Doing history involves collecting, analyzing, and interpreting data (Levstik, 1996), often using web-based primary sources such as digitized photos and letters accessible on the Internet (Schrum & Schrum, 2009), and applying these skills to form an understanding of history and its consequences. Many lessons online are designed specifically for students to do history using primary source materials.  Yet, information about the degree to and ways in which teachers use these materials and their impact on students’ learning is limited.

Purpose and Objectives

The purpose of this mixed-methods study was to examine teachers’ ratings of 37 field-tested social studies lesson plans that incorporated digital primary sources as part of a Library of Congress Teaching with Primary Sources professional development program for K-12 teachers. The U.S. Congress provided funding for it via a Library of Congress grant. Each lesson (also known as a “learning experience”), freely available and searchable to workshop participants, was field-tested and reviewed by at least three teachers other than the lesson author. The study’s objectives were to examine the following research questions about the ratings of social studies lessons that incorporated primary sources from the lesson plan database:

  • What student grouping strategies were used in the lessons?
  • How was technology utilized for the lessons that incorporated digital primary sources?
  • What teaching strategies were used?
  • To what extent did teacher reflections explain how lessons were implemented and why student learning did or did not occur?

Research on Digital Primary Sources in K-12 Education

Although literature on digital primary sources in K-12 education is limited, the body of research is growing about their use in the social studies by teachers (Bolick, 2006; Friedman, 2006a, 2006b; Fry, 2010; Hicks, Doolittle, & Lee, 2004; Lee, Doolittle, & Hicks, 2006; McGlinn, 2007; Swan & Hicks, 2006-2007) and K-12 students (Brown & Dotson, 2007; Dutt-Doner, Cook-Cottone, & Allen, 2007; Farmer, Knapp, & Benton, 2007; Friedman & Heafner, 2008; Tally & Goldenberg, 2005). None of these studies, however, examined the quality of any lesson plans or other materials that have been developed to help teachers use the digital primary sources, as this study examines.

Findings from these studies illustrate a complex picture as to how, when, and why digital primary sources are or are not being used in social studies classrooms. For example, in a study examining the beliefs and practices of six high school teachers using digital primary sources, Friedman (2006b) found three major barriers to the implementation of digital primary sources in the teachers’ classrooms. These were the teachers’ perceptions that using such materials required more time, the emphasis on teaching to the state standards, and limited instructional time to teach the content outlined in the standards.

In another article, Friedman (2006a) elaborated further on the negative impact of standards on the use of digital primary sources. He explains that the pressure to develop curriculum addressing state standards and the requirement to use standardized tests that reflect rote learning resulted in limited use of digital primary sources and “best historical practices” (p. 323). 

In another study by Hicks et al. (2004), 158 high school social studies teachers completed a survey examining the extent to which they used web-based primary sources in their teaching. The researchers found that teachers did, indeed, use digital primary sources in their instruction; however, the applications of primary sources were not necessarily consistent with “best practices” in doing history. In this study, too, teachers reported that using primary sources was more time-consuming.

Lee et al. (2006) had similar results in another study that used the same survey, but with another group of teachers. Here, though, the researchers emphasized that “neither digital nor non-digital historical primary sources will have a major impact in the social studies or history classroom until teachers make more active use of the sources themselves” (p. 299). Therefore, it seems logical that high-quality lesson plans about how to employ the use of web-based digital primary sources might prove beneficial.

McGlinn’s (2007) study also surfaced similar barriers to using web-based primary sources highlighted in other studies. However, her study revealed some new concerns. First, the teachers who participated in her study reported that they would use digital primary sources if they were more “teacher friendly” and visible on the Web. Second, teachers reported difficulty with locating sources and “usable materials within the collection” examined. Third, lesson plans were needed to use digital primary sources, as the following illustrates: “Although he rarely follows lesson plans he finds online verbatim, he did find it extremely helpful to read how other teachers used the resources.” Therefore, it is imperative to examine not only whether digital primary sources are, indeed, being utilized, but also which mechanisms, such as lesson plans, might improve the quality and quantity of their use in classrooms.

In an in-depth case study of three high school teachers conducted by Swan and Hicks (2006-2007), findings were mixed. Although the accessibility and availability of primary sources resulted in increased use of primary sources for instructional purposes in the three classrooms studied, the application of historical inquiry and technology varied a great deal among the three teachers. The researchers determined that the strongest influences on how the teachers used the digital primary sources and technology centered on their own beliefs, the “purpose for teaching” (p. 159), and their level of “pedagogical content knowledge” (p. 163).

Fry (2010) conducted a mixed-methods study of the Teaching With Primary Sources professional development program at Loyola University Chicago. Participants were 15 of 34 in-service teachers who participated in one of three different courses focused on integrating Library of Congress digital primary sources. Fry examined how often and which primary sources teachers used, how teachers used primary sources, and teachers’ perceptions of students’ motivation, interest, and achievement as a result of the primary source professional development.

Findings showed that the use of primary sources increased and the types of primary sources most often utilized were “photo/sketch/poster and documents” (p. 134). Also, results demonstrated that primary sources were used most often “to illustrate concepts as well as provide examples, to integrate inferential, analytical and critical thinking skills, and to elicit content understanding” (p. 134). Teachers also reported that by integrating primary sources, their students were more engaged and experienced authentic learning.

These studies, most examining only a small number of teachers and employing qualitative methods, paint a picture of the barriers and benefits of using digital primary sources by social studies teachers. Findings show that, although access to digital primary sources certainly increases their use, other factors affect their actual implementation in practice. 

Research Method

This study employed a sequential mixed-methods design (Tashakkori & Teddlie, 1998). These types of studies, which fall under the mixed-methods umbrella, consist of two methods that occur in different phases of a study, each applying different methods and conducted sequentially. In this study, a “QUAN → qual” design (Morse, 2003, p. 198) was applied.

Quantitative methods involved descriptive statistical analyses of teachers’ ratings of 37 different social studies lessons posted in the lesson archive of a grant-funded collection of lesson plans created through a process of development, field-testing, peer review, and evaluation. Each lesson was rated by at least three different teachers who participated in the Teaching With Primary Sources professional development program, using a questionnaire after field-testing the lesson in their classrooms.

Although 613 lessons were housed in the database, only 37 had completed questionnaires of lessons field-tested by three different teachers. The questionnaire included Likert scale ratings of the lesson’s effectiveness and challenge, as well as student engagement. The ratings also included appraisal of the lesson as a tool for developing student understanding of a topic and literacy skills. Teachers completed three narrative prompts that described their use of digital primary sources in their own classrooms, analysis of student learning, and recommendations for future implementation.

Teaching With Primary Sources Professional Development Program

The lesson plan database analyzed in this study was developed through a professional development program for K-12 teachers called the Library of Congress Teaching With Primary Sources Northern Virginia Partnership (TPSNVA). The TPSNVA’s partners included four different school districts. The TPSNVA professional development program was designed by school personnel in an effort to promote student learning with Library of Congress resources while advancing current local school district pedagogy initiatives.

The partner school systems established the program with the following three purposes:

  • Fellowship: introduce collections of resources and best instructional practices related to using digital sources in the classroom for K-12 teachers of all subjects.
  • Scholarship: deepen content knowledge and create teaching materials designed for sharing with other teachers  and grades.
  • Leadership: offer opportunities to share expertise and conduct research concerning how teachers teach and students learn with digital resources.

The TPSNVA program provided professional development on using digital primary sources to about 5,000 educators from northern Virginia. The professional development included a variety of activities such as special tours of the Library of Congress exhibits, lectures with curators and reference specialists to develop educator content knowledge, and hands-on technology workshops focusing on integrating digital primary sources into the instructional program. Through the professional development program, teachers created thousands of teaching materials, including lesson plans and learning activities, which were freely accessible on the TPSNVA website for teachers anywhere to share, use, and rate.

Lesson Plan Database and Sample

When examined for this study the lesson plan database contained 969 lesson plans from all of the major subject areas. The database was developed to house and share the lessons created by teachers who participated in the professional development program, as well as the field tests of these lessons. All of the lessons centered on using U.S. Library of Congress primary sources with students. In-service teachers submitted one lesson and a field-test report of another lesson from the database to complete the professional development program.

For this study, all of the social studies lessons were considered for inclusion. However, only 37 out of the 613 social studies lessons (6%) had at least three field tests completed by classroom teachers after being available for 2 years through the searchable database. The number of field tests for each lesson in this study varied from 3 to 13 field tests. Table 1 shows the grade range, total number of field tests, lessons with three or more field tests, field tests examined, and the percentage of field tests examined.

Table 1
Social Studies Lessons – Field Tests by Grade Level

Grade Range
Total Field Tests
Lessons
With 3 or More Tests
Field Tests
Examined
% of Field Tests Examined
Pre-K- 2
115
12
57
31%
3 – 5
121
16
74
40%
6-8
70
7
45
24%
High School
33
2
10[a]
5%
Total
339
37
186
100%
[a] Original lesson was designed for high school students; however, the field tests were completed with elementary and middle school students.

Limitations

This study has numerous limitations. First, the data set consists of ratings of different lessons implemented without observation and do not represent a normal distribution or population. Second, the relationships between variables on the Likert Scale (little and very) are likely unequal. Third, no effort was made to ensure inter-rater reliability since field tests were performed and reported on a voluntary basis and without rater training. Also, the focus of the professional development program was to assist teachers in using digital primary sources to help students learn.

Toward that end, instructional strategies were modeled through demonstration lessons taught by the course instructor of each session. Therefore, lesson plans receiving three or more field tests may have received field tests simply because the strategy had been modeled during the professional development.  The purpose of the professional development was to model, create, and share lessons that used digital primary sources; however, the professional development did not prepare teachers as raters of lesson plans.

Table 2 shows the sample of lessons examined in this study by grade range and subcategory. Note that although two lessons originally designed for high school students received three or more field tests, none of the field tests took place with high school students. There were no lessons with three or more field tests that occurred in a high school classroom.

Table 2
Lessons with Three or More Field Tests by Grade and Subcategory (N = 37)

Pre-K - 2
3 - 5
6 - 8
9 - 12
Total
12 (32%)
16 (43%)
7 (19%)
2[a] (5%)
37 (100%)
[a] Original lesson was designed for high school students; however, the field tests were completed with elementary and middle school students. 

 

Table 3 is a description of the field tests examined in this study. The table shows the total number of field tests submitted and the number of field tests that were attached to the 37 lessons with three or more field tests. These are described by subcategory within the larger subject of social studies. The largest collection of lesson plans was in the subcategory of “United States History” consisting of 56%.

Table 3
Total Field Tests Submitted and Number of Lessons With Three or More Field Tests by Grade and Subcategory

Submitted
Pre-K–2
#  3+ Field Tests
3 - 5
#  3+ Field Tests
6 - 8
#  3+ Field Tests
9 - 12
#  3+ Field Tests
Total
Total
Civics
18
10
2
0
12
12
0
0
32
22
Economics
6
3
7
4
0
0
0
0
13
7
US History
49
21
69
40
49
30
25
10
192
101
Holidays
8
4
0
0
0
0
0
0
8
4
Geography
6
0
12
7
2
0
2
0
22
7
Other
11
4
1
0
0
0
0
0
12
4
World
15
15
20
13
4
3
1
0
40
31
Explorers
2
0
10
10
2
0
0
0
14
10
Government
0
0
0
0
1
0
5
0
6
0
Psychology
0
0
0
0
0
0
0
0
0
0
Total
115
57
121
74
70
45
33
10
339
186
Note. N = 37 Lessons;  N = 186 Field tests

 

Table 4 shows the percentage of field tests by grade range and subcategory. The primary audience for the lesson plans and field tests was the grade range 3 to 5 with 74%. The grade range of Pre-K to 2 had 31%, and grades 6 to 8 had 24%.

Table 4
Field Tests in Sample: No. of Field Tests by Grade and Subcategory

Pre-K - 2
3 - 5
6 - 8
9 - 12
Total
57 (31%)
74 (40%)
45 (24%)
10 (5%) [a]
186 (100%)
[a] Original lesson was designed for high school students; however, the field tests were completed with elementary and middle school students. N = 186 Field tests

 

Qualitative methods involved analysis of comments written in the open-ended section of the lesson review.  Answers to the following open-ended questions from the lesson review questionnaire were analyzed:

Description of Classroom Use: Describe modifications from the original description including the teaching process, Library of Congress primary sources used, and any other changes that you made.

Analysis of Student Learning: Evaluate student learning by giving examples of student successes and difficulties in relation to the learning experience’s understanding goal.

Evaluation and Recommendations: Reflect upon the success of this learning experience and suggest changes for improvement. Explain the results of using primary sources, best instructional practices, and classroom management.

Qualitative data were analyzed using grounded theory (Glaser & Strauss, 1967). Grounded theory involves constant comparative analysis of data through coding of data and theoretical sampling. Once teacher comments were coded into initial categories through open coding, core categories emerged through axial and selective coding (Strauss & Corbin, 1990, 1998) resulting in the theory presented.

Results

Quantitative Results

This section presents quantitative findings of the study by sharing the grouping strategies, uses of technology by teachers to present the digital primary sources, and the teaching strategies used in the lesson plans from the repository.

Grouping Strategies. The research question, “What grouping strategy was used?” was explored to examine the format of lessons that teachers chose to replicate and the degree to which the field-testing teachers used the same formats for implementing the lesson as the lesson plan author. Teachers’ instructions within the lesson plan used various student grouping strategies, including whole group [W], pair [P], small group [SG], individual [I], or some combination of those (see Table 5). Field-testing teachers also reported on the ways students were grouped for implementing the field-test of the lesson (see Table 6).

Table 5
Grouping Strategies Utilized by Teacher Authors of Lesson Plans (N = 37 Lessons)

Whole (W)
Pair (P)
Small Groups (SG)
Individual (I)
W/P
W/SG
W/I
P/SG
P/I
SG/I
W/SG/I
6
1
7
3
4
8
7
0
0
0
1
16%
3%
19%
8%
11%
22%
19%
0
0
0
2%

 

Table 6
Grouping Strategies Implemented by Teachers Described in Field Tests (N = 186 Field Tests)

Whole (W)
Pair (P)
Small Groups (SG)
Individual (I)
W/P
W/SG
W/I
P/SG
P/I
SG/I
W/SG/I
50
12
78
16
2
12
14
0
0
1
1
27%
6%
43%
9%
1%
6%
7%
0
0
0
0

 

In the lesson plans, lesson authors reported using a variety of groupings including whole group and small group 22% and small group 19% (totaling 41%) of the time. However, the most popular grouping strategy teachers described actually using in the field tests (see Table 6) involved placing the students into small groups (43%). These differences could be a result of the lesson plan authors’ including more details in their descriptions of student groupings in the lesson or field-testing teachers not following the teacher instructions in the lesson plan, instead relying primarily on the lesson objectives and materials.

The data on grouping strategies suggest that some of the whole group modeling used by the lesson authors may not have been implemented by field-testing teachers. A future study may examine what information is needed in an online lesson plan to help teachers replicate complicated groupings from lesson plans or the extent to which student groupings for learning are context-based and are, therefore, not replicated from curriculum materials that teachers download.

Use of Technology. The use of technology to teach with digital primary sources varied between teachers printing primary sources to share with students and projecting or using individual computers to use primary sources in the digital form. Table 7 illustrates the use and non-use of PowerPoint for the lessons.  Table 8 displays the use and non-use in the field tests.

Table 7
Use of Projected Images and Printed Images by Teacher Authors (N = 37 Lessons)

Projected
Printed
Individuals on Computer
Projected & Printed
Printed & Individual
Projected & Individual
Projected, Printed, Individual
12
18
0
7
0
0
0
32%
49%
0
19%
0
0
0

 

Table 8
Use of Projected Images and Printed Images by Teacher Field Tests (N = 186 Field Tests)

Projected
Printed
Individuals on Computer
Projected & Printed
Printed & Individual
Projected & Individual
Projected, Printed, Individual
31
126
6
17
1
5
0
16%
68%
3%
9%
1%
3%
0%

 


Although the lesson plans were housed in a secure online database and the focus of each lesson was using primary sources found in the Library of Congress’ website, when the lessons were implemented with students, computers were rarely used by students. The frequency of printing the primary sources for students to examine increased in the field-testing. The decrease in projecting digital primary sources and the increase in printing primary sources reported by field-testing teachers may be connected to the grouping strategy. For example, the author of the lesson plan may have modeled a strategy for examining the primary sources by projecting a source for the whole class to see and think about together before moving into small groups.

This modeling may have provided students with a strategy for working in smaller groups. This might describe the lesson structure of the 22% of teachers who reported using both whole group and small group strategies in their lesson plan directions. The field-testing teacher may have moved straight to small group work with the primary sources, eliminating the modeling that uses a whole group strategy and projects a digital primary source.

One of the most popular teaching strategies modeled in the professional development sessions, “Zoom-In Inquiry,” (the use of an online tool to zoom in only on portions of the digital primary source) requires the use of a projector and cropping a digital image in a PowerPoint presentation. Zoom-In is usually implemented using a whole group strategy. As shown in Table 9, the Zoom-In Inquiry was used in 14% of the lesson plans that received three or more field tests. The popularity of Zoom-In Inquiry strategy may have influenced the high frequency of both whole group instruction and of projecting a digital image in lesson plans receiving three or more field tests, as seen in Table 6 and 8, respectively.

Teaching Strategies. The lessons that received three or more field tests from teachers participating in the professional development program used teaching strategies modeled in the sessions 100% of the time. Each professional development session began with a model activity that demonstrated using digital primary sources and one of the targeted instructional practices for the professional development program, such as teaching for understanding, differentiated instruction, literacy instruction, and technology integration.

All of the teaching strategies were demonstrated for teachers of all grades. Often computer labs were not available for the professional development sessions, so teachers used printed primary sources about half of the time when participating in the model activities. Table 9 identifies the teaching strategy by grade range and the number of lessons with three or more field tests incorporating the teaching strategy. Table 10 provides a description of the lessons by subcategory, grade, and teaching strategy.

Table 9
Teaching Strategy Used in the Lessons With Three or More Field Tests (N = 37 Lessons)

Strategy
Description
Type of Activity
Pre-K - 2
3 - 5
6 - 8
9 - 12
%

Images Draw  You In

Use an image and question to draw students into thinking about a topic

Introduction
0
1
1
0
5%

Poster

Peak interest in a topic and suggest further research by displaying a few sources and questions

Introduction
2
2
0
0
11%

Zoom-In

View pieces of an image one at a time to gather clues to form and revise a hypothesis

Introduction
1
2
2
0
14%

Total of Introductions = 11

3
5
3
0
30%

Document Analysis

Interrogate a primary source like a historian

Main Activity
5
1
2
0
22%

Life in a Box

Work as a historical detective using a set of clues to justify a hypothesis

Main Activity
0
3
0
2[a]
14%

Sort it Out

Sort sources into themes to create an exhibition

Main Activity
4
6
2
0
32%

Total of Main Activity = 25

9
10
4
2
68%

Movie Making

Create a digital documentary

Culminating Project
0
1
0
0
3%

Total Culminating Activity = 1

0
1
0
0
3%

Total Lessons by grade level

12
16
7
2[a]
 
[a]Field tests of lessons categorized as “Other” were combined with US History for this table.

 

Table 10
Lessons by Subcategory, Grade, and Teaching Strategy 9 (N = 37 Lessons)
Subcategory
Pre-K - 2
3 - 5
6 - 8
9 - 12

Civics

2 Sorts 3 Life    

Economics

1 Poster 1 Sort    

US History [a]

2 Analysis
2 Sorts
1 Poster
1 Analysis
3 Sorts
1 Poster
1 Images Draw
1 Zoom
1 Movie Making
2 Analysis
1 Sort 1 Images Draw
2 Zooms
2 Life
2 Life

Holidays

1 Analysis      

Geography

1 Analysis      

World

1 Analysis
1 Zoom
1 Poster
1 Sort
1 Sort  

Explorers

  1 Sort
1 Zoom
   

 

12
16
7
2
[a] Field tests of lessons categorized as “Other” were combined with US History for this table.

 

It is important to note that more elementary than secondary teachers participated in the professional development program. The higher frequency of elementary teachers who field-tested primary source based lesson plans is reflective of the proportions of elementary versus secondary teachers participating in the professional development program rather than the number of elementary teachers seeking primary source based lesson plans from the database.

The Sort It Out teaching strategy requires students in small groups to sort collections of primary sources into different groups with common themes. The high frequency of the use of the Sort It Out strategy (32%) in the lesson plans is probably related to the high frequency of the small group strategy reported in the field tests (43%) and the frequency of printed primary sources (68%). The data collected and analyzed in this study do not delineate whether or not teachers searching these online databases of lesson plans are more likely to implement a lesson plan with materials formatted to print out for small learning groups, or if teachers were drawn to the Sort It Out activity through the professional development and the exposure created in the high frequency of strategy use, which led to the high frequency of small group learning and use of printed materials.

Qualitative Findings

The qualitative findings addressed themes that surfaced through analysis of the 186 teacher narratives answering three prompts: (a) description of classroom use, (b) analysis of student learning, and (c) evaluation and recommendations. The descriptions of classroom use and analysis of student learning supported the assertion that the selection of the primary sources and a specific, explicit method for students to examine the sources is a key factor in determining the success of the lesson. Teachers consistently mentioned problems with the materials being either too easy or hard, the materials having too many images or not enough, needing more time for students to explore the sources, and needing to model for students an example of how the teacher wanted the students to interact with or think about the primary sources.

For instance, the following comment supports this idea: “I would be more intentional about modeling my thinking process (metacognition) when showing students an example that we can do together before they work in groups.” Teacher concerns about the difficulties students experienced in the lessons centered on supporting students in making inferences and drawing conclusions.

Qualitative data about the need for teachers to model for students an example of how to interact with the primary sources helps triangulate the quantitative data of student grouping reported in the field tests. In the qualitative comments, field-testing teachers described primarily using a small group strategy (43%) or using a whole group and small group strategy (6%). However, the teacher authors used the whole group and small group management strategies 22% of the time. This shows incongruence between the design of the lesson by the lesson author and its actual implementation by the field-testing teachers. The data reported suggest that teacher authors may have modeled the learning strategy for students before small group work more often than the field-testing teachers did in practice.

The field-testing teachers’ comments regarding changes that they would make if they implemented the lesson suggested modeling the learning strategy. This raises the question of whether field-testing teachers did not read or understand the lesson instructions, whether the student grouping instructions were unclear in the lesson directions, or whether the field-testing teachers simply wanted to implement the lesson their own way or in a way that best addressed their students’ needs.

Management of small groups also consistently appeared both as a problem and as a strength in lessons. Problems occurred when the groups were not on task, as the following comments show: “hard for students to remain attentive” or “off task/forgot the assignment.” However, other comments described groups working well independently without teacher support. For example the following comment shows, “I do feel they (students) were respecting each other and working collaboratively.” The same lesson plan would often have field tests reporting both problems and successes with small groups. It was often unclear if the field-testing teacher had even read or followed the implementation directions from the online lesson plan or if the teacher used the resources and the general idea of the lesson but applied different teaching methods and grouping strategies for students.

In the analysis of student learning, consistent problems that students experienced included lack of background knowledge, the need for understanding vocabulary, and limited ability to make inferences, as the following quotes from the field-test show: “Students had trouble creating meaning with the eagle in terms of it representing courage,” and “Some of my ELLs (English Language Learners) had difficulty with some of the vocabulary.” Successes reported by field-test teachers consistently identified the high quality of student discussion mentioning use of knowledge and vocabulary, listening to and challenging other students, and considering different points of view.

Field-test teachers often pointed out student levels of engagement, student ability to use new content knowledge, and connections to required curriculum standards as factors contributing to high ratings in terms of the effectiveness of the lesson. Confirming comments include, “Students were highly engaged in the box analysis and provided some of the best writing we had seen from this class,”  and “There was quite a debate about whether education, flowers, and clothes were a need, linked their discussion to math and adaptation.”

The field-test forms for future studies may include these more specific qualities. For example, instead of rating a lesson plan on the development of student knowledge, teachers might rate the lesson plan on the development and use of student background knowledge and academic vocabulary related to the unit of study. Rather than rating lesson plans on the student ability to develop understanding, teachers might reflect on the demand for students to make inferences and cite examples of student quotes from conversation and student work that demonstrate inferences.

The qualitative analysis also demonstrated an emphasis on the materials themselves and the amount of time necessary for completing the lessons. For example, comments such as, “I’d like to get some less abstract examples” and “(have) small posters printed for each group,” showed teachers’ concerns. Because the lessons were designed around students’ learning from primary sources, teacher comments reflected the importance of the identified sources for the lessons including the relationship to the curriculum, readability, and ways students could easily view the sources.

Another theme was time. Advice included statements such as, "Allow plenty of time, this activity takes a solid hour,” “Ask only one spiraling question per image,” and “Did not complete assessment due to time.” These statements imply that the lesson author’s stated time span for the lesson varied greatly from the field-testers’ actual time to teach the lesson.

Implications

This study raises several implications regarding the design and purpose of lesson plans available online for integrating digital primary sources into P-12 teaching, as well as the design and content of professional development whose purpose is to prepare teachers to integrate digital primary sources in their teaching. 

One implication involves the design of lesson plans about how to incorporate digital primary sources and their implementation in practice. McGlinn’s (2007) study showed a need for lesson plans, especially those that were teacher friendly, about how to implement digital primary sources. However, in this study, there were discrepancies between stated student groupings and the presentation methods for sharing primary sources with students, as noted in the lesson plans retrieved online, when compared to the actual reported uses by teachers who field-tested the lessons. These differences illustrate uncertainties about the extent to which teachers employ lesson plans in the first place.

This study also revealed that teachers seemed more interested in locating materials (e.g., digital primary sources) and lesson implementation ideas as opposed to detailed lesson plans about how to use these materials, a finding that seems to partly contradict McGlinn’s conclusions. Therefore, there is a need for further research investigating the fidelity of lesson plans developed for implementation of digital primary sources (which presumably incorporate best practices) and the actual implementation by teachers in other grade levels and contexts.

This finding also points to the potential need for creating lesson plans that incorporate differentiation strategies to address diverse teacher and learner contexts, particularly by providing differentiated processes. Differentiated processes might involve how students learn the content, such as through different student groupings or ways in which teachers engage students with the digital primary sources. However, these strategies may not address diverse contexts because the teachers who completed the field tests were mostly elementary school teachers.

This paper supported Fry’s (2010) and Tally’s and Goldenberg’s (2005) recommendations that future implementations of lesson plans using digital primary sources as a vehicle for learning should focus on materials selected and time allocated. These two items were the two major concerns presented by the teachers in this study. Consequently, careful consideration of the types, quantity, breadth, and depth of digital primary sources selected, as well as the amount of time allocated to implement them in a lesson, should be included in digital primary source lessons developed. Additional research in this area might also contribute to educators’ understanding of the types of online resources most needed and utilized by teachers. It may also be beneficial to explore the extent to which teachers employ exemplars of student work, teacher analyses of student learning, and lesson plan instructions when replicating a lesson plan.

Understanding what teachers need and how they utilize digital primary resources in practice might lead to more effective and efficient implementation of learning with digital primary sources. Such valuable information can help shape the design of online repositories of lesson plans and digital primary sources.

Another implication of this research involves the ways in which teachers introduced digital primary sources to students. The majority of field-testing teachers in this study used PowerPoint to present the digital primary sources. This finding demonstrates a need for teacher professional development at every educational level (e.g., preservice and in-service) to not only model best practices that employ technology but also seek ways to develop teachers’ technological pedagogical content knowledge (TPCK, or technology, pedagogy, and content knowledge [TPACK]). Koehler and Mishra (2008) defined TPCK as 

…the basis of effective teaching with technology…. It requires an understanding of the representation of concepts using technologies; pedagogical techniques that use technologies in constructive ways to teach content; knowledge of what makes concepts difficult or easy to learn and how technology can help redress some of the problems that students face; knowledge of students’ prior knowledge and theories of epistemology; and knowledge of how technologies can be used to build on existing knowledge and to develop new epistemologies or strengthen old ones. (pp. 17-18)

Teacher professional development and education should foster teachers’ TPACK so that more teachers move beyond the use of PowerPoint, when practical, to share primary sources with their students to do history. Clearly, examining primary sources using different media might result in divergent outcomes; therefore, teacher professional development should also model best practices for integrating digital primary sources in teaching (Hicks et al., 2004).     

As digital resources continue to proliferate and online education grows, professional development representing best practices to employ such repositories and their accompanying lesson plans are important factors to examine teachers’ confidence and skills needed to use digital resources in meaningful ways with students. Teachers may be struggling to learn new content related to the digital primary sources themselves, while also employing technology and, potentially, new inquiry techniques that help their own students interpret the primary sources. Video modeling of teacher instructions, scaffolding, and questioning might assist teachers in implementing lesson plans from such online repositories.

Professional development focused on digital primary sources may also improve by sharing lessons online, as well as documenting field tests of these lessons with real students and teachers. This approach could give teachers firsthand information about real-world implementations of digital primary sources and their accompanying lesson plans that might, in turn, improve their own implementations.  These online repositories might also help build collective evidence of successful practices and create supportive learning communities.

Finally, although this study focused on in-service teachers, it also has potential implications for preservice teacher education. First, many teacher education faculty members explore digital primary source materials and their lesson plans with their preservice teacher candidates. Teacher educators could model best practices associated with integrating digital primary sources in teaching, as well as  utilize (and modify if needed) accompanying lessons when they teach and share their own experiences field-testing the lessons. They could also review what worked well, what did not, and how using digital primary sources impacted student learning.

Educational and Scientific Importance

Numerous organizations have spent and will continue spending a considerable amount of resources providing primary source materials on the Web and developing lesson plans to help teachers use them. However, little research has examined the quality of the lessons let alone the results of their implementations. This study provides findings about a sample of social studies lessons that employed U.S. Library of Congress digital primary source materials. The findings may lead to further investigations of lesson plans centered on primary source materials available online, suggestions for maximizing their use (e.g., modeling of best practices), and the types of technology and professional development necessary to develop, teach, and evaluate such lessons.

References

Bolick, C. M. (2006). Digital archives: Democratizing the doing of history. International Journal of Social Education, 21(1), 122-134.

Bolick, C. M., Hicks, D., Lee, J., Molebash, P., & Doolittle, P. E. (2004). Digital libraries: The catalyst to transform teacher education. AACE Journal, 12(2), 213-233.

Brown, C., & Dotson, K. (2007). Writing your own history: A case study using digital primary source documents. TechTrends: Linking Research and Practice to Improve Learning, 51(3), 30-37.

Dutt-Doner, K. M., Cook-Cottone, C., & Allen, S. (2007). Improving classroom instruction: Understanding the developmental nature of analyzing primary sources. RMLE Online, 30(6), 1-20. Retrieved from http://www.amle.org/portals/0/pdf/publications/RMLE/rmle_vol30_no6.pdf

Farmer, J., Knapp, D., & Benton, G. M. (2007). The effects of primary sources and field trip experience on the knowledge retention of multicultural content. Multicultural Education, 14(3), 27-31. Retrieved from the Association for Middle Level Education website: http://www.amle.org/portals/0/pdf/publications/RMLE/rmle_vol30_no6.pdf

Friedman, A. (2006a). State standards and digital primary sources: A divergence. Contemporary Issues in Technology and Teacher Education, 6(3), 313-327. Retrieved from http://www.citejournal.org/vol6/iss3/socialstudies/article1.cfm

Friedman, A. M. (2006b). World history teachers’ use of digital primary sources: The effect of training. Theory and Research in Social Education, 34(1), 124-141.

Friedman, A., & Heafner, T. (2008). Finding and contextualizing resources: A digital literacy tool's impact in ninth-grade world history. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 82(2), 82-86.

Fry, M. L. (2010). Pedagogy and primary sources: Outcomes of the Library of Congress' professional development program, teaching with primary sources at Loyola. (Doctoral dissertation). Retrieved from ProQuest. (AAT 3403565).

Glaser, B.G., & Strauss, A.L. (1967). The discovery of grounded theory: Strategies for qualitative research. New York, NY: Aldine.

Hicks, D., Doolittle, P. E., & Lee, J. (2004). Social studies teachers’ use of classroom-based and web-based historical primary sources. Theory and Research in Social Education, 32(2), 213-247.

Koehler, M., & Mishra, P. (2008). Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.), Handbook of technological pedagogical content knowledge (TPCK) for educators. New York, NY: Routledge/Taylor & Francis Group.

Lee, J. K., Doolittle, P., & Hicks, D. (2006). Social studies and history teachers’ uses of non-digital and digital historical resources. Social Studies Research and Practice 1(3). Retrieved from http://www.socstrp.org

Levstik, L. (1996). Negotiating the history landscape. Theory and Research in Social Education, 24, 393-397.

Levstik, L., & Barton, K. (2001). Doing history: Investigating with children in elementary and middle schools (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

McGlinn, M. (2007). Using the "Documenting the American South" digital library in the social studies: A case study of the experiences of teachers in the field. Contemporary Issues in Technology and Teacher Education, 7(1). Retrieved from http://www.citejournal.org/vol7/iss1/socialstudies/article1.cfm

Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 189-208). Thousand Oaks, CA: Sage Publications.

National Council for the Social Studies. (2006). Curriculum standards for social studies: Expectations of excellence. Washington, DC: Author. Retrieved from http://www.socialstudies.org/standards/introduction

Schrum, K., & Schrum, L. (2009). Exploring the past with 21st-century tools. Social Studies and the Young Learner, 21(4), 24-27.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage Publications

Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage.

Swan, K., & Hicks, D. (2006-2007). Through the democratic lens: The role of purpose in leveraging technology to support historical inquiry in the social studies classroom. International Journal of Social Education, 21(2), 142-168.

Tally, B., & Goldenberg, L. B. (2005). Fostering historical thinking with digitized primary sources. Journal of Research on Technology in Education, 38(1), 1-21.

Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: SAGE Publications.

Author Note

Natalie B. Milman
George Washington University
email: nmilman@gwu.edu

Rhonda Bondie
Fordham University
email: rbondie@fordham.edu