Lee, C. J., & Kim. C. (2014). The second prototype of the development of a technological pedagogical content knowledge based instructional design model: An implementation study in a technology integration course. Contemporary Issues in Technology and Teacher Education, 14(3). Retrieved from https://www.citejournal.org/volume-14/issue-3-14/general/the-second-prototype-of-the-development-of-a-technological-pedagogical-content-knowledge-based-instructional-design-model-an-implementation-study-in-a-technology-integration-course

The Second Prototype of the Development of a Technological Pedagogical Content Knowledge Based Instructional Design Model: An Implementation Study in a Technology Integration Course

by Chia-Jung Lee, University of Georgia; & ChanMin Kim, University of Georgia

Abstract

This study presents a refined technological pedagogical content knowledge (also known as TPACK) based instructional design model, which was revised using findings from the implementation study of a prior model. The refined model was applied in a technology integration course with 38 preservice teachers. A case study approach was used in this implementation study. Data were collected from the participants’ discussion worksheets and lesson plans, along with associated artifacts and the researchers’ field observation notes. Data analysis results revealed that (a) preservice teachers’ had an entry-level understanding of TPACK through discussions on the meaning of TPACK and evaluations of technology-integrated teaching examples; (b) designing several technology-integrated lesson plans improved preservice teachers’ teaching-related knowledge and facilitated their TPACK learning; and (c) preservice teachers’ use of technology was more teacher centered than student centered. Findings, suggestions, and future research possibilities are also discussed.

A previous study (Lee & Kim, 2014) pioneered the first prototype (Prototype I) of an instructional design (ID) model based upon technological pedagogical content knowledge (or technology, pedagogy, and content knowledge [TPACK]) to improve preservice teachers’ knowledge of technology integration. This study presents the refined ID model (Prototype II) and reports on its implementation and results.

The Prototype I model—IDDIRR1 (an acronym for the stages Introduce, Demonstrate, Develop, Implement, Reflect, and Revise)—has been applied to and evaluated in a multidisciplinary technology integration class, in which preservice teachers were from diverse majors. That experience informed the design and implementation of the current model—IDDIRR2.

The implementation of IDDIRR1 yielded two key findings. First, the preservice teachers showed improvement in their technological knowledge, pedagogical knowledge, and content knowledge, while no explicit evidence of improvement was discovered in integrated knowledge (technological pedagogical knowledge [TPK], technological content knowledge [TCK], and TPACK) in the lesson plans that they designed and practiced for teaching (see Appendix A).

It should be noted that TPACK is typically considered an inherently integrated concept and cannot be meaningfully deconstructed into separate components. However, when examining how teachers developed the integrated understanding, knowledge and skills associated with TPACK, it is meaningful to see how teachers are progressing along a number of enabling dimensions, which is an underlying assumption of this study.

Another finding was that preservice teachers’ self-assessed TPACK did not reflect their actual performance using TPACK on technology-integrated teaching tasks (e.g., technology-integrated lesson plans and teaching practice). The participants perceived that they were capable of using technology effectively in teaching activities; they rated their TCK, TPK, and TPACK either good or very good (i.e., either 4 or 5 out of 5). However, their understanding of those domains was not evidenced in their technology-integrated teaching tasks.

The survey used in IDDIRR1 was made based on four surveys: (a) Survey of Pre-service Teachers’ Knowledge of Teaching and Technology (Schmidt et al., 2009), (b) Survey of Technological Pedagogical and Content Knowledge (Sahin, 2011), (c) Assessing Students’ Perceptions of College Teachers’ PCK (Jang, Guan, & Hsieh, 2009), and (d) TPACK in Science Survey Questions (Graham et al., 2009).

The implementation and findings from IDDIRR1 guided this follow-up study, emphasizing that teacher training programs should critically take preservice teachers’ teaching-related backgrounds into consideration when improving their technology integration. Activating a teacher’s pedagogical and content knowledge when introducing technology appears to be an essential aspect of developing TPACK. Without an adequate knowledge base with regard to pedagogy and content, teachers tend to experience difficulty applying appropriate methods to teach certain types of content (Shulman, 1987).

IDDIRR2 included activities aiming to enhance preservice teachers’ pedagogy-related understanding so as to facilitate their TPACK acquisition. The goal of this study was to determine to what extent IDDIRR2 develops preservice teachers’ TPACK. The specific objectives of this study were to (a) develop Prototype II of the TPACK-based ID model, IDDIRR2, based on the findings of IDDIRR1, (b) apply IDDIRR2 to a technology integration course to investigate its effects on the improvement of TPACK, and (c) provide suggestions for the revision of Prototype III and future research possibilities. The following questions were investigated:

  • What are the effects of IDDIRR2 of the TPACK-based ID model on preservice teachers’ TPACK?
  • How do the effects of IDDIRR2 inform the revision of the model?

Theoretical Framework

The IDDIRR2 model was grounded in the TPACK framework (Mishra & Koehler, 2006), the learning-by-design approach (Kalantzis & Cope, 2005), ID models (Gagné, Wager, Golas, & Keller, 2005; Gustafson & Branch, 2002), and design-based research (DBR; DBR Collective, 2003; Reeves, 2006; Van den Akker, Gravemeijer, McKenney, & Nieveen, 2006).

TPACK is a theoretical framework that comprises seven types of knowledge: content knowledge, pedagogical knowledge, technological knowledge, pedagogical content knowledge (PCK), TCK, TPK, and TPACK. This framework emphasizes the integrated knowledge—TPACK—as opposed to considering the knowledge in isolation (Koehler & Mishra, 2009). Teachers with TPACK understand how to present specific content with appropriate pedagogy and technology to effectively support student learning.

The learning-by-design approach has been advised to improve teachers’ TPACK (Koehler & Mishra, 2005). This approach encourages teachers to take the role of designers to develop teaching artifacts for students’ learning needs and allows teachers to experience the complexity of teaching environments (Koehler, Mishra, & Yahya, 2007). In order to maximize the use of the learning-by-design approach to improve preservice teachers’ TPACK in a multidisciplinary course, we drew on the systematic elements of an ID model that integrated learning by design and TPACK to develop IDDIRR2.

IDDIRR2 was the second iteration of DBR efforts to promote preservice teachers’ effective technology integration in consideration of the findings from the IDDIRR1 implementation. Studies that apply DBR involve the iterative design, development, implementation, and evaluation of interventions (e.g., programs, strategies, materials, etc.; Plomp, 2007). Thus, the findings of this implementation study provide suggestions for the revision for the next iteration of the model (Prototype III).

Design Guidelines and the Revised Model

Design Guidelines

One critical characteristic of DBR is to conduct “rigorous and reflective inquiry to test and refine innovative learning environments [interventions] as well as to define new design principles” (Reeves, 2006, p. 95). Design principles serve as guidelines for researchers to develop and test plausible solutions or interventions for research problems. The findings from IDDIRR1 implementation suggested that preservice teachers’ teaching-related knowledge should be developed to improve their TPACK, which led to revisions in design guidelines of IDDIRR2. The following list presents revised guidelines that involve multiple teaching-related elements to facilitate preservice teachers’ TPACK learning:

Guideline 1. Explicit, systematic procedures should be included to provide practical solutions for teacher training programs to enhance preservice teachers’ TPACK.

Guideline 2. Discussion of definitions and teaching examples relevant to TPACK should be included to enhance the understanding of TPACK domains.

Guideline 3. Development of several lesson plans integrating various technological tools should be included to enhance the connection of technology to specific content and pedagogy.

Guideline 4. Implementation of a technology-integrated lesson plan should be included to help transfer knowledge to teaching practice.

Figure 1 shows the comparison of the design guidelines of the two prototypes. Guideline 1 was retained in IDDIRR2, since systematic procedures are crucial elements of an effective ID model (Gustafson & Branch, 2002). Changes to strengthen preservice teachers’ teaching-related knowledge for their TPACK acquisition were emphasized in Guidelines 2, 3, and 4 in IDDIRR2.

figure 1

Figure 1. Design guidelines change in the TPACK-based ID models.

The Revised Model—IDDIRR2

Responding to the revised design guidelines, IDDIRR2 comprises three major instructional steps: Understand TPACK, Engage in TPACK, and Practice TPACK (see Appendix B).

Step 1, Understand TPACK. Step 1 responds to Guideline 2. The purpose is to develop preservice teachers’ understanding of technology integration as well as to enhance their teaching-related knowledge. Instead of being told the definitions of TPACK by the instructor, preservice teachers actively discuss and search for the meanings of TPACK. They also create examples for the domains of TPACK so as to become familiar with this concept. In this step, videos that include technology-integrated teaching examples are also presented. Preservice teachers evaluate and compare the examples based on their understanding of TPACK. Activities in this step not only introduce TPACK to preservice teachers but also provide them with hands-on activities to explore TPACK.

Step 2, Engage in TPACK. In response to Guideline 3, this step aims at improving teaching-related knowledge by engaging preservice teachers in designing several technology-integrated teaching activities. They create technological artifacts as well as lesson plans for each of the technologies that they have learned in class. In other words, learning about and using technological tools go hand-in-hand in promoting technological knowledge. However, the design of lesson plans requires preservice teachers to consider other domains of TPACK, such as students’ needs and content characteristics with technology.

In Step 2, preservice teachers start with familiarizing themselves with a technological tool that has been introduced in the class by creating an artifact using the tool (e.g., Technology A). Then, they carry out the Learning TPACK by Design activity, in which three minor activities are included:

  1. Develop a TPACK-based lesson plan by integrating specific technology (e.g., Technology A)—the development of a lesson plan can be carried out individually or in groups, depending on time management, class size, or other classroom factors.
  2. Gain feedback from peers and the instructor.
  3. Reflect on and Revise the lesson plan accordingly.

The same activities are repeated after the learning of a new technological tool (e.g., Technology B, C, etc.). This step is supposed to help preservice teachers better understand the relationships of technology to content and to students’ learning.

Step 3, Practice TPACK. Step 3 responds to Guideline 4, and its purpose is to transfer preservice teachers’ TPACK to practice. This step also comprises Learning TPACK by Design, but there are two differences compared to Step 2. One is that Step 3 requires preservice teachers to incorporate several technologies that they have either learned in class or explored by themselves into the final lesson plan. The other difference is that preservice teachers are required to implement their final lesson plans. The Implement activity can engage preservice teachers in an authentic teaching environment and is intended to help them realize technology integration at a deeper level.

Implementation Study

Methodology

A case study framework was applied to this study in order to determine if and how the revised model would impact preservice teachers’ TPACK. A case study approach can guide researchers in (a) understanding complex social phenomena and gaining a holistic view of the phenomena, and (b) developing an in-depth description and analysis of the phenomena (Yin, 2009). The case study approach was applied so as to acquire a comprehensive understanding of how the intervention (the IDDIRR2 model) works in teacher education (a complex social phenomenon).

Context and Participants

The IDDIRR2 model was implemented in the two sections of a technology integration course for preservice teachers taught by one of the researchers during spring semester 2012 at a large university in the southeastern United States. The 3-hour credit course was 16 weeks long, and the class met three times per week for an hour. Nineteen students enrolled in each section (Sections 1 and 2) and participated voluntarily in this study (34 female, 4 male). The age of one participant was 36, while the ages of the rest of the participants ranged from 19 to 24. The average of the participants’ ages was 21 (SD = 3.06).

One participant was an in-service kindergarten teacher and an undergraduate student. Two participants were science and mathematics education majors. The rest of the participants were from diverse majors, as follows: advertising, animal science, chemistry, child and family development, communication sciences and disorders, consumer economics, health promotion, international affairs, public relations, psychology, and recreation and leisure studies. Overall, 10 out of the 38 participants had taken or were taking education-related courses when they took this course. Since most of the participants had no teaching-related background, this context is similar to the context of the IDDIRR1 model.

The following technological tools were taught in the course:

  1. Communication and collaboration tools (e.g., Google Docs, in2Books, podcasting tools, the GLOBE program, blogging tools, Delicious).
  2. Graphic software (e.g., floorplanner)
  3. Video making tools (e.g., Microsoft Photo Story, iMovie, Slowmation).
  4. Image editing tools (e.g., Picnik, Picasa, etc.)
  5. Concept-mapping tools (e.g., Inspiration and Bubbl.us)
  6. Google Site.

Procedures

The course was designed to follow the three steps of the IDDIRR2 model. Participants were informed of the course goals and scheduled tasks during the first week. They were also informed that they would not only learn about technology but also integrate it into teaching contexts. Every participant created a Google website during the first week, where they submitted all the course assignments, including technological artifacts, lesson plans, and class discussion forms.

Step 1, Understand TPACK. In Week 5, the instructor used a TPACK-introductory video (view TPACK 101 at http://vimeo.com/16291486), in which the seven domains of TPACK were explicitly explained. Then, the participants themselves formed small discussion groups with 3-5 people each (both sections included five groups, Groups L1 to L5 in Section 1 and Groups M1 to M5 in Section 2) to work on TPACK Worksheet-1, in which questions were designed to build the participants’ knowledge base of TPACK (e.g., “Define CK using your own words”).

Each group discussed their understanding of technological knowledge, pedagogical knowledge, and content knowledge by giving definitions and creating examples representing the three domains. TPACK Worksheet-1 included only the questions about technological knowledge, pedagogical knowledge, and content knowledge because the implementation study results of IDDIRR1 showed that the participants with limited teaching-related knowledge had difficulties in understanding all of the domains of TPACK at this stage.

In Week 6, two class periods were spent to help preservice teachers learn more about integrated knowledge of TPACK (focusing on TCK, TPK, and TPACK). The concepts were introduced using two videos. Video 1 (presented in class period 1), an example of teaching with effective TPACK integration, showed an elementary teacher sharing her story about her use of technology (e.g., online interactive map) to help students learn abstract concepts (e.g., cardinal directions) using student-centered pedagogy (e.g., hands-on activities).

In contrast, Video 2 (presented in class period 2), the non-example, presented a teacher who used an online game to introduce living and nonliving things without utilizing the effective affordance of online games. After watching both videos, each group worked on TPACK Worksheet-2 to discuss questions regarding integrated knowledge of TPACK. Examples of the questions included “In what activities were students engaged when using technological tools (TPK)?” and “Why is the technological tool used by the teacher in the video helpful for that topic (TCK)?”

Step 2, Engage in TPACK. From Week 6 to Week 13, 13 class periods were spent engaging the participants in further development of their understanding of and ability to meaningfully deploy TPACK. This step focused on providing opportunities to the participants to create a lesson plan for each of the technologies that they had learned in this class so as to experience its educational potentials. Based on the activities of Step 2, for example, the instructor introduced Microsoft Photo Story first, and then every participant created a digital artifact using Photo Story. Next, they engaged in Learning TPACK by Design activities. They were asked to develop a lesson plan incorporating Photo Story and discuss the lesson plan with peers or the instructor to gain feedback, and they had to reflect on and revise the lesson plan thereafter.

In total, the participants engaged in three projects that required them to go through the activities of Step 2 six times. The three projects were as follows:

Project 1 (individual work): Each participant explored one of the following technologies—blogging tools, podcasting, the GLOBE Program (http://www.globe.gov), or in2Books (http://in2books.epals.com)—and developed a lesson plan integrating the tool.

Project 2 (individual work on the digital artifact and collaborative work on the lesson plan): Each participant created an artifact using Photo Story and then found peers to develop a lesson plan integrating the tool. The participants worked with peers to design a lesson plan because they were also given opportunities to learn from peers (five groups were formed in each class section, Groups N1 to N5 in Section 1 and Groups O1 to O5 in Section 2).

Project 3 (individual work): Each participant created four digital artifacts using four technological tools (e.g., Slowmation, image editing tools, concept mapping tools, online games, etc.) and developed four lesson plans that integrated the created digital artifacts, respectively.

Step 3, Practice TPACK. From Week 13 to Week 15, nine class periods were spent engaging the participants in the Step 3 activities. The participants themselves created small groups with three to five people each to work on their final projects (five groups were formed in each class section, Groups P1 to P5 in Section 1 and Groups Q1 to Q5 in Section 2). Every group developed a lesson plan incorporating several technologies that they had learned in class and also explored by themselves. They had to create corresponding digital artifacts mentioned in their lesson plans. Each group also created a student website (see Figure 2 for an example) for later teaching purposes, in which teaching activities and digital artifacts were inserted.

figure 2 Figure 2. Examples of student website.

Second, groups discussed teaching ideas with the instructor to gain feedback. Third, every group taught the class using the student website for 30-35 minutes, and all the group members were required to teach (implement). The rest of the classmates acted as students, and they gave feedback to the teaching group. Finally, each group reflected on the feedback and specified what parts of the original lesson plan should be revised. Figure 3 illustrates the activities of the three steps.

figure 3 Figure 3. TPACK learning activities in three steps of the model.

Data Collection

Data were collected based on the three steps of the IDDIRR2 model and included (a) TPACK discussion worksheets, (b) individuals’ and groups’ lesson plans and corresponding digital artifacts, and (c) the researchers’ field observation notes. Table 1 presents the data sources along with the steps of the model and relevant research questions.

Table 1
Data Sources and Data Analysis

Research Questions
Steps of the Model
Data Sources
Data Analysis
RQ1: What are the effects of IDDIRR2 of the TPACK-based ID model on preservice teachers’ TPACK? Step 1:Understand TPACK TPACK discussion worksheets,
researchers’ field observation notes
Description and analysis
Step 2: Engage in TPACK Individuals’ and groups’ lesson plans and corresponding digital artifacts,
researchers’ field observation notes
Deductive data analysis (the LoU framework),
quota sampling
Step 3: Practice
TPACK
Groups’ lesson plans and corresponding digital artifacts,
researchers’ field observation notes
RQ2: How do the effects of IDDIRR2 inform the revision of the model?

N/A

Findings of RQ1 Analysis of the findings of RQ1

Data Analysis

In Step 1 the approaches description and analysis (Wolcott, 1994) were applied. Simons (2009) explained description as “staying close to the data as originally recorded” (p. 121), and analysis as “moving beyond the purely descriptive to systematically identify key factors and relationships, themes and patterns from the data” (p. 121). These two approaches are not mutually exclusive and can be blended to suit the research case.

Data collected in Step 1 were the researcher’s observation of the participants’ TPACK discussion and the groups’ written responses to TPACK Worksheets. Description can provide a picture to readers regarding the researcher’s observation. Analysis can help provide analytic data that transform the participants’ responses to TPACK Worksheets into meaningful patterns and relationships.

In Step 2 and Step 3 deductive reasoning (Mayring, 2000) was applied, and the Levels of Use framework (LoU; Hall, Dirksen, & George, 2006) was used as a coding scheme for the deductive reasoning process. Each of the lesson plans was given one of the eight levels from the LoU framework to determine its level of applying innovation (technology integration). Appendix C lists the description of each LoU. The LoU framework has been applied to measure the extent to which teachers actually use innovations, such as new curriculum, teaching materials, and technologies (Ellsworth, 2000; Kim, Kim, Lee, Spector, & DeMeester, 2013). Since this research attempted to identify the effects of the model on improving preservice teachers’ TPACK, LoU was considered suitable to assess their TPACK learning that was observed in technology integration artifacts (i.e., lesson plans).

In Step 2, quota sampling (Castillo, 2009) was also applied for the efficiency of the data analysis. The participants produced 200 lesson plans in this step. We used quota sampling to segment the participants into three performance groups (High, Middle, and Low) that allowed us to analyze their lesson plans from organized perspectives.

Reality (Internal Validity)

Strategies of audit trail, data triangulation, and colleague examination were applied to promote the internal validity of this study. First, audit trail was used to establish the rigor of the study. Merriam (1988) explained, “In order for an audit to take place, the investigator must describe in detail how data were collected, how categories were derived, and how decisions were made throughout the inquiry” (p. 172). This study elaborated the process of how this prototype of the TPACK-based ID model was developed by incorporating the suggestions from the previous prototype, how the design guidelines were transformed into practical activities, and how the collected data were analyzed by the coding scheme so as to allow the interpretation of the participants’ TPACK acquisition.

Data triangulation was deployed to secure the research reality. Data triangulation refers to the use of multiple sources of data to verify the emerging findings (Merriam, 1995). In this study, the participants’ learning process was recorded and evidenced in various types of data, including their discussion worksheets, lesson plans, and digital artifacts. The field observation notes taken by the instructor-researcher were also used to triangulate the data collected from the participants.

Colleague examination was used as a confirmatory strategy. Colleague examination refers to “asking peers or colleagues to examine the data and to comment on the plausibility of the emerging findings” (Merriam, 1995, p. 54-55). Accordingly, we (the researchers) consulted each other regularly for 2 years about the development of the study to ensure a rigorous process to derive valid knowledge and findings.

Findings

In response to Research Question 1 with respect to the effects of the IDDIRR2 model, the following findings are described according to the three steps of IDDIRR2.

Step 1: Entry-Level Understanding of TPACK Was Observed in Discussion Worksheets

Two TPACK worksheets were used to help the participants build the knowledge base of TPACK in this step. The participants found three to five peers to discuss questions on TPACK Worksheets 1 and 2 (five groups were formed in each section of the course, Groups L1 to L5 in Section 1 and Groups M1 to M5 in Section 2).

TPACK Worksheet-1. In this worksheet, each group was required to discuss the definitions and create examples for technological knowledge, pedagogical knowledge, and content knowledge. All of the groups completed the worksheet in 8-12 minutes. During the discussion process, the groups actively searched for the provided online materials to respond to questions. The instructor-researcher walked around the class to see if any group had problems. No group asked questions or demonstrated difficulties in responding to the questions. At the end of the activity, groups shared their responses with the class.

Their responses were mostly accurate. For instance, the examples of content knowledge created by the groups included proof of mathematics, literary interpretation, and “an English teacher teaching about citations knows how to create proper citations.” However, there were some minor aspects of their responses in need of correction. For example, one group thought that “a lesson plan” belongs to the pedagogical knowledge domain. The instructor explained that the response was too general, because a lesson plan is supposed to integrate content knowledge, pedagogical knowledge, and/or technological knowledge. Overall, groups’ learning process and responses to TPACK Worksheet-1 showed that they acquired technological knowledge, pedagogical knowledge, and content knowledge.

TPACK Worksheet-2. In this worksheet, groups started to discuss integrated knowledge of TPACK (e.g., TCK, TPK, etc.). This worksheet had two parts of TPACK-related questions designed based on the two teaching videos presented to the class. Students spent two class periods completing the two parts of Worksheet-2, respectively.

  • TPACK Worksheet-2, class period 1.Compared to TPACK Worksheet-1, groups demonstrated difficulties answering the questions. In class period 1 Video 1, which demonstrates an effective TPACK teaching example, was presented for about 8 minutes, and then the groups spent 30 minutes working on the first part of TPACK Worksheet-2 (a few groups continued to work even after the class ended). The instructor observed that groups had difficulties interpreting how the technologies used in the video represented the content (TCK) and facilitated students’ learning (TPK). The instructor provided an explanation by comparing the good example shown in Video 1 (e.g., visualizing cardinal directions in online maps) to traditional teaching methods (showing directions on a paper map) to indicate the difficulty in teaching if there is no support of technology. With support from the instructor, all the groups completed the first part of the worksheet, in which their explanations indicated how the technologies used in the good example helped students learn the content (see Appendix D for selected responses).
  • TPACK Worksheet-2, class period 2.In the second class, Video 2 was presented for 7-9 minutes (the non-example teaching). The groups then used the rest of the class time, about 30 minutes, completing the second part of TPACK Worksheet-2. Groups demonstrated fewer difficulties responding to questions because the knowledge acquired from class period 1 could be applied to this activity. The analysis of the worksheet responses indicated that the groups’ responses were based on the TPACK concepts, but the responses were imprecise or superficial. Only three groups (L1, L4, and M5, 30% of the participating groups) provided responses to the TCK question considering content characteristics, and two groups (M3 & M5, 20% of the participating groups) provided responses to the TPK question considering students’ learning.

Taking the groups’ responses to the TCK question as an example, as shown in Appendix D, the response from Group 5 in Section 1 (L5) was imprecise in that the group did not consider critically the necessity of applying the technology (online games) to the specific content (living and nonliving things). The responses from Group 3 (M3) and Group 4 (M4) in Section 2 were superficial in that they did not provide alternative methods to improve the non-example technology integration.

Only 30% of the participating groups, such as Group 5 in Section 2 (M5), considered the relationship between the content and technology as shown in the following response: “Online games are helpful, but not necessary. For this topic, it might be more beneficial to actually go outside instead of using the [online] game.”

TPACK Worksheet-2 also asked the groups to compare the two teaching videos in terms of the quality of technology integration by giving scores (1 to 10, 10 being the highest) as well as a rationale for their rating. The average score of Video 1 was 8.2 and that of Video 2 was 4.1, which indicated that groups could identify the quality of technology integration of the two teaching examples. In the explanation, all the groups provided responses from the perspectives of students’ learning and content characteristics (TPACK) to acknowledge the better quality of Video 1. The last part of Appendix D shows the examples from the groups’ responses.

Overall, the groups accurately described how the technology used in Video 1 (the effective teaching example) supported students’ content learning. The goal of Step 1 was achieved in that the participants demonstrated a basic understanding of TPACK by properly explaining how a teacher integrated TPACK in teaching practice. However, the groups demonstrated insufficient knowledge in evaluating the suitability or necessity of the applied technology in Video 2 (a non-example).

Step 2: TPACK Understanding Was Enhanced and Observed in Lesson Plans

During this step, preservice teachers created technology-integrated lesson plans in order to enhance their TPACK. Two hundred lesson plans were created, of which 190 lesson plans were individual created (each 38 participants developed five lesson plans) and 10 lesson plans were group created. Quota sampling (Castillo, 2009) was used to segment the participants into three performance groups (High, Middle, and Low).

Based on the levels assigned to the participants’ first lesson plans (Lesson Plan 1) using LoUs, the participants were segmented into one of three performance groups—High (Level 3), Middle (Level 2), and Low (Level 1). Then, five to six participants were selected from each of the performance groups. This segment allowed the interpretation of the data in consideration of the participants’ (initial) abilities.

Table 2 summarizes the LoUs given to the individuals’ and the groups’ lesson plans created in Step 2. Letters A to F refer to the six participants selected from each of the performance groups, of which participants A, B, C were from Section 1 and participants D, E, F were from Section 2 of the class. Letters N1 to N5 refer to the five groups formed in class Section 1, and O1 to O5 refer to the five groups formed in class Section 2.

Table 2
Lesson Plans Rated Using the LoU Framework

 


Project /Mode
Individuals
Groups
High Performance
Middle Performance
Low
Performance
N1
N2
N3
N4
N5
O1
O2
O3
O4
O5
A
B
C
D
E
F
A
B
C
D
E
F
A
B
C
D
E
Project 1
(Individual plan)
N/A
Lesson Plan 1
4b
3
3
4b
3
3
2
2
2
2
2
2
1
1
1
1
1
Group Mode
3
2
1
Project 2
(Group plan)
N/A
1
1
3
3-4b
3
1
2
5
3
3
Lesson Plan 2
Project 3
(Individual plan)
N/A
Lesson Plan 3
3
4b
3
4b
3
1
2
2
3
2
1
3
1
1
1
2
1
Lesson Plan 4
4b
3
3
3
4b
2
2
2
3
2
4b
3
3
1
2
1
2
Lesson Plan 5
3
3
2
2
3
2
2
1
3
3
3
2
1
2
2
0
3
Lesson Plan 6
2
2
2
2
2
2
2
2
2
2
2
2
2
0
2
1
2
Individual Mode
3
3
2,3
2
3
2
2
2
3
2
N
2, 3
1
1
2
1
2
Group Mode
3
2
1
Notes: A-F refers to six individual participants. A to C were from Section 1 and D to F were from Section 2 of the course.N1 to N5 refers to the five groups in Section 1, and O1 to O5 refers to the five groups in Section 2. Digital numbers refer to a level assigned to individually or group- created lesson plans, which are rated by LoU: Level 1-Orientation; Level 2-Preparation; Level 3-Mechanical use; and Level 4b-Refinement.

 

Project 1. Project 1 was an individually created lesson plan (Lesson Plan 1). Participants integrated one technological tool that they had learned from the course up to that point (e.g., choosing from blogging tools, podcasting tools, etc.). However, they were also encouraged to explore an additional technology that was not covered in class and integrate it with the required tool into the lesson plan.

The participants whose projects were rated at LoU Level 3 or higher were grouped into High Performance, in which the technology was used in student-centered ways (Level 4b) or to promote a higher level of cognitive processing (Level 3). The participants whose projects were rated at Level 2 were grouped into Middle Performance, in which technology was used to support content understanding or a lower level of cognitive processing. The participants whose projects were rated at Level 1 were grouped into Low Performance, in which technology was used to deliver teachers’ lectures. Figure 4 illustrates the levels given to the participants’ Project 1 among the three performance groups, and Table 3 shows the examples of Lesson Plan 1 created by Participant A from each of the performance groups.

figure 4

Figure 4. Levels given to the participants’ Project 1 in three performance groups. (A-F refers to Participant A to Participant F. 0-6 refers to the level assigned to each participant’ lesson plan using LoU: Level 1 – Orientation, Level 2 – Preparation, Level 3 – Mechanical Use, and Level 4(b) – Refinement.

 

Table 3
Examples of Lesson Plan 1 Selected From the Three Performance Levels


Participants 
Selected Lesson Plan 1
LoU Level
LA
(Low Performance group, Participant A)
I think that Photo Story or iMovie would be a very beneficial tool to teach middle schoolers about track and field sports, which includes running and my experience… Because sports are so popular in our day, a movie would provide students with a visual of how my own experience as a runner looks like. I could show that video and the pictures in my class as an introduction to my class (and give them a URL to study from home). I believe this would be a beneficial way for students to learn about a personal story as well as get information about track and running.
1
(Orientation)
MA
(Middle Performance group, Participant A)
I plan on introducing my lesson to my students by demonstrating how to use Mapquest [an online web mapping tool] and Google Maps. I will give them two different points on the map and show them how to put the information into the computer and how to interpret what they see on the map. If they need to get directions to a certain point, I will demonstrate how to use Mapquest and see the different turns it takes to get to their destination and the distances between each turn. Students will then be given a starting point and a destination and will then have to answer questions about distance and time.
2
(Preparation)
HA (High Performance group, Participant A) I plan to teach eighth grade Visual Arts…Students will be using blogging to communicate the meaning behind their art work. They will also use blogging as an organizational source, allowing them to archive their work to refer back to later and trace their growth as an art student over the course of the semester… Blogging is also a tool for collaboration because students will use it to be able to comment on the blogs of their peers while they are defending their own work on their personal blogs… Power Point is to be used to display a combination of
the artwork of the whole class. At the end of every project, students will each make a Power Point slide displaying their artwork.
4b (Refinement)


Project 2. In Project 2, the participants found two to four peers by themselves to develop collaboratively Lesson Plan 2, which integrated Photo Story. Five groups were formed in each section of the course (Groups N1 to N5 in Section 1 and Groups O1 to O5 in Section 2). Table 2 lists LoU evaluations for each group’s lesson plan. Three groups were rated at Level 1 (N1, N2, and O1), one group was rated at Level 2 (O2), four groups were rated at Level 3 (N3, N5, O4, and O5), one group was rated at Level 3-4b (N4), and one group was rated at Level 5 (O3).

Table 4 shows the sample lesson plans from Group 1 (N1), Group 3 (N3), and Group 4 (N4) from Section 1 and Group 3 (O3) from Section 2 that were rated at Level 1, Level 3, Level 3-4b, and Level 5, respectively. The three groups whose lesson plans were rated at Level 1 tended to use Photo Story to present the content or support teachers’ lectures. For example, Group N1 tended to use Photo Story to show students pictures regarding scientific landforms.

Table 4
Selected Examples From Lesson Plan 2

 


Group 
Selected Lesson Plan
LoU Level
N1 (Group 1 in Section 1) I think that Photo Story or iMovie would be a very beneficial tool to teach middle schoolers about different landforms. Because we can only travel so much, a movie would provide students with a visual of what these different land forms look like. I could show these pictures to my class every day for a week (and give them a URL to study from home), and then show them the same pictures during the test. I believe this would be a beneficial way for students to learn the pictures associated with landforms, instead of merely learning a word.
1 (Orientation)
N3 (Group 3 in Section 1) Topic: Novels in American Literature
Subject: Book Reports
Grade Level: 10th
Teaching Process: The students will be asked to read a novel of their choice (from a given list), and instead of relying on traditional methods such as essays, the students will be required to make their own Photo Story on a given topic. The topics can be about the characters of the book, the themes, or any other approved topic.
3
(Mechanical use)
N4 (Group 4 in Section 1) Topic: Routines
Subject: Gymnastics
Grade Level: 7th -10th
Teaching Process:
1) We can use iMovie to find YouTube clips of different skills. From there we can make a movie of the YouTube clips, and we can show the full movie to the gymnasts, so they can get new ideas of different skills and see how they are performed.
2) We can also use iMovie to put clips of our gymnasts actually performing their new routines into normal motion and then in slow motion. From there we can create a full length video and have a movie practice with our gymnasts.
3) We can then put the video into iMovie and have a movie day practice, showing the gymnasts what they’re doing right and wrong in their routines.
3-4b
(Mechanical use-Refinement)
O3 (Group 3 in Section 2) We plan on teaching Pre-K and the topic for this week is the letter R. We can show different video clips of the letter R and the associated words that start with R… We can also narrate the video to explain it in our own words. By using our own narration, we can get the students involved in the activities. For example, “Say the letter R.” Show them a picture and say, “What do you think this is? Using Photo-story, they [students] can use it at home and actually get practice so that they are well-educated on the subjects taught in class… Also, they are learning how to narrate their own story. We can also let them narrate additional words that start with R for homework. This tool allows them to think critically about their surroundings…The children can go home and listen to these stories as bed time stories with their parents or they can listen and watch the videos for fun of their favorite videos. On top of that, I can also use photo-story to assign homework assignments that the children and their parents can do at home…Before sending home assignments, we plan on having a parent workshop to teach them how to use photo story.
5
(Integration)

 

The four groups whose lesson plans were rated at Level 3 attempted to have students create a slideshow using Photo Story to demonstrate and reflect on their learning. For example, Group N3 required students to make a digital story after reading a novel to present the characters in the book.

Group N4 that was rated at Level 3-4b planned to videotape the gymnasts’ actual performance of bar routines in order to help students solve authentic problems that gymnasts often confront (Level 4b). However, the lesson plan was rated 3-4b because it was not beyond the description of Level 3, in which teacher-centered strategies dominated the learning process. The lesson plan from Group O3 was rated at Level 5, in that students’ use of technology went beyond the classroom for authentic issues (integrating real life surroundings) and had to collaborate with parents.

Project 3. In Project 3, the participants used four technological tools to create four digital artifacts and four associated lesson plans (Lesson Plans 3 to 6). In Lesson Plan 3, the participants chose one tool among Photo Story, Slowmation, or blogging tools. In Lesson Plans 4, 5, and 6, the participants were required to integrate an image editing tool, a concept-mapping tool, and an online game, respectively. Table 2 shows the LoUs given to the participants’ four lesson plans in Project 3. Levels given to the four lesson plans in the High Performance group ranged from Level 1 to Level 4b, while the mode of the group levels was Level 3. Levels given to the four lesson plans in the Middle Performance group ranged from Level 1 to Level 4b, while the mode of the group levels was Level 2. Levels given to the four lesson plans in the Low Performance group ranged from Level 0 to Level 3, while the mode of the group levels was Level 1.

The levels given to the four lesson plans in Project 3 were also compared to the levels given to Lesson Plan 1 in Project 1 in order to see if there was any improvement in the quality of their work. Figure 5 illustrates the changes in the levels of technology integration comparing Project 1 to Project 3 among the three performance groups.

figure 5 Figure 5. Changes of levels of technology integration in comparison of Project 1 to Project 3. (A-F refers to Participant A to Participant F. 0-6 refers to the level given to each participant’s lesson plan using LoU: Level 1 – Orientation, Level 2 – Preparation, Level 3 – Mechanical Use, and Level 4(b) – Refinement.

As shown in Figure 5, in the High Performance group Participants A, C, and D (labeled HA, HC, and HD) maintained their performance, in that the highest level given to their Project 3 was the same as that given to their Project 1 (e.g., HA’s Lesson Plan 4 was rated at Level 4b, the same level as her Lesson Plan 1). Participant B (HB) and Participant E (HE) improved their performance, in that the highest level given to their Project 3 was Level 4b, while their Project 1 was rated at Level 3. However, the performance of Participant F (HF) was worse (Level 3 in Project 1 compared to Level 2 in Project 3). Overall, 5 out of 6 participants in the High Performance group either improved (2 participants, 33%) or maintained (3 participants, 50%) the level of technology integration.

In the Middle Performance group, Participants A and B (HA and HB) maintained their performance, in that the highest level given to their lesson plans in Project 3 was Level 2, which was the same level given to their Project 1. Participants C, D, E, and F (labeled MC, MD, ME, MF) improved their performance in that the highest levels given to their lesson plans were Level 3 or Level 4b (compared to their Project 1 rated at Level 2). Overall, 4 out of 6 participants in the Middle Performance group improved the level of technology integration (67%).

In the Low Performance group, all 5 participants improved the level of technology integration. The highest levels given to the 5 participants’ lesson plans were Level 3 (labeled LA and LE) or Level 2 (LB, LC, and LD), while the level given to their Project 1 was Level 1.

Step 3: TPACK Understanding Was Not Utilized in the Teaching Practice

In this step, the participants themselves found two to four peers to form a group to work on the final project—developing a technology-integrated lesson plan and implementing a section of the plan in class for 30 minutes. Five groups were formed in both class sections (Groups P1 to P5 in Section 1 and Groups Q1 to Q5 in Section 2). All 10 groups integrated various technologies that they had learned in class to present the learning content (TCK), such as Photo Story, image editing tools, concept maps, and online games. They also explored and integrated online videos, animation tools, and digital photos to support content instruction.

The groups’ lesson plans were evaluated using LoU. Two groups’ lesson plans (P5 and Q2) were rated at Level 3, and the rest of the groups were rated at Level 2. The groups that were rated at Level 3 used technology to support students’ higher order thinking skills. For example, Group 5 in Section 1 (P5) taught the topic Five Senses, in which the students listened to different digital sounds and analyzed the sounds so as to determine what the sounds were.

The groups that were rated at Level 2 mainly used technology to support lower levels of content understanding. For example, six groups used concept maps to give students an overview of the upcoming learning content (P1, P2, P3, P4, Q2, and Q3), and three groups asked students to fill in a concept map with blank boxes after the teaching to assess their comprehension of the content (P5, Q3, and Q5). They also created videos to demonstrate an experiment regarding water evaporation and condensation (P2), explain concepts of the learning content (P2, P3, P4, P5, and Q1), or create a video letter for parents (Q5).

Compared to the lesson plans created in Step 2 (Engage in TPACK), student-centered strategies of technology integration decreased in Step 3 (Practice TPACK). For example, in Step 2, 6.2% of the lesson plans from the High and Middle Performance groups were rated at Level 4b that included student-centered activities (e.g., each student created a blog to post an image with description each day to document and reflect on daily life). However, in Step 3, all 10 groups’ technology applications were teacher centered. Their learning of TPACK in Step 2 was not used in Step 3 when they had to implement the lesson plan in class.

Summary of Findings

In response to Research Question 1, the effects of the IDDIRR2 model on improving preservice teachers’ TPACK are summarized in terms of the three steps of the model as follows:

Understand TPACK. Opportunities for the participants to discuss TPACK and view, assess, and compare TPACK integration teaching examples facilitated their basic understanding of TPACK. However, the participants were not able to evaluate and provide specific suggestions with regard to the non-TPACK example.

Engage in TPACK. LoUs were improved from Project 1 to Project 3; 33% improvement occurred in the High Performance group, 67% improvement occurred in the Middle Performance group, and 100% improvement occurred in the Low Performance group. The highest levels given to Project 1 among the High, Middle, and Low Performance groups were Level 4b, Level 2, and Level 1, respectively, while the highest levels given to Project 3 among the three performance groups were Level 4b, Level 4b, and Level 3. This improvement indicates that the participants’ TPACK was improved.

Practice TPACK. Two groups’ implementations of the final lesson plans were rated at Level 3, and the remaining eight groups were rated at Level 2, indicating 100% teacher-centered strategies of technology application. The participants’ understanding of student-centered strategies of technology application (Level 4 and above) in Step 2 (Project 3 had 7% of student-centered lesson plans) was not utilized in actual teaching in Step 3.

Discussion

The IDDIRR2 model is a revised TPACK-based ID model that involved much more active application of TPACK than did IDDIRR1. In a sense, this active application is consistent with Merrill’s First Principles of Instruction, which he has summarized on many occasions publicly with the statement that people learn what they do (Merrill, 2002, 2009). The findings indicated that the preservice teachers’ TPACK was enhanced through the active TPACK discussions (Step 1) and designing several technology-integrated lesson plans (Step 2). However, their TPACK was not fully utilized in teaching practice (Step 3).

In terms of the design guidelines, IDDIRR2 focused on providing pedagogy-enhanced activities to improve preservice teachers’ teaching-related knowledge so as to facilitate their learning of TPACK. The findings showed that Guideline 2 and Guideline 3 were effective. Still, several aspects of Guidelines 2, 3, and 4 are in need of revision to improve the model.

First, in Step 1 the preservice teachers could differentiate the quality of technology integration between a technology-integration teaching example and a non-example easily. However, when they were asked to give detailed explanations of a TPACK example and non-example based on TPACK concepts, their descriptions of the TPACK example were more detailed and accurate than those of the non-example. They may have lacked the ability to analyze critically and give constructive suggestions for the non-example.

This finding seems to be consistent with the principles of cognitive processing, because evaluation requires higher order skills than does comprehension (Bloom, 1956). For the preservice teachers, comprehending that the TPACK example shows a more effective technology integration practice than the non-example may have been easier than evaluating specific components of each example. Although this study showed that group discussions of teaching examples helped preservice teachers learn TPACK, more improvements may have been observed if IDDIRR2 had included an element guiding the instructor in supporting preservice teachers to evaluate technology integration practices effectively.

Second, in Step 2, LoUs were improved among all performance groups, comparing each individual’s initial lesson plan (Lesson Plan 1 in Project 1) to one of the four final lesson plans (in Project 3) that was rated at the highest LoU. In the High Performance group, 2 out of 6 preservice teachers’ lesson plans were rated at Level 4b in Project 1, while 4 preservice teachers’ lesson plans were rated at Level 4b in Project 3 (33% improvement).

In the Middle Performance group, all 6 preservice teachers’ lesson plans were rated at Level 2 in Project 1, while 4 preservice teachers’ lesson plans were rated at either Levels 3 or 4b in Project 3 (67% improvement).

In the Low Performance group, all 5 preservice teachers’ lesson plans were rated at Level 1 in Project 1, while all 5 preservice teachers’ lesson plans were rated at either Levels 3 or 2 in Project 3 (100% improvement). The percentages indicated that the preservice teachers’ TPACK was improved by designing lesson plans that better integrated technology than did their initial lesson plans.

In Step 2 and Step 3, however, the preservice teachers applied technologies mostly in teacher-centered ways. Among the 68 lesson plans developed in Project 3 of Step 2, only five lesson plans (7%) were rated at Level 4b, which means that about 93% of the lesson plans were teacher centered. In Step 3, no student-centered strategies (0%) were found in the 10 groups’ lesson implementation. Their lesson plans were rated either at Level 2 or Level 3. This result implies that the 5 individuals whose plans were rated at Level 4b in Step 2 did not demonstrate influence on the group work in Step 3.

It is also possible that the preservice teachers thought of lesson implementation as presentation, so their teaching resembled giving lectures rather than interaction with students. Although findings showed that preservice teachers demonstrated improvement in their lesson plans in Step 2, more improvements may have occurred in Step 2 and Step 3 if this model had included a component to teach preservice teachers student-centered strategies for technology integration. Future research should consider that preservice teachers’ understanding of the importance of active interaction between students and technology is critical to students’ learning results.

Redesign of the Model

In response to Research Question 2, several aspects of the model should be improved to advance preservice teachers’ TPACK. Step 1 should include a component having the instructor provide more effective support to help preservice teachers evaluate the given examples. For example, the instructor could provide an immediate explanation when learners encounter difficulties solving problems and give corrective feedback for them to revise their answers (Merrill, 2002). The support can facilitate preservice teachers’ more accurate evaluation of both highly effective and less-effective teaching examples.

Second, in Step 2 and Step 3 the technologies that the preservice teachers integrated into their lessons were applied in more teacher-centered ways than student-centered ways. The model should be revised to include components aiming to help preservice teachers understand the importance of student-centered strategies and develop the ability to practice the strategies. Preservice teachers should understand that student-centered technology application gives students opportunities “to seek rather than to comply, to experiment rather than to accept, to evaluate rather than to accumulate, and to interpret rather than to adopt” (Hannafin & Land, 1997, p. 175). If preservice teachers are grouped according to subject areas, preservice teachers may have more opportunities to discuss with peers about suitable methods of applying technology in consideration of the characteristics of content.

Limitations of the Study and Future Research Suggestions

There are several limitations in the following aspects of this study that should be addressed. First, quota sampling was used to segment and select samples based on the study needs instead of random selection (Castillo, 2009). However, this method may have resulted in sampling biases. Second, IDDIRR2 attempted to improve preservice teachers’ teaching-related knowledge by having them discuss TPACK teaching examples actively (Step 1). In this step, the instructor walked around the classroom and provided explanation to facilitate groups’ discussion. However, it was possible that the instructor’s engagement was a disruption rather than a help for groups. It was also likely that the instructor’s explanations could only partially answer questions that the groups had. Thus, more systematic support (e.g., guiding questions or corrective feedback) from the instructor should be designed.

IDDIRR2 also had preservice teachers design several technology-integrated lesson plans to improve their teaching-related knowledge (Step 2). However, this model did not emphasize student-centered strategies of technology application (LoU 4 and above), which could explain why the participants’ lesson plans were mostly rated at Level 2 or Level 3 in Step 2 and Step 3. The preservice teachers may have practiced teaching in a teacher-centered manner because their students were also classmates. Future research should enhance preservice teachers’ teaching-related knowledge specifically about student-centered strategies, so as to help them practice technology integration based on students’ needs.

Third, the validity of the model should be improved. The model was implemented only in a technology course in which one of the researchers was also the instructor. It was likely that the dual roles had the potential to influence the data collection and interpretation. As a result of this limitation, Design Guideline 1 (i.e., explicit and systematic procedures) could not be empirically examined because the researcher was the only practitioner. In addition, this study had only one colleague as the data reviewer. If more reviewers were involved, the objectivity and validity of finding interpretations might have been advanced. Future research should implement the model in diverse contexts, have different instructors carry it out, and include more validation strategies (e.g., multiple peers for peer review and member checking) to increase the validity of the model.

Implications for Research and Practice

This study findings highlight the importance of preservice teachers’ teaching-related knowledge in their TPACK acquisition. Although it has been thought that TPACK should be cohesively taught as integrated knowledge (Mishra & Koehler, 2006), this study’s findings suggest that the establishment of the isolated domains of TPACK (e.g., pedagogical knowledge) is essential in learning integrated knowledge. The model used to teach preservice teachers in a multidisciplinary technology integration course is applicable to many teacher education settings. In reality, technology integration courses are often offered to preservice teachers from diverse majors (National Center for Education Statistics, 2008).

Conclusion

The IDDIRR2 model is the second prototype (Prototype II) of the design-based research to develop a TPACK-based ID model for teacher training programs to improve preservice teachers’ TPACK in a multidisciplinary technology integration course. The design guidelines of this prototype focused on improving preservice teachers’ pedagogy-related knowledge to facilitate their TPACK acquisition. The results showed that the preservice teachers’ TPACK was improved in Steps 1 and 2. However, when they actually used their TPACK in teaching (Step 3), technologies were used more in teacher-centered ways than student-centered ways. Findings of this prototype will inform the development of the next prototype (Prototype III), in which the effective components of IDDIRR2 will be retained, while the limitations will serve as vital information for revision.

 

References

Bloom, B. S. (1956). Taxonomy of educational objectives, the classification of educational goal–Handbook I: Cognitive Domain. New York, NY: McKay.

Castillo, J. J. (2009). Quota sampling. Retrieved from the Explorable Psychology Experiments website: http://explorable.com/quota-sampling

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.

Ellsworth, J. B. (2000). Surviving change: A survey of educational change models. New York, NY: ERIC Clearing House on Information and Technology.

Gagné, R. M., Wager, W. W., Golas, K. C., & Keller, J. M. (2005). Principles of instructional design. Belmont, CA: Wadsworth/Thomson Learning.

Graham, R. C., Burgoyne, N., Cantrell, P., Smith, L., St. Clair, L., & Harris, R. (2009). Measuring the TPCK confidence of inservice science teachers. TechTrends, 53(5), 70-79.

Gustafson, K. L., & Branch, R. M. (2002). What is instructional design? In R. A. Reiser & J. A. Dempsey (Eds.), Trends and issues in instructional design and technology (pp. 16-25). Saddle River, NJ: Merrill/Prentice-Hall.

Hannafin, M. J., & Land, S. M. (1997). The foundations and assumptions of technology-enhanced student-centred learning environments. Instructional Science, 25, 167-202.

Hall, G. E., Dirksen, D. J., & George, A. A. (2006). Measuring implementation in schools: Levels of use. Austin, TX: Southwest Educational Development Laboratory.

Jang, S. J., Guan, S. Y., & Hsieh, H. F. (2009). Developing an instrument for assessing college students’ perceptions of teachers’ pedagogical content knowledge. Procedia Social and Behavioral Sciences, 1(1), 596–606.

Kalantzis, M., Cope, B., & The Learning by Design Group. (2005). Learning by design. Melbourne, AUS: Victorian Schools Innovation Commission in association with Common Ground Publishing Pty Ltd.

Kim, C., Kim, M., Lee, C., Spector, J. M., & DeMeester, K. (2013). Teacher beliefs and technology integration. Teaching and Teacher Education, 29, 76–85.

Koehler, M. J., & Mishra, P. (2005). Teachers learning technology by design. Journal of Computing in Teacher Education, 21(3), 94-102.

Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70. Retrieved from http://www.citejournal.org/vol9/iss1/general/article1.cfm

Koehler, M. J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy, and technology. Computers and Education, 49(3), 740-762.

Lee, C. J., & Kim, C. (2014). An implementation study of a TPACK-based instructional design model in a technology integration course. Educational Technology Research and Development. Educational Technology Research and Development, 62(4), 437-460.

Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research, 1(2). Retrieved from http://217.160.35.246/fqs-texte/2-00/2-00mayring-e.pdf

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59.

Merrill, M. D. (2009). First principles of instruction. In C. M. Reigeluth & A. Carr (Eds.), Instructional design theories and models: Building a common knowledge base (Vol. 3). New York, NY: Routledge Publishers.

Merriam, S. B. (1988). Case study research in education: A qualitative approach. San Francisco, CA: Jossey Bass.

Merriam, S. B. (1995). What can you tell from an N of 1?: Issues of validity and reliability in qualitative research. PAACE Journal of Lifelong Learning, 4, 51-60.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A new framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.

National Center for Education Statistics. (2008). Educational technology in teacher education programs for initial licensure (U.S. Department of Education Report No. NCES 2008-040). Retrieved from http://nces.ed.gov/pubs2008/2008040.pdf

Plomp, T. (2007). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 9-35). Enschede, The Netherlands: SLO Netherlands Institute for Curriculum Development.

Reeves, T. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 82-109). London, UK: Routledge.

Sahin, I. (2011). Development of survey of technological pedagogical and content knowledge (TPACK). Turkish Online Journal of Educational Technology, 10(1), 97-105.

Schmidt, D., Baran, E., Thompson, A., Koehler, M. J., Shin, T, & Mishra, P. (2009, April). Technological pedagogical content knowledge (TPACK): The development and validation of an assessment instrument for preservice teachers. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Simons, H. (2009). Case study research in practice. Los Angeles, CA: Sage.

Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-22.

Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Introducing education design research. In J. V. D. Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 3–7). New York, NY: Routledge.

Wolcott, H. (1994). Transforming qualitative data: Description, analysis and interpretation. Thousand Oaks, CA: Sage.

Yin, R. K. (2009). Case study research: Design and methods (4thed.). Thousand Oaks, CA: Sage.

Author Notes

Chia-Jung Lee
University of Georgia
Email: clily@uga.edu

ChanMin Kim
University of Georgia
Email: chanmin@uga.edu


Appendix A
An Example of the Combination of Content Knowledge, Pedagogical Knowledge, and Technological Knowledge and Research Explanation

The implementation results of the IDDIRR1 model indicated that the preservice teachers’ understanding of TPACK was the combination rather than the integration of content knowledge, pedagogical knowledge, and technological knowledge. For example, a group of preservice teachers demonstrated an online game involving living and non-living things, and they referred to that as TCK because they thought that an online game (which might be thought of as an instance of technological knowledge) could be used to present learning content (content knowledge). Their understanding of integrated knowledge (TPK, TCK, or TPACK) was limited simply to combining technology, pedagogy, and content rather than integrating them coherently and seamlessly in a unit of instruction.

This distinction is similar to a distinction between a lesson that involves multiple media (e.g., first a video clip, and then a PowerPoint presentation, and then a discussion forum) as opposed to a multimedia lesson that interleaves such things together in a mutually supportive manner. While it does take effort to select individual media items to include in a lesson, it takes much more design effort to interleave those things so that the learner is seamlessly engaged in an ongoing learning process. As emphasized by Mishra and Koehler (2006), a teacher’s TPACK should involve an understanding of the relationships among the three types of knowledge and how they will engage learners and mutually support learning.


Appendix B
Prototype II of the TPACK-Based ID Model

Appendix B


Appendix C
The Coding Scheme (Applying LoU[a] to This Research[b])

Levels of Use
Application of LoU to This Research
0: Nonuse 

State in which the user has little or no knowledge of the innovation, no involvement with the innovation, and is doing nothing toward becoming involved.

 

0: Nonuse 

State in which the preservice teacher has little or no knowledge of technology integration into teaching, no involvement with the innovation, and is doing nothing toward becoming involved. For example:
– A lesson is planned and/or implemented without the use of technology.
– Instructional resources are limited to paper-based materials (e.g., worksheets).

1: Orientation

State in which the user has recently acquired or is acquiring information about the innovation and/or has recently explored or is exploring its value orientation and its demands upon user and user system.

1: Orientation

State in which the preservice teacher has recently acquired or is acquiring information about technology integration and/or has recently explored or is exploring its value orientation and its demands upon the educational system. For example:
-The preservice teacher uses technology to prepare instructional materials (e.g., using a word processor to create worksheets), manage classroom tasks (e.g., sending emails, grading students’ work, counting attendance, etc.), or make the instruction convenient (e.g., using a projector).

2: Preparation

State in which the user is preparing for the first use of the innovation.

2: Preparation

State in which the preservice teacher starts to use technology in teaching. For example:
-The preservice teacher uses technology to support students’ understanding or comprehension of the learning content using lower-level cognitive skills (e.g., memorization, organization).
-Students are given opportunities to use technology to learn under preservice teachers’ direction (i.e., teacher-centered strategies for technology integration).

3: Mechanical use

State in which the user focuses most effort on the short-term, day-to-day use of the innovation with little time for reflection. Changes in use are made more to meet user needs than client needs.

3: Mechanical use

State in which the preservice teacher focuses most effort on the efficient use of technology integration with little time for reflection. Changes in use are made more to meet the preservice teacher’s needs than students’ needs. For example:
-The preservice teacher guides students in using technology to learn the content by means of constructing concepts, building in-depth understanding, doing scientific inquiry (e.g., exploring, analyzing, and synthesizing data), and thinking critically following the preservice teacher’s instruction and direction (supporting higher-level cognitive skills using teacher-centered strategies for technology integration).

4a: Routine use

Use of the innovation is stabilized. Few if any changes are being made in ongoing use. Little preparation or thought is being given to improving innovation use or its consequences.

4a: Routine use

Use of technology in teaching is stabilized. Few if any changes are being made in ongoing use. Little preparation or thought is being given to improving the use of technology or students’ learning results. For example:
-The preservice teacher consistently and regularly guides students in using technology to learn higher-level cognitive skills while starts to give students opportunities to select or explore technologies that are suitable for their learning (the beginning of student-centered strategies for technology integration).

4b: Refinement

State in which the user varies the use of the innovation to increase the impact on clients within immediate sphere of influence. Variations are based on knowledge of both short- and long-term consequences for clients.

4b: Refinement

State in which the preservice teacher varies the use of technology to improve students’ learning within immediate sphere of influence. Variations are based on knowledge of both short- and long-term learning results for students. For example:
-The preservice teacher is a facilitator of students’ learning and supports students in deciding what technology can best facilitate or present their learning (high level of student-centered strategies for technology integration).

5: Integration

State in which the user is combining own efforts to use the innovation with related activities of colleagues to achieve a collective impact on clients within their common sphere of influence.

5: Integration

State in which the preservice teacher use technology for teaching to make a collective impact of technology integration on student learning by allowing students to use technology collaboratively with others out of the classroom. For example:
-The preservice teacher provides opportunities for or encourages students to use technology collaboratively with partnerships beyond the classroom (e.g., parents, professors, scientists, etc.) that promote their higher-level learning skills.

6: Renewal

State in which the user reevaluates the quality of use of the innovation, seeks major modifications of or alternatives to present innovation to achieve increased impact on clients, examines new developments in the field, and explores new goals for self and the system.

6: Renewal

State in which the preservice teacher reevaluates the quality of technology integration, seeks major modifications of or alternatives to achieve increased impact on students, examines new developments in the field, and explores new goals for self and the educational system. For example:
-The preservice teacher makes efforts to have the learning settings seamlessly integrate with technology, in which students are engaged in student-centered, higher-order, and collaborative learning activities. Learning is impossible without the use of technology at this level.

[a] Levels of Use by G. E. Hall, D. J. Dirksen, and A. A. George, 2006, Austin: SEDL. Copyright © 2006, SEDL. Reprinted by the corresponding author with permission of SEDL.
[b] The identification of terms in LoU in corresponding to this research: User refers to preservice teacher; innovation refers to the use of technology in teaching or technology integration; clientrefers to student; increase the impact refers to student learning.

 


Appendix D
Example Responses of Groups’ Discussion on TPACK-Based Questions

TPACK-based Questions

Video 1 (The Example of Effective TPACK Integration)

Video 2 (Non-example of TPACK Integration)

TCK:
Q1. How did the tool(s) represent/transform the content into forms that are comprehensible or that made it easier for learners to realize the content (Video 1)?
Q2. Are the technological tool(s) unique, necessary, and helpful for that topic (Video 2)?
This tool [MapQuest and kid Pix] brings an abstract and spatial idea to life in video. This allows students to visualize the cardinal directions without a map. Once the students understand the directions in the real world, they will be able to apply them to a map more successfully. (L1)
The technological tools were able to take the students on virtual field trips, which allowed them to see the content and visualize them. (M1)
-Yes, it allowed the class to interact and discuss characteristics of living and nonliving things in order to come to find the correct answer. (L5)
– They [online games] are not necessary because there are many other mediums to explain this concept better. (M3)
-They are unique and more engaging for the class as a whole but it is not necessary. I think it was [still] helpful for students. (M4)
TPK:
What activities were the students engaged in when using technological tool(s)? (TPK)
Because fourth-grade minds are thinking in such concrete ways, they cannot grasp such an abstract concept without visual cues like video…students are creating their own construct of the cardinal directions when they are using the interactive map for themselves. Instead of seeing the directions on a 2D map in a book, the students are engaged in an activity where there is a goal or destination. The students have to apply their knowledge of directions in order to maneuver their way through the map. (L1)
Mapquest provides the students a map to read and [they] understand how to directionally get to a place. Students understand the concepts of NSEW [cardinal points] better when having to actually do hands-on activities. Because the students are using Mapquest to get to their destination, it makes them construct and organize the knowledge and concepts to achieve their goal of directing themselves using a map. (M2)
-The teacher could use more meaningful tools to teach the students that help the students grasp the concepts being taught. The tools that could be used to teach the subject should offer a better demonstration of living and non-living things. (L2)
-It was meaningful because she [the teacher] taught it then immediately implemented it by showing them how to apply it. (L3)
-The use of the computer between the student and their own desktop enhanced the lesson by being entertaining. (M4)
TPACK:
Comparing the two teaching videos, in which one do you think that the technology better represented/transformed the content into forms that are comprehensible and that made it easier for learners to realize the content? Why?
The first video used technology when it was more necessary for her students. Cardinal directions are an abstract concept that would be difficult for the students to learn by traditional methods. Maps are better represented on a computer with technology because they allow students to visualize a large idea in a single space. The video on living/non-living things can be easily represented in real life because the students have dealt with these objects in their everyday lives. The students could better identify the differences between living and nonliving things by dealing with objects in the real world. (L1)
We think the first video was [better] enhanced with technology. The second lesson would [have] been beneficial if the teacher incorporated hands-on activities or other engaging tasks. The first lesson was unique and enhanced because the teacher projected maps and pictures, and students could visualize material being presented. (M4)