Schrum, L., Thompson, A., Maddux, C., Sprague, D., Bull, G., & Bell, L. (2007). Editorial: Research on the effectiveness of technology in schools: The roles of pedagogy and content Contemporary Issues in Technology and Teacher Education [Online serial], 7(1). https://citejournal.org/volume-7/issue-1-07/editorial/editorial-research-on-the-effectiveness-of-technology-in-schools-the-roles-of-pedagogy-and-content

Editorial: Research on the Effectiveness of Technology in Schools: The Roles of Pedagogy and Content

by Lynne Schrum, Editor, Journal of Research on Technology in Education; Ann Thompson, Editor, Journal of Computing in Teacher Education; Cleb Maddux, Research Editor, Computers in the Schools; Debra Sprague, Editor, Journal of Technology and Teacher Education; Glen Bull, Editor, Contemporary Issues in Technology and Teacher Education; & Lynn Bell, Editor, Contemporary Issues in Technology and Teacher Education

The Challenge

The field of educational technology is under external pressure to provide evidence of identifiable learning outcomes that can be attributed to technology. Leaders within the educational technology research community agree about the importance of such evidence. Each year the Society for Information Technology and Teacher Education (SITE) jointly sponsors a National Technology Leadership Summit (NTLS) to consider such issues in concert with partner associations. Ten national education associations are currently partners in the National Technology Leadership Coalition (www.NTLCoalition.org).

One goal of the coalition is to proactively facilitate needed research that will advance the profession. We would like to share the collective perspective of the editors of five educational technology journals who are members of the National Technology Leadership Coalition. In order to reach as diverse an audience as possible, an abbreviated version of this report is also being published in Learning and Leading with Technology.

Framing the Issues

Evidence that technology supports improved student learning can be gained only through credible research, but the way in which research issues are framed plays an important role in the results obtained. In the past media comparison was a common design used by researchers interested in educational technology. These kinds of studies compared the effectiveness of one medium with another on a variety of dependent variables. Such studies were popular for many reasons; they are conceptually simple and they are appealing on an intuitive level. The search for relative advantage is at the heart of many scientific studies, and, “Which is better?” is a natural question for policy makers (and beginning researchers) to pose.

At the beginning of the 1980s, Richard Clarke conducted a well-known meta-analysis of this type of educational technology research and concluded that media are “mere vehicles that deliver instruction but do not influence achievement any more than the truck that delivers our groceries causes changes in our nutrition” (1983, p. 445). This perspective might be termed the transmission model of educational technology – the view that technology is a delivery mechanism with no unique capacity or capabilities that might intrinsically affect learning.

This view stands in stark contrast to the view guiding much of the research to that point, which might be thought of as the “exposure model” of educational technology. After all, what Clarke (rightly) concluded was that mere exposure to technology confers no particular educational benefits.

Of course, the same is true of pencils, paper, books, teachers, classrooms and all other educational authorities and artifacts. We would not expect that placing a child and a book in the same room would necessarily result in educational benefits. Exposure to books is a necessary but not sufficient condition for books to be educationally effective, but the critical variables will almost certainly prove to include both the content of the book and the way the child interacts with that content. The same is true of technology. To use Clarke’s rather prosaic analogy, in order for the grocery truck to be effective in improving a person’s nutrition, the person has to be on the truck’s delivery route and the truck also has to be delivering something besides doughnuts and French fries.

Clark’s observation implies a powerful conclusion: There is probably no generic technology effect on teaching and learning. However, the transmission model of instruction is itself flawed, because it treats all instruction as generic and fails to differentiate by content being taught or by teaching strategies employed.

During the same era as Clark’s meta-analysis, Lee Shulman suggested that teacher education research of that era was overlooking the central role of content and subject matter, a phenomenon he called the “missing paradigm”:

The missing paradigm refers to a blind spot with respect to content that now characterizes most research on teaching and, as a consequence, most of our state-level programs of teacher evaluation and teacher certification….What we miss are questions about the content of the lessons taught, the questions asked, and the explanations offered. (Shulman, 1986, p. 7-8)

Shulman believed that crucial aspects of pedagogical practice are uniquely connected to specific content areas. He went on to coin the phrase pedagogical content knowledge to describe this relationship.

Shulman’s and Clark’s observations cast new light on educational technology research. Research questions and designs that fail to differentiate by the content being studied, the pedagogical strategies employed, and the way that technology interoperates with these variables will probably continue to find that merely using a technology medium is not educationally beneficial. But research that explores how technology interacts with pedagogy and content may disprove Clark’s claim that “media do not influence learning under any conditions” (1983, p. 445).

Technology Pedagogical Content Knowledge

The field has recently begun to move toward consensus that different technologies do have unique pedagogical affordances and that the effects of these affordances can only be understood in the context of a specific content area (and related learning outcomes) and a specific pedagogy. Schulman’s concept has been extended to encompass “technological pedagogical content knowledge” (Mishra & Koehler, 2006). The implication is that properly prepared teachers can take advantage of the unique features of technology to teach content in ways they otherwise could not (Garofalo, Drier, Harper, Timmerman, & Shockey, 2000).

For example, science teachers can use planetarium software such as Starry Night to teach astronomy concepts in a variety of ways. Some teachers may take students to the computer lab to use the software, but they assign worksheets guiding students to merely confirm concepts stated in the textbook – still a somewhat traditional pedagogy. Other teachers may employ the same software to facilitate inquiry, engaging students in making and testing predictions and discovering astronomical patterns. Students’ resulting comprehension of the content may differ based on the teachers’ pedagogy, even though both groups used the same technology.

Definition of Learning Outcomes

For the present, our premise remains a theoretical possibility rather than a demonstrated outcome. Only a minority of studies involving educational technology address learning (cognitive) outcomes. Few specify all three dimensions of the context surrounding the technology use – pedagogy, content, and technological affordances.

For example, a recent study of classes using streaming video reported higher student test scores in certain content areas over classes not using streaming video (see Boster, Meyer, et al., 2006). However, neither the curricular content nor the pedagogical use of the technology was described in a way that would permit replication of results. All that is reported is that digital movies were shown in the classroom. It is likely, though, that different teachers employed different approaches and pedagogical strategies. When specific instructional methods are not specified, it is difficult to understand the implications, or to know how such outcomes might be reliably replicated.

The specific curricular objectives being addressed must be understood in such cases. Table 1 (below) outlines some of the categories of learning outcomes found in each content area (Bell, Schrum, Thompson, & Bull, 2007).

 

Table 1
Some Categories of Student Learning Outcomes by Subject Area

 

Subject Area
Categories of Learning Outcomes
English
  • Language acquisition (e.g., vocabularly)
  • Reading comprehension
  • Writing ability
  • Interpretation and analysis
Mathematics
  • Skills: Computation, geometry, graphing
  • Mathematical reasoning, multiple representations
  • Understanding, interpreting, transferring to other contexts, problem solving
Science
  • Subject knowledge and understanding, including a disciplinary way of knowing
  • Skills – predict-observe-explain, data analysis, model construction, application of knowledge, problem solving, inquiry, replication
  • Representations, geospatial reasoning
Social Studies
  • Historical thinking
  • Historical inquiry
  • Subject knowledge
  • Citizenship

The specific category of learning outcome within a content area is significant. For example, legislators and policy makers have focused primarily on subject knowledge as an important objective in social studies. In contrast, social studies education researchers have tended to address learning objectives such as historical inquiry and thinking skills in their studies. As a result, few studies to date have examined the effect of technology on student learning outcomes found on high stakes tests in social studies.

Next Steps

An ongoing goal of NTLS is continuation of dialog about needed research in the field of educational technology. An editorial entitled, “A Proactive Approach to a Research Agenda for Educational Technology” was published in the Journal of Research on Technology in Education to begin the dialog. A May 2006 article in Learning and Leading with Technology specifically described key research issues identified by teacher educator associations in the content areas of mathematics, science, English, and social studies.

As a result of this year’s NTLS, teacher educators in mathematics, science, social studies, English, reading, early childhood education, and special education have agreed to take the next step by summarizing the state of the research in their fields relating to specific technologies and student learning. These conclusions will be summarized in a research monograph. The intent is to facilitate research on the relationship between specific technologies and student learning of school curriculum by searching out existing models of research and advancing the discussion about the characteristics of exemplary research.

Until the pedagogical methods that uniquely take advantage of a technology’s pedagogical affordances to achieve content-specific learning objectives are identified, it will not be possible to prepare teachers to make effective use of current and emerging technologies. Future research reports must include these variables when learning outcomes are described.

M.D. Roblyer (2005) noted that the field of educational technology currently lacks a clear theoretical foundation as a framework for research. Dialog on this topic may move the field closer to a common framework for productive research in the future. This process will also allow us to reflect on considerations that should be incorporated into the review process for the educational technology journals that collectively serve as NTLS sponsors.

The ultimate goal is to ensure that research on technology and innovation is useful to both educators in schools and those who prepare them for these roles. By presenting and analyzing instances in which effective application of TPCK has resulted in differences in learning outcomes in each of the core content areas, we hope to provide models that will stimulate more research in this vein. We invite input and recommendations regarding noteworthy research related to the effect of technology on student learning in specific content areas and will report conclusions and outcomes as they emerge.

References

Bell, L., Schrum, L., Thompson, A., & Bull, G. (2007). Introduction. In L. Bell & A. Thompson (Eds.), Framing research on technology and student learning in the content areas: Implications for teacher educators. Manuscript in preparation.

Boster, F. J., Meyer, G. S., Roberto, A. J., Inge, C., & Strom, R. (2006). Some effects of video streaming on educational achievement. Communication Education, 55, 46-62.

Clark, R. C. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445-59.

Garofalo, J., Drier, H., Harper, S., Timmerman, M.A., & Shockey, T. (2000). Promoting appropriate uses of technology in mathematics teacher preparation. Contemporary Issues in Technology and Teacher Education, 1(1), 66-88.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A new framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.

Roblyer, M. D. (2005). Educational technology research that makes a difference: Series introduction. Contemporary Issues in Technology and Teacher Education, 5(2). Retrieved March 12, 2007, from https://citejournal.org/vol5/iss2/seminal/article1.cfm

Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14.

 

 

Loading