Schrum, L., Thompson, A., Maddux, C., Sprague, D., Bull, G., & Bell, L. (2007). Editorial: Research on the effectiveness of technology in schools: The roles of pedagogy and content Contemporary Issues in Technology and Teacher Education [Online serial], 7(1). Available: http://www.citejournal.org/vol7/iss1/editorial/article1.cfm
Editorial: Research on the Effectiveness of Technology in Schools:
The Roles of Pedagogy and Content
Journal of Research on Technology in Education
Journal of Computing in Teacher Education
Cleb Maddux, Research Editor
Computers in the Schools
Debra Sprague, Editor
Journal of Technology and Teacher Education
Glen Bull and Lynn
Contemporary Issues in Technology and Teacher Education
The field of educational technology is under external pressure to provide evidence
of identifiable learning outcomes that can be attributed to technology. Leaders
within the educational technology research community agree about the importance
of such evidence. Each year the Society for Information Technology and Teacher
Education (SITE) jointly sponsors a National Technology Leadership Summit (NTLS)
to consider such issues in concert with partner associations. Ten national education
associations are currently partners in the National Technology Leadership Coalition
One goal of the coalition is to proactively facilitate needed research that
will advance the profession. We would like to share the collective perspective
of the editors of five educational technology journals who are members of the
National Technology Leadership Coalition. In order to reach as diverse an audience
as possible, an abbreviated version of this report is also being published in
Learning and Leading with Technology.
Framing the Issues
Evidence that technology supports improved student learning can be gained only
through credible research, but the way in which research issues are framed plays
an important role in the results obtained. In the past media comparison was
a common design used by researchers interested in educational technology. These
kinds of studies compared the effectiveness of one medium with another on a
variety of dependent variables. Such studies were popular for many reasons;
they are conceptually simple and they are appealing on an intuitive level. The
search for relative advantage is at the heart of many scientific studies, and,
“Which is better?” is a natural question for policy makers (and
beginning researchers) to pose.
At the beginning of the 1980s, Richard Clarke conducted a well-known meta-analysis
of this type of educational technology research and concluded that media are
“mere vehicles that deliver instruction but do not influence achievement
any more than the truck that delivers our groceries causes changes in our nutrition”
(1983, p. 445). This perspective might be termed the transmission model of educational
technology – the view that technology is a delivery mechanism with no
unique capacity or capabilities that might intrinsically affect learning.
This view stands in stark contrast to the view guiding much of the research
to that point, which might be thought of as the “exposure model”
of educational technology. After all, what Clarke (rightly) concluded was that
mere exposure to technology confers no particular educational benefits.
Of course, the same is true of pencils, paper, books, teachers, classrooms
and all other educational authorities and artifacts. We would not expect that
placing a child and a book in the same room would necessarily result in educational
benefits. Exposure to books is a necessary but not sufficient condition for
books to be educationally effective, but the critical variables will almost
certainly prove to include both the content of the book and the way the child
interacts with that content. The same is true of technology. To use Clarke's
rather prosaic analogy, in order for the grocery truck to be effective in improving
a person's nutrition, the person has to be on the truck's delivery route and
the truck also has to be delivering something besides doughnuts and French fries.
Clark’s observation implies a powerful conclusion: There is probably
no generic technology effect on teaching and learning. However, the transmission
model of instruction is itself flawed, because it treats all instruction as
generic and fails to differentiate by content being taught or by teaching strategies
During the same era as Clark’s meta-analysis, Lee Shulman suggested that
teacher education research of that era was overlooking the central role of content
and subject matter, a phenomenon he called the “missing paradigm”:
The missing paradigm refers to a blind spot with respect to content that
now characterizes most research on teaching and, as a consequence, most of
our state-level programs of teacher evaluation and teacher certification….What
we miss are questions about the content of the lessons taught, the questions
asked, and the explanations offered. (Shulman, 1986, p. 7-8)
Shulman believed that crucial aspects of pedagogical practice are uniquely
connected to specific content areas. He went on to coin the phrase pedagogical
content knowledge to describe this relationship.
Shulman’s and Clark’s observations cast new light on educational
technology research. Research questions and designs that fail to differentiate
by the content being studied, the pedagogical strategies employed, and the way
that technology interoperates with these variables will probably continue to
find that merely using a technology medium is not educationally beneficial.
But research that explores how technology interacts with pedagogy and content
may disprove Clark’s claim that “media do not influence learning
under any conditions” (1983, p. 445).
Technology Pedagogical Content Knowledge
The field has recently begun to move toward consensus that different technologies
do have unique pedagogical affordances and that the effects of these affordances
can only be understood in the context of a specific content area (and related
learning outcomes) and a specific pedagogy. Schulman’s concept has been
extended to encompass “technological pedagogical content knowledge”
(Mishra & Koehler, 2006). The implication is that properly prepared teachers
can take advantage of the unique features of technology to teach content in
ways they otherwise could not (Garofalo, Drier, Harper, Timmerman, & Shockey,
For example, science teachers can use planetarium software such as Starry Night
to teach astronomy concepts in a variety of ways. Some teachers may take students
to the computer lab to use the software, but they assign worksheets guiding
students to merely confirm concepts stated in the textbook – still a somewhat
traditional pedagogy. Other teachers may employ the same software to facilitate
inquiry, engaging students in making and testing predictions and discovering
astronomical patterns. Students’ resulting comprehension of the content
may differ based on the teachers’ pedagogy, even though both groups used
the same technology.
Definition of Learning Outcomes
For the present, our premise remains a theoretical possibility rather than
a demonstrated outcome. Only a minority of studies involving educational technology
address learning (cognitive) outcomes. Few specify all three dimensions of the
context surrounding the technology use – pedagogy, content, and technological
For example, a recent study of classes using streaming video reported higher
student test scores in certain content areas over classes not using streaming
video (see Boster, Meyer, et al., 2006). However, neither the curricular content
nor the pedagogical use of the technology was described in a way that would
permit replication of results. All that is reported is that digital movies were
shown in the classroom. It is likely, though, that different teachers employed
different approaches and pedagogical strategies. When specific instructional
methods are not specified, it is difficult to understand the implications, or
to know how such outcomes might be reliably replicated.
The specific curricular objectives being addressed must be understood in such
cases. Table 1 (below) outlines some of the categories of learning outcomes
found in each content area (Bell, Schrum, Thompson, & Bull, 2007).
Some Categories of Student Learning Outcomes by Subject Area
Categories of Learning Outcomes
- Language acquisition (e.g., vocabularly)
- Reading comprehension
- Writing ability
- Interpretation and analysis
- Skills: Computation, geometry, graphing
- Mathematical reasoning, multiple representations
- Understanding, interpreting, transferring to other contexts, problem
- Subject knowledge and understanding, including a disciplinary way
- Skills – predict-observe-explain, data analysis, model construction,
application of knowledge, problem solving, inquiry, replication
- Representations, geospatial reasoning
- Historical thinking
- Historical inquiry
- Subject knowledge
The specific category of learning outcome within a content area is significant.
For example, legislators and policy makers have focused primarily on subject
knowledge as an important objective in social studies. In contrast, social studies
education researchers have tended to address learning objectives such as historical
inquiry and thinking skills in their studies. As a result, few studies to date
have examined the effect of technology on student learning outcomes found on
high stakes tests in social studies.
An ongoing goal of NTLS is continuation of dialog about needed research in
the field of educational technology. An editorial entitled, “A Proactive
Approach to a Research Agenda for Educational Technology” was published
in the Journal of Research on Technology in Education to begin the
dialog. A May 2006 article in Learning and Leading with Technology
specifically described key research issues identified by teacher educator associations
in the content areas of mathematics, science, English, and social studies.
As a result of this year’s NTLS, teacher educators in mathematics, science,
social studies, English, reading, early childhood education, and special education
have agreed to take the next step by summarizing the state of the research in
their fields relating to specific technologies and student learning. These conclusions
will be summarized in a research monograph. The intent is to facilitate research
on the relationship between specific technologies and student learning of school
curriculum by searching out existing models of research and advancing the discussion
about the characteristics of exemplary research.
Until the pedagogical methods that uniquely take advantage of a technology’s
pedagogical affordances to achieve content-specific learning objectives are
identified, it will not be possible to prepare teachers to make effective use
of current and emerging technologies. Future research reports must include these
variables when learning outcomes are described.
M.D. Roblyer (2005) noted that the field of educational technology currently
lacks a clear theoretical foundation as a framework for research. Dialog on
this topic may move the field closer to a common framework for productive research
in the future. This process will also allow us to reflect on considerations
that should be incorporated into the review process for the educational technology
journals that collectively serve as NTLS sponsors.
The ultimate goal is to ensure that research on technology and innovation is
useful to both educators in schools and those who prepare them for these roles.
By presenting and analyzing instances in which effective application of TPCK
has resulted in differences in learning outcomes in each of the core content
areas, we hope to provide models that will stimulate more research in this vein.
We invite input and recommendations regarding noteworthy research related to
the effect of technology on student learning in specific content areas and will
report conclusions and outcomes as they emerge.
Bell, L., Schrum, L., Thompson, A., & Bull, G. (2007). Introduction. In
L. Bell & A. Thompson (Eds.), Framing research on technology and student
learning in the content areas: Implications for teacher educators. Manuscript
Boster, F. J., Meyer, G. S., Roberto, A. J., Inge, C., & Strom, R. (2006).
Some effects of video streaming on educational achievement. Communication
Education, 55, 46-62.
Clark, R. C. (1983). Reconsidering research on learning from media. Review
of Educational Research, 53, 445-59.
Garofalo, J., Drier, H., Harper, S., Timmerman, M.A., & Shockey, T. (2000).
Promoting appropriate uses of technology in mathematics teacher preparation.
Contemporary Issues in Technology and Teacher Education, 1(1), 66-88.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content
knowledge: A new framework for teacher knowledge. Teachers College Record,
Roblyer, M. D. (2005). Educational technology research that makes a difference:
Series introduction. Contemporary Issues in Technology and Teacher Education,
5(2). Retrieved March 12, 2007, from http://www.citejournal.org/vol5/iss2/seminal/article1.cfm
Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational
Researcher, 15(2), 4-14.