A national symposium on the role of efficacy research in the development, adoption, and implementation of educational technology will take place in 2017. The Educational Technology Efficacy Research symposium represents the culmination of a year-long collaboration among stakeholders: academic researchers, entrepreneurs, school district and university leaders, investors, philanthropists, K-12 teachers, and college professors. This work is motivated by a belief that technology has unmet potential to improve student outcomes. The goal is to move efficacy to the center of the discourse concerning technology in education and, in doing so, create pathways for innovations that work to scale and make an impact. The effort proceeds on the belief that with regard to learning technologies, it is learning that is the primary consideration and most contemporary educational activities involve some kind of technology. The challenge is to determine which uses of various technologies promote learning and improve instruction in a wide variety of circumstances. While learning is always the primary consideration, the secondary consideration is the use of a technology to support learning.
Ten working groups are investigating the role of efficacy research as it relates to (a) K-12 district and school decision making, (b) higher education decision making, (c) research spending, (d) evidence and quality of efficacy in research approaches, (e) institutional competence, (f) investors and entrepreneurs, (g) the role of federal goals and funding, (h) educational philanthropies, (i) end users, and (j) crowdsourcing. The members of Working Group E are investigating the knowledge and competence that faculty members and leaders in educational institutions need to possess with respect to learning technologies in order to prepare competent teachers and school leaders, given the expectation that technologies and societal priorities are likely to change.
The first phase of this effort for Working Group E included interviews with senior faculty members at schools of education that are providing leadership in this area. In addition, parallel interviews were conducted with the director of the U.S. Office of Educational Technology and representatives of several teacher educator associations. Several of these interviews are published in a parallel document as context for the preliminary recommendations outlined below. This work was conducted within the framework of the National Educational Technology Plan (2016a) and the related Educational Technology in Teacher Preparation Policy Brief (2016b).
This work is grounded in the recognition that the context for adoption and use of technology in schools has changed in recent years. The traditional academic model is based on the concept that research on emerging educational innovations will be conducted and subsequently published in peer-reviewed journals. This peer-reviewed research is intended to guide decisions regarding adoption and use of technologies. All too often researchers end up only communicating with other researchers and occasionally with policy makers. Connecting theory, research, practice, and policy making is an important endeavor that is often overlooked.
Disruptive changes are occurring in this model. The decreasing cost of developing some software applications has meant that they are proliferating at a rate faster than academic researchers can review them. Further, the research process and subsequent peer-review typically takes several years. Consequently, research results may no longer be applicable by the time that they are published. In recent years, some journals have accelerated the publication process, and the now-standard rapid development cycle has begun to influence hardware and software development. For instance, rapid prototyping tools such as 3D printers make it easier to design and prototype new hardware and bring it to market. With regard to educational technologies, the emphasis on rapid developments sometimes brings an educational product to market before it is ready or without adequate support for its effective use by teachers. Research is increasingly focusing on studies conducted in actual school settings, but doing so places an increasing burden on schools to try out new technologies and their use in somewhat fixed curricula and often, without attention to specific learner needs.
Limited Information About the Efficacy of Technology
As a result, limited objective information is available to guide educators in the selection of available technologies and how they can be effectively used and supported. Awareness is dawning in teacher preparation programs that the paradigm has changed. One faculty member in our interviews commented,
Peer reviewed research will continue to play a very important part in influencing what we know and understand about technology. But the time lag between a research study being conducted, reviewed, published, and being disseminated to the field is far greater than the pace at which new technologies are emerging.
Flipped learning is a good example of this. Teachers were implementing flipped learning paradigms in their classrooms and learning from this practice for several years before the first book on this topic was published. It was even longer before the first peer-reviewed research on this topic was available. We should not ask teachers to wait until the peer-reviewed research becomes available. Once the peer-reviewed research is available, we want teachers to pay attention to it. However, if they wait until definitive research results are available, it will significantly retard advances in the field. In any event, there is no practical way to suppress use of emergent technologies until peer-reviewed research is available.
Although many studies now investigate the uses of a technology and the teacher training, professional development, and ongoing support needed to make effective use of a technology to promote learning, the impact of these research results is problematic. In particular, the dawning recognition of this paradigm shift has not yet been translated into widespread changes in teacher preparation practice. This document is intended as the beginning of a dialog regarding how we might best collectively proceed. We found general agreement among those interviewed that some type of action is required. One faculty member commented,
This is an issue that’s really important to me. I remember spending an entire semester in my teacher preparation program learning a technology that was not available in my classroom. I spent an entire semester of my teacher education program learning a technology that I have never used, never will be able to use with my students, and that was out of date within two to three years. As I consider the different institutions in which I have worked, many of them appear to be doing similar things.
On the other hand, many schools are using technologies not available in teacher preparation programs. This comment speaks to the U.S. Office of Educational Technology recommendation that integration of technology in teacher preparation programs should be “program wide and program deep.” In other words, the uses of technology should be incorporated into courses throughout the curriculum in a meaningful way rather than being restricted to a single course.
This goal often gives way to the reality of institutional pressures, however. The director of an elementary education program commented that she agreed that this goal is a good one, but thought that it was impractical in her setting. Her program was attempting to increase enrollment levels by reducing the number of credit hours required. This meant that it was not feasible to incorporate technology in the formal course offerings. In her context, preservice teachers received exposure to technology in their field placements, if at all. The elementary education director noted that while one of the collaborating school divisions made extensive use of technology, the other school divisions in which preservice teachers completed teaching internships did not. Consequently, exposure to technology in this teacher education program is not systemic or reliable. A faculty member in another teacher preparation program reported,
I have talked to our college leadership. They have concerns about the number of topics that must already be covered in the teacher education curriculum. A typical comment is that the curriculum is already packed. There is nowhere to add new content. I can understand their perspective but I also see this as a serious deficit.
Any meaningful plan to prepare teachers to use technology on a wide-scale basis will need to realistically address institutional constraints and pressures. The same is true when it comes to integrating technology effectively within school curricula. Telling preservice and in-service teachers what to do with technology is simply inadequate. Both preservice and in-service teachers need to be shown how to make effective use of a technology and how to determine that their use is, in fact, effective. They need opportunities to practice and evaluate the results of their technology use.
A related issue is the mindset with which technology use is approached. A faculty member commented,
We have traditionally focused primarily upon technical skills: how to create a website, how to create a blog, how to create a digital portfolio. Some of that is necessary. It is certainly not sufficient. If we are just teaching them skills then we are not teaching them what they are going to need in the classroom. We are also creating an expectation that in order to learn a new technology, they need to take a technical skills course. That is not how it works in the classroom. In the classroom, teachers need to be able to learn things on the fly, they need to teach themselves new technologies as they come out and they need to have a very courageous and adventurous attitude about technology in classrooms.
Again, the emphasis should be on learning and not on a particular technology. Technologies change, so the priority should be on how, why, when, and with whom a particular use of a technology supports learning.
A Pedagogical Framework
The faculty member cited in the previous comment believes that it is more important to teach students how they can approach a new technology than to teach any specific skill. A broader issue is the pedagogical framework in which use of technology is embedded. Some teacher preparation programs are moving toward introducing the use of technology in the context of problem-based or project-based learning. Moreover, Shulman (1986) observed that pedagogical understanding as well as content knowledge (PCK) are required for effective instruction. Twenty years later, Mishra and Koehler (2006) argued that this concept is applicable to instructional uses of technology. In other words, technology, pedagogy, and content knowledge (TPACK) are required for effective use of technology in schools. The point is that the components of PCK and TPACK are interrelated and not best treated separately. Separately, these instructional elements are necessary but insufficient to result in the effective use of technology.
Since pedagogical practice varies across disciplines, it follows that best use of technology may also vary across disciplines. The American Association of Colleges of Teacher Education published a landmark work, the AACTE Handbook of Technological Pedagogical Content Knowledge (2008), with chapters devoted to pedagogical approaches to the use of technology in different content areas.
One implication of this concept is that technology should be incorporated into pedagogy and content courses rather than covering it in an isolated technology course. Spreadsheets and graphing calculators are an integral part of the math education curriculum, but are less relevant to the social studies curriculum, which may focus on technologies such as use of online primary source documents to facilitate inquiry (for example). It is impractical, therefore, to cover this range of technologies within a single technology course. We also suggest that the emphasis on digital literacy within teacher education programs in some universities needs to be expanded to include the notion of critical literacy that includes critical reasoning skills and not just a focus on technology.
The concept of TPACK has been cited thousands of times, and has been influential within a narrowly circumscribed circle of faculty members whose professional careers focus on some aspect of emergent technologies. However, the concept is less well known among decision-makers who are responsible for designing the teacher education curriculum. This fact was graphically illustrated by the comments of two deans of education who were invited to participate in a 2016 White House Innovation Summit. These deans were selected to participate because of the reputations that their programs enjoy for technological leadership.
One dean explained that she had integrated Apple technologies throughout the teacher education program. Another dean described her plan for requiring all faculty members to use Google Chromebooks. She reported that one of her faculty members was resistant to this use, but she was confident that she could devise incentives that would lead to 100% participation. When asked if they believed that technology use varies across disciplines, one dean said in surprise, “No, of course not.” The second dean replied, “Why should it?” The assumption that a “one size fits all” preparation to use technology in teacher education is more common than not. Similarly, the director of another teacher education program known for technological leadership was also unfamiliar with the concept of TPACK. The framework within which these institutional leaders view technology has a significant effect on the way in which future teachers are prepared at their institutions.
Preparing Future School Leaders
This lack of understanding regarding the links between technology, pedagogy, and content knowledge also extends to K-12 educational leaders such as superintendents and technology coordinators. One faculty member commented,
The state and national standards for preparation of principals and superintendents have very little to do with technology at the district level and certainly down to the classroom level. There is a serious gap in what principals and superintendents know about what technology is supposed to be able to do in a school. They are not familiar with misconceptions about what technology can and cannot do in schools. Their curriculum focuses on the nature of the job as personnel managers and evaluators and does not focus on specific things like technology. Now, given the amount of money that schools and districts spend on these things I find that to be quite remarkable.
Similarly, central office technology coordinators who make purchasing decisions more often than not are information technology professionals rather than educational technology specialists, although there are some exceptions. In those roles, they are concerned with issues such as software licensing, hardware contracts, and government E-Rate funding initiatives. They are frequently tasked with some type of training or professional development for teachers to use technology. But, they often come from backgrounds where they probably are not aware of evaluations of technology other than perhaps some type of cost comparison or perhaps usability studies. So, they sometimes miss the instructional learning theory component of why something should or should not work well in a school.
These comments were echoed by the chair of a leading educational leadership program. This individual had served in the role of principal and superintendent prior to accepting a university position. She commented,
The technology choices made at the district level do not always meet the needs of teachers. To the extent that this is the case, it may be in part because superintendents and technology coordinators do not always have a good understanding of the way that technologies differ across content areas. Principals are generally not familiar with the way in which use of technologies differ conceptually across content areas. Administrators generally do not have that information. I do not know how you would have the time in the educational leadership curriculum to address each content area individually.
Advances in technology present teacher education programs with challenging issues to consider. There is a foundation of best practice that suggests possible directions for the future. A recurrent theme that emerged is the possibility of preparing teachers to assess learning outcomes in their classrooms. Joseph South, an educational researcher, technology consultant, and former Director of the Office of Educational Technology, suggested,
We need to teach teachers to conduct action research in their own classrooms. They should approach the use of any new tool in this way. This recognizes the reality that …we do not know what technologies are going to be invented in the future. The chances of teachers having consistently reliable information upon which to make a decision is low. Therefore, we must prepare teachers to make reasoned responses so that they base their choices on evidence.
Another faculty member concurred,
Assessment literacy can help a teacher thrive from day one. Assessment literacy is hard for preservice teachers to develop. Many practicing teachers lack really strong assessment literacy skills. However, this is crucial for areas such as technology. Often peer reviewed research will not appear for several years after a technology becomes available, if at all. What the teachers can do, though, is assess their own students’ experiences or uses of the technology.
A department chair concurred with this perspective,
Teachers’ perceptions of technology can be useful: “How easy was it to use?” “Did it resolve the issues they thought it would?” If the teacher doesn’t find it easy to use or solving the problems that they think it should solve, then it is not going to be used in the classroom. We should also consider student use: “What did the students think about it?” “How easy is it for the students to use?” “Does it really make a difference in their learning?”
As a former teacher, I think that teachers’ perceptions of their students is valid evidence that should also be considered. If the students are saying that an innovation is impacting their learning, I don’t know how you can discount that. We should collect and aggregate this data.
A school board will always insist that you would need to look at student achievement data, too. Which is a piece of it, but can’t be considered the whole reason why you would adopt or not adopt a certain technology. Even under the best of circumstances, it can be challenging to disaggregate the reasons for shifts in achievement scores and attribute them to a single factor such as technology.
Professional Learning Networks
A network of peers offers a mechanism for aggregating and crowdsourcing teachers’ individual assessments of technology. This also offers a support mechanism for ongoing professional development and learning. One faculty member commented,
I like Joseph South’s suggestion of equipping future teachers with a list of questions that they should ask about adoption of any new technology. But we should also link teachers to a sounding board of other teachers in a professional learning network (PLN).
These networks are places where teachers have opportunities to solve immediate problems. They can also further leverage the technology because many teachers who begin using these networks to learn about new technologies or how to problem solve with them later engage in collaborative projects with other teachers. This can lead to co-creation of activities. In some cases, their K-12 students may also begin collaborating as well. Connecting preservice teachers into networks that will still exist beyond their teacher education is a crucial step.
The same could be true for principals and superintendents, who are often even more isolated than teachers. There is only one of them in a school and a few of them in a district. They may even be in competition with the other principals in their districts in some cases. So they also have a strong need to collaborate beyond their school or their district. A professional learning network offers them opportunities to find other districts and principals who have already tried a certain technology, and who have already invested money in it. They can communicate with those principals to ask “What have your experiences been? I see that you’ve just adopted X technology, what’s your experience in your district been?”
Going forward, we can make better use of technology in schools by building on existing frameworks and expanding to achieve greater reach and scale in our schools and in our teacher/leader education preparation programs. Existing mechanisms and processes can be used as a starting point. A coalition of teacher educator associations is in the process of developing a series of Technology Competencies for Teacher Educators. A parallel set of standards for superintendents is currently being revisited as well. This is an opportune time to consider goals and objectives to improve our institutional competence. Some preliminary recommendations that might be considered are:
- Introduce technology in a pedagogical and content-specific context rather than in isolation.
- Rather than focusing on teaching specific technologies that may quickly become obsolete, prepare teachers to learn how to learn about new technologies that will emerge throughout their professional careers.
- Equip teachers with the knowledge and skills needed to evaluate learning outcomes associated with use of new technologies.
- Provide superintendents and principals with a pedagogical framework that allows them to understand how technology use may differ across grade levels and content areas.
- Connect preservice teachers and future education leaders to professional learning networks that they will continue to use throughout their professional careers.
- Connect the use of technology with pedagogical approaches in content areas supported by learning science and instructional design findings, such as in the area of project-based learning.
- Ensure that teacher education and educational leadership faculty members have the requisite knowledge and skills to deliver a curriculum that is consistent with the preceding recommendations.
One educational leader commented,
If anyone knew how much school districts are spending on technology and how much is sitting in the teachers’ closet, not ever being used, the public would rise up in arms.
Taking the necessary steps to address this need will not be easy or trivial. One respondent commented,
The thing that needs to happen next is the hard work of figuring out how to actually implement the goals that we have outlined. It is easy to say, “Yes, we agree with the idea of making technology program-wide and program-deep.” But getting that to actually happen is more complicated.
Many if not most methods faculty believe that the curriculum is already very packed. They wonder, “How could we fit in new topics such as technology integration?” So they are going to view technology as yet another topic added to an already overflowing plate.
Despite the challenges of implementing the reforms suggested, the cost of inaction is greater than the cost of action. Joseph South summed up the challenge in this way:
Schools of education cannot remain on the technological sidelines. In order to make effective use of the enormous national investment in educational technologies, schools of education must prepare future educators to make effective technological decisions. If schools of education are to remain relevant in an increasingly technological future, they must determine how best to do this.
Moreover, American society needs to properly support education at all levels to remain a leader of the world and globally competitive in nearly every domain. A high priority going forward is to systemically and systematically improve learning and instruction at every level in the American education system.
AACTE Committee on Innovation and Technology. (Ed.). (2008). The handbook of technological pedagogical content knowledge (TPCK) for educators. Mahwah, NJ: Lawrence Erlbaum Associates.
Mishra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108(6), 1017-1054.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4-14.
U.S. Department of Education. (2016a). Education technology and teacher preparation brief. Washington, DC: Office of Educational Technology.
U.S. Department of Education. (2016b). National educational technology plan. Washington, DC: Office of Educational Technology.