K-12 Technology Leaders: Reported Practices of Technology Professional Development Planning, Implementation, and Evaluation

Effective technology professional development (tech-PD) has the potential to impact and shape teacher technology integration practices (Longhurst et al., 2016; Martin et al., 2010; Meyers et al., 2009; Wells, 2007). To change teacher technology integration practices, studies have shown that effective tech-PD includes personalized, sustained, and contextual approaches (Lawless & Pellegrino, 2007; Longhurst et al., 2016; Meyers et al., 2016; Wells, 2007). While these approaches for effective tech-PD are discussed extensively throughout the literature, often teachers report tech-PD as being ineffective (Bill & Melinda Gates Foundation, 2014; U.S. Department of Education [DOE], 2016).

Studies have suggested that teachers have perceived tech-PD as ineffective due to a lack of personalized, sustained, or contextual approaches (Bissonnette & Caprino, 2015; Duran, Brunvand, Ellsworth, & Şendağ, 2011). Therefore, exploring the approaches being utilized when tech-PD experiences are planned and implemented is important to determine if alignment exists between current practices and research-based effective approaches.

Often, the person responsible for tech-PD experiences is the K-12 technology leader (e.g., technology coordinator, technology coach, integration specialist, or innovation specialist). The K-12 technology leader is a growing role that often includes, but is not limited to, the in-house planning, implementation, and evaluation of tech-PD for a school or district (Frazier & Hearrington, 2017; Peterson, 2015; Stanhope & Corn, 2014; Sugar & Holloman, 2009; Sugar & van Tryon, 2014).  Therefore, exploring K-12 technology leaders’ reported practices is important in terms of planning, implementing, and evaluating tech-PD.

Since effective tech-PD has the potential to impact teacher practices (e.g., Longhurst et al., 2016), gaining a better understanding of technology leaders’ reported practices will allow  comparison of those practices to research-based effective tech-PD approaches. This comparison enables a determination of any gaps between technology leaders’ reported practices and research-based effective tech-PD approaches, which may be one reason for teachers’ negative perceptions of tech-PD.

The current study investigated the reported practices of K-12 technology leaders who are members of the International Society for Technology in Education when planning, implementing, and evaluating tech-PD.  Technology leaders’ tech-PD practices were examined through questionnaire, interview, and artifact data.  Specifically, this study sought to answer three research questions:

1. How do ISTE-associated K-12 technology leaders plan tech-PD experiences?
2. How do ISTE-associated K-12 technology leaders implement tech-PD experiences?
3. How do ISTE-associated K-12 technology leaders evaluate tech-PD experiences?

Importance of Tech-PD in K-12 Schools

In the United States, some studies and reports have suggested that technology in K-12 schools has not been used to its full potential (Bauer & Kenton, 2005; Cuban, Kirkpatrick, & Peck; 2001; Ertmer & Ottenbreit-Leftwich, 2013; Grundmeyer, 2013; Hixon & Buckenmeyer, 2009; Swan & Dixon, 2006, U.S. DOE, 2016, 2017). This lack of technology use by K-12 teachers has also been attributed to ineffective tech-PD (Donavan, Green, & Hartley, 2010; OECD, 2015).

Similarly, effective tech-PD has been linked to beneficial changes in teacher technology integration practices and has been suggested as critical to the successful implementation of technology in K-12 schools (Blocher, Armfield, Sujo-Montes, Tucker, & Willis, 2011; Longhurst et al., 2016; Meyers et al., 2016; Schrum & Levin, 2013). For example, in a 2016 study, Longhurst et al. found that teachers who participated in a 2-year tech-PD implementation increased their technological knowledge and skills, changed their technology integration practices, and helped increase student achievement at a significantly higher rate than did their peers who participated in only 1 year of tech-PD and those who did not participate.

Studies have also shown that ineffective classroom technology integration can often be attributed to ineffective tech-PD that failed to equip teachers with adequate knowledge, skills, and integration examples (Duran et al., 2011; Lawless & Pellegrino, 2007; Project Tomorrow, 2014).  For example, Duran et al. (2011) investigated a tech-PD involving 207 teachers and administrators on the topic of wikis.  Of the teachers who participated in the follow-up questionnaire (n = 16), those that did not report continued use of the wikis (n = 5) described the tech-PD as ineffective, citing it as being too short, lacking time to practice, and lacking continued and personalized support.

In other words, when teachers have perceived tech-PD as ineffective, they have been less likely to integrate that specific technology into their classroom practices (Duran et al., 2011; Lawless & Pellegrino, 2007; Potter & Rockinson-Szapkiw, 2012). Therefore, to support teachers’ integration of technology, tech-PD needs to be effective. The challenge remains to differentiate ineffective tech-PD from effective tech-PD.

Ineffective Technology Professional Development

Studies have suggested that tech-PD has often been conducted as one-size-fits-all workshops (Bill & Melinda Gates Foundation, 2014; Desimone, Porter, Garet, Yoon, & Birman, 2002; Lawless & Pellegrino, 2007). Typically, one-size-fits-all tech-PD workshops are conducted as a single, one-time event, focused on a specific application or technology, as opposed to offering strategies to integrate technology into teaching (Lawless & Pellegrino, 2007).

Despite its prevalence, the one-size-fits-all tech-PD workshop approach has been shown to be ineffective in influencing teachers’ technology integration practices (Desimone et al., 2002; Meyers et al., 2016; Sugar, 2005). Additionally, studies have shown that teachers were more likely to perceive PD as ineffective if it did not include personalized support for specific technology integration needs and abilities (Bissonnette & Caprino 2015; Duran et al., 2011) or if it was not perceived as being situated in context (Bissonnette & Caprino 2015; Telese, 2012).  While expansive research on effective tech-PD planning and implementation approaches exist, studies and reports continue to suggest that teachers perceive tech-PD as ineffective (Bill & Melinda Gates Foundation, 2014; U.S. DOE, 2016).

Effective Technology Professional Development

Numerous studies have documented effective tech-PD planning and implementation approaches (e.g., Longurst et al., 2016; Meyers et al., 2016).  For tech-PD to be effective, it should impact teacher technology integration practices (Lawless & Pellegrino, 2007; Schrum & Levin, 2013). We reviewed multiple empirical studies examining the planning and implementation approaches of effective tech-PD, which are outlined below in Table 1.  This review was conducted to illustrate research-based effective tech-PD approaches so that we might better compare these approaches to the reported practices of K-12 technology leaders in our study.

Table 1
Planning and Implementation Approaches of Effective Tech-PD

Overall, these studies suggest that effective tech-PD is aligned with individual teacher needs (e.g., Duran et al., 2011) and offered in a sustained and continuous manner (e.g., Longhurst et al., 2016).  Additionally, the content and delivery of tech-PD should be hands on (e.g., Duran et al., 2011) and situated within the actual classroom or school context where it will be used (e.g., O’Hara, Pritchard, Huang, & Pella, 2013). One way to achieve many of these approaches is through the use of coaching or mentoring (e.g., Swan & Dixon, 2006) or the use of communities of practices (e.g., Meyers et al., 2016).

Finally, effective tech-PD should be evaluated using multiple evaluation measures in order to determine the effectiveness (e.g., Lawless & Pellegrino, 2007).  Overall, these effective tech-PD approaches offer an alternative to the standard, one-time workshop, which is typically reported as being ineffective (e.g., Desimone et al., 2002).

Planning, Implementing, and Evaluating Technology Professional Development

In terms of technology leaders, the International Society for Technology in Education (ISTE) is a popular organization that provides professional development to educational technology leaders at the K-12 level. Currently, 26% of ISTE members describe their primary job role as being a technology coordinator, coach, or director (ISTE, 2018). At the most recent ISTE conference in 2017, over 3,000 technology leaders/coaches attended (ISTE, 2018). We drew our sample from this organization’s membership. ISTE also publishes The Technology Coordinator’s Handbook (Frazier & Hearrington, 2017) to help advise technology leaders in their design of tech-PD.  The authors suggest three main components of designing tech-PD: planning, implementing, and assessing/evaluating. thers have also suggested similar simplified instructional design models.

In an educational technology textbook, Newby, Stepich, Lehman, Russell, and Ottenbreit-Leftwich (2010) suggested a three-step instructional design model to help guide the design of tech-PD: the Plan, Implement, Evaluate (PIE) model.  This model has previously been used in scholarship to analyze factors affecting the quality of tech-PD (Yıldırım, Kurşun, & Göktaş, 2015).  The PIE model has also been used in teacher education programs as a framework to support preservice teachers’ development of effective technology integration practices (e.g., Lehman & Richardson, 2004). We used this simple instructional design model to understand, classify, and analyze the tech-PD design decisions of technology leaders.

Examining the connection between tech-PD design and this model more specifically, we found the literature to recommend that professional development experiences should be purposefully planned and implemented, with clear goals identified for evaluation purposes (Desimone, 2009; Guskey, 2002; Newby et al., 2010; Wells 2007). During the planning phase, the technology leader should create a lesson plan for the experience, as well as collect data related to the teachers and their needs (Newby et al., 2010).  Collecting this data is typically referred to a needs assessment, with a goal of identifying the current knowledge/comfort teachers’ hold on the topic, and finding the distance between that knowledge and the end goal (Lee, 2005; Sleezer & Russ-Eft, 2010; Swart & Kaufman, 2009).

The data collected in a needs assessment can take on a variety of forms, such as surveys, observations, or even interviews (Kopcha, 2010; Lee, 2005; Sleezer & Russ-Eft, 2010; Swart & Kaufman, 2009).  Overall, the planning of effective tech-PD should incorporate a formal needs assessment in order to understand the current needs and ability levels of the teacher participants (Kopcha, 2010; Lee, 2005). This recommendation aligns with the aforementioned effective tech-PD approach of being personalized to individual teacher needs (e.g., Duran et al., 2011). In other words, by conducting a formal needs analysis, technology leaders are better able to align and personalize tech-PD to individual teacher needs.

The implementation phase is the “when, where, and how” (Newby et al., p. 11) of the tech-PD experience.  In general, tech-PD experiences have been implemented as single, one-size-fits-all workshops, which have not been effective in impacting teacher technology integration practices (Desimone, 2002; Meyers et al., 2016; Sugar, 2005).  To impact teacher technology integration practices, effective tech-PD implementation should follow the recommendations in the literature and be sustained and continuous (Duran et al., 2011; Kopcha, 2012; Longhurst et al., 2016; Meyers et al., 2016; O’Hara et al., 2013; U.S. DOE 2017).

Follow-up support should be provided to teachers, and coaching and mentoring approaches have been shown to be particularly effective (Glazer, Hannafin, & Song, 2005; Kretlow & Bartholomew, 2010; Swan & Dixon, 2006).  In short, both the planning and implementation design steps should be aligned with the effective tech-PD approaches presented in Table 1.

When evaluating tech-PD, the end result should be evaluated in terms of change in teachers’ technology integration practices (Lawless & Pellegrino, 2007; Wells 2007). However, once a tech-PD experience has been implemented, technology leaders need a strategy for identifying whether changes in teachers’ technology integration practices have occurred.

The evaluation of tech-PD has been considered essential, yet it often fails to move beyond the collection of self-reported teacher perceptions (Gaytan & McEwen, 2010; Lawless & Pellegrino, 2007; Smolin & Lawless, 2011). Kopcha and Sullivan (2007) noted that when self-reports of teacher perceptions are the only source of evaluation data, that data can be misleading and may contain self-presentation bias. Lawless and Pellegrino (2007) pointed out that self-reported data enables researchers only to determine teacher perceptions of tech-PD, as opposed to what has been learned or how the tech-PD has led to changes in teacher technology integration practices.

To understand the full impact of tech-PD on teacher technology integration practices, multiple measures of evaluation should be employed (Lawless & Pellegrino, 2007; Wells, 2007).  Teacher surveys can provide value toward understanding teacher perceptions of tech-PD experiences; however, by examining data that is not self-reported by teachers (e.g. classroom observations, student interviews, student achievement scores), technology leaders can have a more complete understanding of how their tech-PD experiences have impacted teacher technology integration practices (Lawless & Pellegrino, 2007; Wells, 2007).

Role of Technology Leader

Technology leaders have a variety of titles and responsibilities in K-12 settings.  Their titles can include technology coordinators, technology coaches, integration specialists, eLearning specialists, innovation specialists, and so on (Peterson, 2015; Stanhope & Corn, 2014; Sugar, 2009; Sugar & van Tryon, 2014).

A K-12 technology leader may have responsibilities solely focused on technology integration, or there may be situations where technology leaders also cover dual roles as teachers or administrators (McLeod, Richardson, & Sauers, 2015; Richardson & McLeod, 2011; Yu & Prince, 2016). Technology leaders fulfill diverse responsibilities, including providing technical support, writing technology budgets, making purchasing decisions, researching technology solutions, and planning and implementing tech-PD (Frazier & Hearrington, 2017; Peterson, 2015; Stanhope & Corn, 2014; Sugar & Holloman, 2009; Sugar & van Tryon, 2014).  For the purpose of this study, technology leaders were defined as those who have experience planning and implementing tech-PD. All technology leader participants of this study were also members of ISTE.

Method

This study employed a mixed-mode cross-sectional survey design (Dillman, Smyth, & Christian, 2014) to examine the reported tech-PD planning, implementation, and evaluation practices of technology leaders who are members of ISTE.  This specific mixed-mode survey design was selected in order to utilize multiple data sources to better ensure the reliability and validity of the data (Dillman et al., 2014). The primary data were collected from a 23-item online questionnaire.

For the purposes of this study, six close-ended demographic questions and three open-ended questions were analyzed as the primary data sources to address the study’s research questions. The remaining questions that were not used for this study focused on technology leaders’ general perceptions of tech-PD and are being used for a follow-up study.  To triangulate questionnaire results and the approaches reported by ISTE technology leaders to plan, implement, and evaluate tech-PD experiences, we collected follow-up information from six technology leaders through semistructured interviews (Galletta, 2013) and their own tech-PD artifacts (Merriam & Tisdell, 2015).

Procedures

The questionnaire was created using the online platform Qualtrics and distributed via email to the state and affiliate chapters of ISTE. Data from questionnaire responses were triangulated with follow-up data from questionnaire participants who agreed to participate in semistructured interviews.  Interviews were conducted until data saturation occurred matching the major themes identified in the questionnaire responses (Fusch & Ness, 2015). Technology professional development design artifacts were collected from interview participants and analyzed to triangulate interview responses (Merriam & Tisdell, 2015).

Participants

This study explored how ISTE members who identified as K-12 technology leaders reported planning, implementing, and evaluating tech-PD.  All participants who responded to the online questionnaire were members of ISTE as well as self-identified technology leaders (e.g., technology coaches, technology coordinators, eLearning specialists, etc.).  The questionnaire received 190 total responses. However, 35 responses were excluded from counts, as the responses were too vague.  Respondents who identified as having never planned or implemented tech-PD were also removed from the study (n = 2).

As a result of this process 86 technology leaders and 67 dual-role teachers and technology leaders were included in the study as the participants (n = 153).  The majority (74%) had more than 10 years of experience in a K-12 environment.  Additionally, 94% reported using technology for educational purposes on a daily basis. The participants described themselves as having elementary (37%), secondary (38%), and K-12 (25%) focused roles. Email requests for interviews were sent to questionnaire respondents who agreed to participate in a follow-up interview. A total of six interviews were conducted; three participants were technology leaders and three were dual-role teachers and technology leaders.  All six interview participants submitted artifacts that represented their tech-PD design practices.

Data Collection Instruments

Questionnaire. The questionnaire consisted of 23 open- and close-ended questions. There were six demographic questions (e.g., grade levels taught, and number of years teaching) and others were open-ended questions focusing on tech-PD. We focused on three open-ended questions as the primary data source: (a) How do you plan your technology professional development experiences? (b) How do you implement your technology professional development experiences? and (c) Do you evaluate your tech-PD experiences, and if so, how?

Interviews. Semistructured interviews were conducted for 45-60 minutes with three technology leaders and three dual-role teachers and technology leaders.  The interview protocol (see appendix ) included questions about tech-PD planning, implementation, and evaluation experiences. The interview also asked participants to explore a single tech-PD planning, implementation, and evaluation experience in depth using a behavioral event interview (BEI) framework (McClelland, 1978; Schmidt, Finch, & Faulkner, 1992).  The BEI framework facilitated an in-depth reflection on the technology leader’s practices by examining the planning, implementation, and evaluation of a single tech-PD experience.

Design Artifacts.  Each interviewee was asked to submit artifacts related to the planning, implementation, and evaluation of the single tech-PD experience described in the BEI.  A variety of artifacts were submitted, including outlines, blog posts, planning documents, surveys, presentations, and support resources.  Of the six interviews conducted, six collections of design artifacts were received. To maintain interviewee confidentiality, these artifacts are not available in the appendices.

Data Analysis

The primary data for this study were gathered from open-ended questionnaire responses.  The data were triangulated using data from interview responses and artifact analysis. The open-ended questionnaire data were analyzed using content analysis (Patton, 2002) to identify core consistencies and discover emerging themes.  The unit of analysis was the complete group of ISTE technology leaders. Through the inductive content analysis procedures described later in this paper, we were able to generate a collection of emergent themes that aligned with the study’s specific research questions.

The content analysis process began by reading through technology leaders’ questionnaire responses to the three open-ended questions.  Two researchers individually read through all open-ended questionnaire responses and used memoing to identify and take notes on potential emergent themes (as in Miles & Huberman, 1984).  Memoing was employed in order to help the researchers clarify and organize their thinking, while working toward the extraction of emergent themes (Miles & Huberman, 1984).  For example, when examining responses for the question, “How do you plan your technology professional development experiences?” a memo of “planning based on administrative needs” was added to responses such as “requests from administration” and “direction from administration,” noting that these types of responses seemed to be addressing the same emergent theme.

We then met together and reviewed all potential emergent themes from both researchers. When themes overlapped, they were combined, resulting in a condensed list of finalized themes.  For example, planning tech-PD based on “teacher requests” and “teacher needs” were both combined into the category of “individual teachers’ needs.” We expanded on this list of finalized themes by adding definitions for clarity and example responses from the questionnaire (see Table 2).

Table 2
Example of Emergent Themes for the Question, “Do You Evaluate Your Technology Professional Development and If So, How?”

With the finalized themes, each researcher individually coded all open-ended questionnaire responses.  Each response could be coded for as many themes as were applicable. For example, when asked, “How do you plan your technology professional development experiences?” a response of “Choices are made based upon what tools are available, and what the teacher needs are pertaining to their curriculum” was coded under three themes: (a) availability of resources; (b) individual teacher needs; and (c) curriculum or standards.

Upon completion of individual coding, the two researchers met to review the codes for each response. When the codes for a response did not align between the researchers, the researchers discussed until they came to a consensus (as in Saldaña, 2015).  For example, a response to the question, “How do you implement your technology professional development experiences?” of “I prefer work sessions, rather than [teachers] watching me profess what I know,” was coded as “Hands-on” by one researcher and “Modeling” by the other.  After discussion, the final code of “Hands-on” was assigned. The response was not specific enough to indicate whether modeling was occurring, only that teachers were given work time.  When agreement had been reached for all coded responses, the researchers determined the most frequently occurring themes for each research question.

Once the finalized themes were organized by research question, two researchers reviewed the transcribed interview data to examine whether emergent themes were also supported within the interviews. Quotes from the interviews were used as supporting evidence to triangulate questionnaire responses for each research question.  For example, when discussing how she implemented her tech-PD, a technology leader’s statement, “Modeling [the technology] right off the bat and saying ‘try this.’ And now that [they] are already trying it, they can see how easy it is,” was used to support the emergent themes of “Modeling” and “Hands-on” in terms of how ISTE technology leaders were implementing tech-PD experiences.

Finally, design artifacts were analyzed (Merriam & Tisdell, 2015) by each research question, comparing the planning and implementation of tech-PD described in the interview to the evidence provided in the artifacts.  For example, the design artifacts of a technology leader who reported planning tech-PD to include support for a variety of possible participant questions were examined to see if those support resources were present. To improve the trustworthiness of the findings, as well as the rigor of the data analysis, the researchers utilized multiple data sources (Merriam & Tisdell, 2015) and investigator triangulation (Patton, 2002).

The limitations of this study relate to self-reported questionnaire and interview response data, which may have a potential for self-presentation bias (Kopcha & Sullivan, 2007). Through data triangulation and the use of multiple data sources, including tech-PD design artifacts, we have attempted to mitigate this potential bias.  Additionally, with only 153 questionnaire respondents and six interview respondents, we are not able to generalize our findings to all technology leaders across the US.

Results and Discussion

Based on the data from the three sources (questionnaire, interview, artifacts), we identified ISTE technology leaders’ reported practices for planning, implementing, and evaluating tech-PD experiences. Each of these three design steps are described here, with evidence to support the emergent themes. We compared the emergent themes to relevant literature in order to connect our findings with previous research findings and recommendations.

ISTE Technology Leaders Plan Tech-PD Based on Needs

Based on data from the open-ended questionnaire responses, interviews, and artifacts, nine major themes emerged as to how technology leaders plan their tech-PD experiences (see Table 3).  The most common theme was that technology leaders plan their tech-PD experiences based on teachers’ needs (n = 92, 60%).  For example, in the questionnaire responses, one dual role high school teacher and technology coach described how he used teacher input to identify relevant content: “I design my PD based on teacher feedback…and the ability of the PD to be differentiated based on an assessment of teacher skills and interests.” Another technology specialist described using teacher input during her planning: “Generally I seek out input from teachers as to what they want to learn.”

Table 3
Most Common Emerging Themes From Responses to the Questionnaire Item, “How Do You Plan Your Technology Professional Development Experiences?”

The ISTE technology leaders in the questionnaire mentioned different ways of gathering this information from their teachers, including sending out surveys, observing teachers’ needs from working with them in the classroom, emailing teachers, and collecting informal and anecdotal feedback. For example, one technology specialist reported that she relied on “observational data to provide some suggestions and supports” while planning her tech-PD experiences.

Additional technology leader considerations for planning tech-PD included administrative needs (24%), school needs (11%), or district needs (11%). For example, one technology leader stated in the questionnaire that for his planning process, “administration dictates what it is and what it should be like.”

The idea of planning tech-PD that aligned with teachers’ individual needs was also a common theme in interview responses and was discussed by all six interviewees. For example, one K-12 technology leader described planning tech-PD by “finding the starting point of where teachers currently are instead of where we think they are [by having them] make lists of the technologies they use in their classrooms.” One K-12 director of instructional technology raised a similar point, noting that teachers have different needs based on their ability levels: “My first thought when I teach the classes is, ‘we’re probably going to have people in there that can’t even send an email attachment,’ so I always keep that in mind.”  She plans her tech-PD to include “the most simplified version as possible” but also allows for more advanced teachers to “just go on and not follow lockstep.”

Like questionnaire respondents, interviewees all mentioned different ways of gathering current needs from their teachers, including surveys, anecdotal conversations, and feedback from previous tech-PD sessions.  For example, when discussing the planning of tech-PD, one director of instructional technology reported that she ends her tech-PD sessions by asking each participant, “How did [the tech-PD] align with your goals, and what more do you need?”  She then uses those reported needs to inform the planning of her future tech-PD sessions.

This focus on planning tech-PD based on teachers’ needs was also supported in the artifact analysis.  For example, in a blog post outlining her tech-PD planning process, one dual role computer science teacher and technology coach discussed preparing a variety of Makerspace examples based on the different subject areas and curricular topics her teachers would be interested in covering.

Despite these responses supporting the planning of tech-PD based on teachers’ individual needs, one dual role secondary computer science teacher and technology leader discussed in his interview the difficulties that arise when trying to differentiate tech-PD: “It’s really hard to develop professional development with a group of teachers that is differentiated enough for their needs…. Depending on who they are, they might need a lot more one-on-one help, and that’s a big investment.”

In addition, technology leaders reported difficulty correctly assessing teacher needs, noting that teachers’ self-reported data sometimes did not align with what teachers actually needed.  As one technology leader responded in the questionnaire, “Many times, teachers don’t know what they want to learn.”  This assertion aligns with Kopcha and Sullivan’s (2007) finding that self-reported teacher data can often be unreliable.

Based on the literature, studies have expressed the importance of basing tech-PD on teachers’ needs (e.g., O’Hara et al., 2013). However, only 60% of ISTE technology leaders mentioned planning tech-PD based on teachers’ needs. This result implies that 40% of this group of ISTE technology leaders are planning tech-PD based on other considerations, such as administrative, school, and district needs. This circumstance could be one reason why teachers’ perceptions of tech-PD continues to be negative (e.g., Bill & Melinda Gates Foundation, 2014; U.S. DOE, 2016).

As the literature has suggested (e.g., Bissonnette & Caprino 2015), tech-PD not aligned to individual teacher needs has often been perceived as ineffective. While 60% (n = 92) of our technology leaders specifically mentioned focusing on individual teacher needs, planning tech-PD based on the combined needs of teachers, administrators, the school, and the district was mentioned by all but 14 (9%) technology leaders.  In other words, technology leaders typically use school stakeholders to help identify technology concepts that need to be addressed. Those who did not mention focusing on needs, instead, described relying on research, advice from their professional learning network, or their own experiences.  For example, one technology leader reported in the questionnaire that he plans his tech-PD based on “research and learning what has worked from others,” noting the value of centering tech-PD around what has been reported as previously successful.

The reason teachers may perceive that tech-PD lacks focus on their own needs may be the combination of the fact that differentiated tech-PD is difficult to plan and that teachers may not always be aware of their needs. In other words, teachers may not be aware of their needs until after receiving tech-PD or attempting to implement new technology in the classroom.  Thus, planning for those unknown needs becomes a difficult challenge, particularly when technology leaders are also attempting to differentiate tech-PD.

ISTE Technology Leaders Implement Tech-PD Through a Variety of Methods

Based on data from open-ended questionnaire responses, interviews, and artifacts, nine major themes emerged as to how technology leaders described implementing their tech-PD experiences (see Table 4).

Table 4
Most Common Emerging Themes From Responses to the Questionnaire Item, “How Do You Implement Your Technology Professional Development Experiences?”

In terms of their implementation process, technology leaders discussed a wide variety of approaches when implementing tech-PD.  In the questionnaire, technology leaders were asked, “How do you implement your technology professional development experiences?” Technology leaders described many unique approaches for tech-PD experiences. While using a “hands-on” approach was the most common questionnaire theme (n = 49, 32%) and was mentioned by all six interviewees, a wide variety of other themes emerged for how tech-PD was implemented.  For example, in the questionnaire, one middle school technology coach described the importance of incorporating multiple approaches within tech-PD experiences:

We offer a variety of types of training, from introductory overviews of emerging tech, to one-on-one sessions, to stand-alone video tutorials (mostly screencasts), to printed directions.  My philosophy is that it’s best to provide PD in as many formats as possible. Just as in the classroom, no one method works for everyone.

Interviewees also mentioned the importance of variety in terms of tech-PD implementation approaches.  One K-12 Innovation Specialist discussed incorporating variety throughout her typical tech-PD implementation to provide teachers with a range of experiences so they can implement the pieces of the tech-PD that are most relevant and applicable to them:

We want the participants to experience [the PD] on their own and not be told everything from a person standing up at the front of the room, because that’s not what we want our teachers to embody, because that’s not what we want our students to experience….We do all of it blended so they can have access to all of the different activities and curriculum…so they can take it and transfer it over two their curriculum and use it however they want.

This theme of variety was also found in the artifact analysis. Of the six interviewees, all six showed evidence of including a variety of tech-PD approaches. For example, one technology leader’s tech-PD implementation used a mix of presentations, hands-on time, and activities focused on making the content applicable and relevant.  A dual role secondary mathematics teacher and technology leader described beginning his tech-PD with a presentation, and then switching to modeling and hands-on work time, so that teachers could better try “things to see what [the technology] was and to see how it worked.”

Technology leaders’ main rationale for incorporating variety seemed to be to support teachers’ individual needs.  For example, one technology leader from the questionnaire stated, “I make a variety of technology options available, and I am prepared to provide just-in-time training should any teacher want to use a tech tool that they aren’t currently feeling comfortable with.”  In other words, technology leaders reported wanting to make their tech-PD sessions relevant and applicable by utilizing a variety of approaches to ensure that teachers across different grades, subjects, and ability levels all had their needs met.

These results suggest that in addition to implementing many of the effective tech-PD approaches outlined in the literature such as being hands-on (e.g., Hixon & Buckenmeyer, 2009) and applicable and relevant (e.g., O’Hara et al., 2013), technology leaders also attempted to incorporate a variety of approaches to address teachers’ individual needs and preferences. However, even effective tech-PD approaches may not be perceived as beneficial by teachers if the planning and implementation does not take into account individual teacher needs (e.g., Bissonnette & Caprino, 2015; Duran et al., 2011).  As previously stated, 40% of technology leaders reported planning tech-PD that was not based on individual teacher needs.

Overall, there appeared to be a strong relationship between the planning and implementation phases of tech-PD, centered around the individual needs of teachers.  The planning of tech-PD is typically reported as being based on the individual needs of teachers.  Additionally, it appears technology leaders are utilizing a variety of implementation approaches to better meet the diverse needs of the teachers they serve.

For example, one technology leader discussed how he shows teachers “a variety of exemplars and then ask[s] them what they want to investigate further” based on their specific needs.  Another stated, “All teachers do not teach the same; therefore, personalization [of tech-PD] is necessary.  All teachers have a variety of learning curves and learning styles.” This participant emphasized the variety of implementation strategies she used based on individual teacher needs. This relationship between the planning and implementation phases also aligns with the PIE model (Newby et al., 2010), which discusses how tech-PD planning should impact implementation, and implementation should impact future planning (p. 13). This theme of an interconnected relationship between planning and implementation, centered around the individual needs of teachers, would appear to be contradictory to teachers’ reported negative perceptions of tech-PD from other research studies (e.g., Bill & Melinda Gates Foundation, 2014; U.S. DOE, 2016).

ISTE Technology Leaders Do Not Conduct Systematic Evaluations of Tech-PD

Based on data from open-ended questionnaire responses, interviews, and artifacts, the following themes emerged for how technology leaders evaluate tech-PD.  In an open-ended questionnaire response, technology leaders described their process for evaluating their tech-PD (see Table 5). In addition, six technology leaders described their general tech-PD evaluation process during interviews.

In the questionnaire, 91% (n = 139) of respondents reported conducting some form of tech-PD evaluation (see Table 5).  The most common form of evaluation was surveys of self-reported teacher feedback (n = 64, 42%). Most technology leaders reported evaluating tech-PD with self-reported data from teachers in the form of surveys and in-person feedback.  Only four questionnaire respondents discussed conducting classroom observations, and none mentioned the incorporation of student data.

One technology leader discussed how she conducted “informal observations of classroom practices” to better understand how tech-PD material was implemented in the classroom.  Seven of the technology leaders who reported not conducting evaluations (n = 16, 10%) mentioned a desire to incorporate this practice into future sessions: “No, but I should. I used to do a google survey at the end. Need to get back in the practice!”

Table 5
Most Common Emerging Themes From Responses to the Questionnaire Item, “Do You Evaluate Your Technology Professional Development and If So, How?”

All six interview respondents mentioned evaluating tech-PD with self-reported teacher data, more specifically, teachers’ perceptions of the tech-PD. Two respondents discussed going beyond self-reported teacher data.  One technology leader mentioned she used classroom observations to evaluate teachers’ perceptions of tech-PD: “We get some feedback by going to classrooms and observing teachers.”  She then used the data collected during observations to inform specific “coaching strategies that can be discussed with the teacher afterwards.” A second technology leader discussed her process for collecting student feedback and reflections: “Every semester we ask all of our middle school students for feedback” on how technology had been integrated into their classrooms, she said.  This data is then used to “make changes, tweaks, and adaptations to the curriculum over the summer, and then to inform the planning of tech-PD and collaborative time to for teachers.”  While this type of evaluation does not take into account student achievement data, it does move beyond self-reported teacher data by examining student perceptions of technology use in the classroom and incorporating that reflection data into the planning of future tech-PD sessions.

Two interviewees specifically mentioned their desire to improve upon their tech-PD evaluation practices, stating that the use of teacher self-reported data was a perceived area of weakness.  One dual role computer science teacher and technology coach stated that his evaluation practices were likely to be biased due to self-reported data:

[We do] a lot of self-evaluation currently. We would like to do more formal evaluation.…We do surveys every once in a while, but I find that if [teachers] know who is going to be receiving it, they sometimes fudge those.

A different secondary computer science teacher and technology coach echoed this sentiment, saying the survey feedback lacked the substance necessary to impact future tech-PDs: “[Survey results are] rarely very revealing to some degree. It’s difficult to get a good survey.… They don’t often tell us a whole lot.”

Some studies have shown that teachers’ self-reported data may be biased and does not correctly evaluate whether tech-PD has been effective (Kopcha & Sullivan, 2007). Lawless and Pellegrino (2007) recommended that to measure the effectiveness of tech-PD, evaluators should move beyond self-reported teacher feedback to classroom observations and the collection of student data.  Moving beyond self-reported teacher feedback allows technology leaders to better understand the impact tech-PD has had on teacher technology integration practices (Lawless & Pellegrino, 2007; Wells, 2007).

However, although best practices for evaluating effective tech-PD may require moving beyond self-reported data, practical difficulties exist in the implementation of this solution. For example, Sugar and Holloman (2009) found that technology leaders often have additional responsibilities, such as providing technical support, planning and implementing technology policies, and making technology purchases.  While ISTE technology leaders need to work on incorporating evaluation data from a wider variety of sources, the practicality of conducting classroom observations and collecting student achievement data may not be realistic due to time constraints. In other words, although self-reported data is not considered best-practice, it may be the only feasible evaluation strategy given time constraints.

Furthermore, self-reported data provides technology leaders with an understanding of teacher perceptions of tech-PD. It is important for technology leaders to understand teachers’ perceptions, since other studies and reports have found that teachers report tech-PD as negative (Bill & Melinda Gates Foundation, 2014; U.S. DOE, 2016).  Therefore, for technology leaders who are not currently conducting evaluations, self-reported teacher data is a beneficial place to begin. Relying only on teacher self-reported evaluation data might also be a reason that a misalignment appears to exist between technology leaders’ planning and implementation practices and teachers’ negative perceptions. If technology leaders were able to conduct classroom observations and examine student data, they might be more aware of gaps that could exist between tech-PD experiences and teachers’ technology integration practices.

Implications

Based on the data, most ISTE technology leaders reported using research-supported best practices for planning and implementing tech-PD (e.g., Longhurst et al., 2016; Meyers et al., 2016).  The majority of ISTE technology leaders’ reported planning and implementing tech-PD around teachers’ individual needs.  However, this result seems to contradict reports of teachers perceiving tech-PD as ineffective due to a lack of focus on their needs (Bill & Melinda Gates Foundation, 2014; U.S. DOE, 2016). If many ISTE technology leaders are using best practices to plan and implement tech-PD, why would teachers continue to report having negative perceptions toward tech-PD experiences? The following recommendations for ISTE technology leaders are based on our findings and may increase the likelihood that teachers will find tech-PD effective and useful:

Planning Should Incorporate Needs Assessments

One reason for teachers’ reported negative perceptions from other studies may be that, although many ISTE technology leaders consider teachers’ needs, not all take teachers’ needs into account. According to our questionnaire results, only 60% reported planning tech-PD based on individual teachers’ needs, and only three specifically reported conducting a formal needs assessment (e.g., Kopcha, 2010).  The remainder reported using methods like anecdotal conversations and self-reported survey data to plan based on teacher needs. Other technology leaders focused their planning on school, administrative, and district needs, which may not align with teachers’ needs.  This finding raises the question, should the planning of tech-PD necessarily incorporate teachers’ needs, or are there times when planning based on administrative or district needs may better provide teachers with what they need to be successful?   For example, one technology leader reported in her interview that, at times, she plans tech-PD sessions where the focus is not necessarily hands on (which research shows teachers prefer), but is instead “aimed at developing a growth mindset” (which is based on an administrative request).

Regardless of the source of needs (teacher, administrator, school, or district), a formal needs assessment should be an integral component of the tech-PD planning process.  While collecting data on teachers’ individual needs may be one way to make tech-PD more personalized and, therefore, perceived by teachers as more effective, technology leaders must consider administrative, school, and district needs as well.  For example, in her interview, one secondary technology coach described navigating the needs of different school stakeholders:

[For teachers] technology PD has to be immediately applicable in class.  Administrators, they want the theory…. Teachers don’t always recognize that as important.  Instead, [teachers] want an understanding of how to take an application and use it in class tomorrow. It has to be practical to [teachers], so you have to kind of sneak the other stuff in when they don’t realize it.

In other words, technology leaders must often act as the bridge or link between different sets of stakeholder needs and plan their tech-PD accordingly.  This idea of bridging or linking stakeholder needs has also been discussed in research on K-12 literacy coaching and leadership (e.g., Calo, Sturtevant, & Kopfman, 2015).

Based on these findings, we recommend that ISTE technology leaders conduct a needs assessment across all stakeholder groups when planning tech-PD.  This step will allow technology leaders to gain a more comprehensive understanding of different sets of stakeholder needs (Lee, 2005; Sleezer & Russ-Eft, 2010; Swart & Kaufman, 2009).  When conducting a needs assessment, collecting only self-reported survey data may not be reliable (Kopcha & Sullivan, 2007). Therefore, we also recommend incorporating multiple sources of data into needs assessments, including interviews and classroom observations (e.g., Kopcha, 2010).

We also recommend planning tech-PD that can be differentiated to individual teacher needs.  For example, Gamrat, Zimmerman, Dudek, and Peck (2014) examined the use of a digital badge system to support teachers’ online tech-PD.  Teachers were allowed to choose tech-PD activities that were relevant to their individual needs and were awarded badges for the completion of these activities.  Expert mentors were provided to teachers for support, but teachers were given complete control over the selection of learning activities.  The authors found that providing this flexible, differentiated tech-PD allowed teachers to select learning activities that were relevant to their needs, which was perceived as helpful and beneficial.

In addition, we also recommend finding methods to establish a shared technology vision across schools and districts (e.g., Hew & Brush, 2007; Reigeluth, Carr-Chellman, Beabout, & Watson, 2009). The process of establishing a shared technology vision should involve teachers, administrators, community members, and other K-12 stakeholders. Once a shared vision has been established, it can act as a guide for technology integration practices (Hew & Brush, 2007), including tech-PD planning and implementation, which can help schools and districts move away from one-time tech-PD workshops and move toward more sustained and continuous tech-PD approaches.

Implementation Should Be Sustained and Continuous

Overall, ISTE technology leaders reported implementing tech-PD in ways that aligned with effective research-based approaches (e.g. hands-on, relevant, and applicable).  However, technology leaders in our study did not commonly report using sustained and continuous implementation approaches.  Coaching and mentoring is one example of sustained tech-PD (e.g., Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009; Israel, Carnahan, Snyder, & WIlliamson, 2012; Swan & Dixon, 2006) but was mentioned by only 15 technology leaders (10%).  Similarly, providing follow-up (or just-in-time) support is typically considered a form of sustained tech-PD (e.g., Kopcha, 2012; Lawless & Pellegrino, 2007; Swan & Dixon, 2006), and was only mentioned by 10 questionnaire respondents (7%).

During the behavioral event interview, all six interviewees described a tech-PD session that was implemented as a one-time workshop; however, three mentioned providing follow-up support for those sessions.  Typically, however, this follow-up support was more passive in nature, with technology leaders offering their email addresses to participants or providing web pages with support resources, rather than actively following-up with participants.  For example, one secondary computer science teacher and technology coach reported receiving occasional email follow-up questions, but no interviewees reported actively following-up with support that was not requested by participants.

All reported that follow-up support was provided at the impetus of the teacher, not the technology leader. This finding aligns with Glazer and Hannafin’s (2008) finding that technology leaders may prefer “to be regarded as a supportive peer…assuming [teachers will] seek help when needed” (p. 56).  However, studies suggest that technology leaders who take a more active coaching role by cooperatively developing lesson plans, providing one-on-one support and encouragement, and providing just-in-time support can have a significant impact on teacher technology integration practices (e.g., Lowther, Inan, Daniel Strahl, & Ross, 2008).  Implementing a peer mentoring technology coaching model (e.g., Glazer & Hannafin, 2008) may be one method for schools and districts to shift toward more sustained and continuous tech-PD.

The lack of sustained and continuous tech-PD may be one reason why teachers in other studies have reported negative perceptions (e.g., Bissonnette & Caprino, 2015; Duran et al., 2011).  ISTE technology leaders in this study may not have the time or resources to implement the kind of necessary follow-up, but it is critical to teachers’ successful technology implementation. Therefore, when planning tech-PD, technology leaders should establish a plan for follow-up and continued support. Studies have shown that this follow-up support is likely to be more impactful than one-time workshops (e.g., Lawless & Pellegrino, 2007).

Evaluation of Tech-PD Should Move Beyond Self-Reported Data

The majority of questionnaire respondents (n = 139, 91%) reported conducting tech-PD evaluations, as did all six interviewees. However, most evaluation data were self-reported teacher perceptions of the tech-PD.  This finding aligns with Gaytan and McEwen’s (2010) literature review, which suggests that most tech-PD evaluations fail to move beyond self-reported data collection.  Therefore, we recommend moving beyond self-reported data to include observations and student achievement data whenever possible (Lawless & Pellegrino, 2007).

In general, evaluation data should be used to determine if tech-PD has led to changes in teacher technology integration practices (Lawless & Pellegrino, 2007; Wells 2007).  As one technology leader in this study said, the collection of student reflection data can be beneficial in understanding students’ perceptions of teacher technology integration practices, as well as understanding if changes in teacher technology integration practices have occurred.  In turn, that student reflection data can also be used to inform the planning of future tech-PD.

Using evaluation data to inform the planning of future tech-PD is also recommended by the PIE model (Newby et al., 2010).  For example, if student reflection data indicates that certain features of a technology resource are not being be used, focus can be placed on those features in future tech-PD sessions.  In addition to student data, pre and post classroom observations can also be useful in helping technology leaders determine if tech-PD has led to changes in teacher technology integration practices.

Finally, for those ISTE technology leaders who are not conducting evaluations of tech-PD experiences, we strongly recommend embedding evaluation methods to capture the effects of tech-PD and to be more responsive to teacher feedback and needs.  Since conducting systematic evaluations that move beyond self-reported data can be intensive and time consuming, we recommend the establishment of research-practice partnerships as one way to accomplish these types of evaluations.

Research-Practice Partnerships Should Be Established

Given the multiple responsibilities of technology leaders (e.g., Sugar & Holloman, 2009), planning tech-PD through formal needs assessments, implementing tech-PD that is sustained and continuous, and evaluating tech-PD by moving beyond self-reported data, may be an incredibly difficult task to manage.  Therefore, we recommend that researchers work toward establishing more research-practice partnerships (e.g., Nelson & Webb, 2016; Thomas et al., 2012).  For example, Nelson and Webb (2016) described a research-practice partnership where the technology coordinator at a local college of education spent 25% to 50% of her time providing onsite coaching to local K-12 teachers.  This coaching occurred on a monthly basis and was found to be effective in helping the district launch their new technology integration initiative. The researchers concluded that the research-practice partnership approach “was successful in launching an on-site, sustainable technology integration project that will improve teaching and learning in the school district” (p. 3).

Through these types of partnerships, planning, implementation, and evaluation tasks can be shared and further research can be conducted into the efficacy of tech-PD experiences.  Finally, these types of partnerships can also help school districts establish a shared technology vision and engage in systematic change processes to shift their policies and practices (e.g., Joseph & Reigeluth, 2010).

Conclusion

Results from this study suggest that ISTE technology leaders are planning and implementing tech-PD in ways that align with the effective design approaches discussed throughout the literature.  However, these tech-PD experiences may not always be planned based on individual teachers’ needs, which may be due to technology leaders’ requirement to meet the needs of district initiatives or administrator requests to provide specific trainings not necessarily aligned with teachers’ individual needs.  Additionally, while the implementation of these tech-PD experiences may be aligned with the majority of effective approaches outlined in the literature, they are often not reported as being sustained and continuous.

Finally, while these ISTE technology leaders reported conducting evaluations of their tech-PDs, those evaluations did not typically move beyond self-reported teacher data.  While realistic constraints may discourage collection of evaluation data from classroom observations and student achievement scores, moving beyond self-reported teacher data may help identify gaps between the planning and implementation of tech-PD and teachers’ negative perceptions. Based on these results, it appears that a current misalignment may exist between ISTE technology leaders’ tech-PD planning and implementation practices and the negative perceptions that research suggests many teachers hold toward tech-PD experiences.

Additionally, as this study specifically focused on technology leaders within ISTE, more research is needed to further clarify the specific practices of K-12 technology leaders across the US, as well as to help identify why misalignments between technology leaders and teachers may exist. Finally, future researchers should consider engaging in research-practice partnerships (e.g., Nelson & Webb, 2016; Thomas et al., 2012).  These partnerships can offer assistance to technology leaders with the numerous best-practice tasks associated with planning, implementing, and evaluating tech-PD, while at the same time giving researchers an opportunity to explore effective tech-PD approaches.

References

Bauer, J., & Kenton, J. (2005). Toward technology integration in the schools: Why it isn’t happening. Journal of Technology and Teacher Education, 13(4), 519.

Bill & Melinda Gates Foundation. (2014). Teachers know best: Teachers’ views on professional development.

Bissonnette, J. D., & Caprino, K. (2015). A look at ineffective and effective professional development: Moving toward action research. Mid-Atlantic Education Review, 2(1), 12-22.

Blocher, J. M., Armfield, S. W., Sujo–Montes, L., Tucker, G., & Willis, E. (2011). Contextually based professional development. Computers in the Schools, 28(2), 158-169. doi: 10.1080/07380569.2011.577398

Calo, K. M., Sturtevant, E. G., & Kopfman, K. M. (2015). Literacy coaches’ perspectives of themselves as literacy leaders: Results from a national study of K–12 literacy coaching and leadership. Literacy Research and Instruction, 54(1), 1-18.

Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High access and low use of technologies in high school classrooms: Explaining an apparent paradox. American Educational Research Journal, 38(4), 813-834. doi: 10.3102/00028312038004813

Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Washington, DC: National Staff Development Council.

Depta, M. (2015). Best practices in implementation of technology change in the K-12 context. American Journal of Educational Research, 3(12B), 41-56. doi: 10.12691/education-3-12B-9

Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181-199. doi: 10.3102/0013189X08331140

Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81-112.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ: John Wiley & Sons.

Donovan, L., Green, T., & Hartley, K. (2010). An examination of one-to-one computing in the middle school: Does increased access bring about increased student engagement? Journal of Educational Computing Research, 42(4), 423-441. doi: 10.2190/EC.42.4.d

Duran, M., Brunvand, S., Ellsworth, J., & Şendağ, S. (2011). Impact of research-based professional development: Investigation of inservice teacher learning and practice in Wiki integration. Journal of Research on Technology in Education, 44(4), 313-334. doi:10.1080/15391523.2012.10782593

Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of research on Technology in Education, 42(3), 255-284.  doi: 10.1080/15391523.2010.10782551

Ertmer, P. A., & Ottenbreit-Leftwich, A. (2013). Removing obstacles to the pedagogical changes required by Jonassen’s vision of authentic technology-enabled learning. Computers & Education, 64, 175-182. doi: 10.1016/j.compedu.2012.10.008

Frazier, M., & Hearrington, D. (2017). The technology coordinator’s handbook. Eugene, OR: International Society for Technology in Education.

Fusch, P. I., & Ness, L. R. (2015). Are we there yet? Data saturation in qualitative research. The Qualitative Report, 20(9), 1408.

Galletta, A. (2013). Mastering the semi-structured interview and beyond: From research design to analysis and publication. New York, NY: NYU Press.

Gamrat, C., Zimmerman, H. T., Dudek, J., & Peck, K. (2014). Personalized workplace learning: An exploratory study on digital badging within a teacher professional development program. British Journal of Educational Technology, 45(6), 1136-1148.

Gaytan, J. A., & McEwen, B. C. (2010). Instructional technology professional development evaluation: Developing a high quality model. The Journal of Research in Business Education, 52(2), 77.

Glazer, E., & Hannafin, M. (2008). Factors that influence mentor and teacher interactions during technology integration collaborative apprenticeships. Journal of Technology and Teacher Education, 16(1), 35.

Glazer, E., & Hannafin, M. J., & Song, L. (2005). Promoting technology integration through collaborative apprenticeship. Educational Technology Research and Development, 53(4), 57-67. doi: 10.1007/BF02504685

Grundmeyer, T. (2013). Adopting technology: Using student qualitative data and Gartner’s Hype Cycle. Journal of Education and Training Studies, 2(1), 207-216

Guskey, T. R. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), 45-51.

Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55(3), 223-252. doi: 10.1007/s11423-006-9022-5

Hixon, E., & Buckenmeyer, J. (2009). Revisiting technology integration in schools: Implications for professional development. Computers in the Schools, 26(2), 130-146. doi:10.1080/07380560902906070

International Society for Technology in Education. (2018). Meet your market. Retrieved from https://conference.iste.org/2018/exhibitors/demographics.php

Israel, M., Carnahan, C. R., Snyder, K. K., & Williamson, P. (2012). Supporting new teachers of students with significant disabilities through virtual coaching: A proposed model. Remedial and Special Education, 34(4), 195-204. doi: 10.1177/0741932512450517

Joseph, R., & Reigeluth, C. M. (2010). The systemic change process in education: A conceptual framework. Contemporary Educational Technology, 1(2), 97-117.

Kopcha, T. J. (2010). A systems-based approach to technology integration using mentoring and communities of practice. Educational Technology Research and Development, 58(2), 175-190. doi: 10.1007/s11423-008-9095-4

Kopcha, T. J. (2012). Teachers’ perceptions of the barriers to technology integration and practices with technology under situated professional development. Computers & Education, 59(4), 1109-1121. doi:10.1016/j.compedu.2012.05.014

Kopcha, T. J., & Sullivan, H. (2007). Self-presentation bias in surveys of teachers’ educational technology practices. Educational Technology Research and Development, 55(6), 627-646. doi: 10.1007/s11423-006-9011-8

Kretlow, A. G., & Bartholomew, C. C. (2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education, 33(4), 279-299. doi: 10.1177/0888406410371643

Lawless, K. A., & Pellegrino, J. W. (2007). Professional development in integrating technology into teaching and learning: Knowns, unknowns, and ways to pursue better questions and answers. Review of Educational Research, 77(4), 575-614. doi: 10.3102/0034654307309921

Lee, H. J. (2005). Developing a professional development program model based on teachers’ needs. Professional Educator, 27, 39-49.

Lehman, J. D., & Richardson, J. (2004, October). Making connections in teacher education: Electronic portfolios, videoconferencing, and distance field experiences. Proceedings from the 27th annual Association for Educational Communications and Technology (AECT) conference (pp. 418-428).

Longhurst, M. L., Coster, D. C., Wolf, P. G., Duffy, A. M., Lee, H., & Campbell, T. (2016). Multi‐year professional development grounded in educative curriculum focused on integrating technology with reformed science teaching principles. School Science and Mathematics, 116(8), 430-441.

Lowther, D. L., Inan, F. A., Daniel Strahl, J., & Ross, S. M. (2008). Does technology integration “work” when key barriers are removed? Educational Media International, 45(3), 195-213.

Martin, W., Strother, S., Beglau, M., Bates, L., Reitzes, T., & McMillan Culp, K. (2010). Connecting instructional technology professional development to teacher and student outcomes. Journal of Research on Technology in Education, 43(1), 53-74. doi: http://dx.doi.org/10.1080/15391523.2010.10782561

McCleland, D.C. (1978). Guide to behavioral event interviewing. Boston, MA: McBer and Company.

McLeod, S., Richardson, J. W., & Sauers, N. J. (2015). Leading technology-rich school districts: Advice from tech-savvy superintendents. Journal of Research on Leadership Education, 10(2), 104-126. doi: 10.1177/1942775115584013

Merriam, S. B., & Tisdell, E. J. (2015). Qualitative research: A guide to design and implementation. Hoboken, NJ: John Wiley & Sons.

Meyers, C. V., Molefe, A., Brandt, W. C., Zhu, B., & Dhillon, S. (2016). Impact Results of the eMINTS Professional Development Validation Study. Educational Evaluation and Policy Analysis, 38(3), 455-476.

Miles, M. B., & Huberman, A. M. (1984). Drawing valid meaning from qualitative data: Toward a shared craft. Educational Researcher, 13(5), 20-30.

Mouza, C. (2009). Does research-based professional development make a difference? A longitudinal investigation of teacher learning in technology integration. Teachers College Record, 111(5), 1195-1241.

Nelson, R. F., & Webb, L. S. (2016, January). A school-university instructional technology coaching model. Proceedings of the Annual International Conference On Education & E-Learning (pp. 1-3). doi: 10.5176/2251-1814_EeL16.2

OECD.  (2015), Students, computers and learning: Making the Connection, PISA. Paris, FR: OECD Publishing. doi: 10.1787/9789264239555-en

O’Hara, S., Pritchard, R., Huang, C., & Pella, S. (2013). Learning to integrate new technologies into teaching and learning through a design-based model of professional development. Journal of Technology and Teacher Education, 21(2), 203-223.

Patton, M. (2002). Qualitative research and evaluation methods. Thousand Oaks. Sage.

Peterson, L. R. (2015, March). Technology coaches: The heart of technology integration. Proceedings of the Society for Information Technology & Teacher Education International Conference (pp. 1392-1395). Waynesville, NC: Association for the Advancement of Computing in Education.

Potter, S. L., & Rockinson‐Szapkiw, A. J. (2012). Technology integration for instructional improvement: The impact of professional development. Performance Improvement, 51(2), 22-27. doi: 10.1002/pfi.21246

Project Tomorrow. (2014). The new digital learning playbook: Advancing college and career ready skill development in K-12 schools. Retrieved from http://www.tomorrow.org/speakup/pdfs/SU13Educatorreport_WEB.pdf

Reigeluth, C. M., Carr-Chellman, A., Beabout, B., & Watson, W. (2009). Creating shared visions of the future for K-12 education: A systemic transformation process for a learner-centered paradigm. In L. Moller, J. B.Huett, & D. M. Harvey (Eds.), Learning and instructional technologies for the 21st century (pp. 1-19). Boston, MA: Springer US.

Richardson, J. W., & McLeod, S. (2011). Technology leadership in Native American schools. Journal of Research in Rural Education (Online), 26(7), 1.

Saldaña, J. (2015). The coding manual for qualitative researchers. Thousand Oaks, CA: Sage.

Saunders, R. (2014). Effectiveness of research-based teacher professional development. Australian Journal of Teacher Education, 39(4), 10. doi: 10.14221/ajte.2014v39n4.10

Schmidt, B. J., Finch, C. R., & Faulkner, S. L. (1992). Helping teachers to understand their roles in integrating vocational and academic education: A practitioner’s guide (MDS-276). Berkeley, CA: National Center for Research in Vocational Education.

Schrum, L., & Levin, B. B. (2013). Lessons learned from exemplary schools. TechTrends, 57(1), 38-42. doi: 10.1007/s11528-012-0629-6

Sleezer, C. M., & Russ-Eft, D. (2010). Needs assessment. In J. L. Moseley & J. C. Dessinger (Eds.), Handbook of improving performance in the workplace, Vol. 3: Measurement and evaluation (pp. 97-112). Silver Spring, MD: International Society for Performance Improvement. doi: 10.1002/9780470587096.ch6

Smolin, L., & Lawless, K. A. (2011). Evaluation across contexts: Evaluating the impact of technology integration professional development partnerships. Journal of Digital Learning in Teacher Education, 27(3), 92-98. doi:10.1080/21532974.2011.10784663

Stanhope, D. S., & Corn, J. O. (2014). Acquiring teacher commitment to 1:1 initiatives: The role of the technology facilitator. Journal of Research on Technology in Education, 46(3), 252-276. doi:10.1080/15391523.2014.888271

Sugar, W. (2005). Instructional technologist as a coach: Impact of a situated professional development program on teachers’ technology use. Journal of Technology and Teacher Education, 13(4), 547-571.

Sugar, W., & Holloman, H. (2009). Technology leaders wanted: Acknowledging the leadership role of a technology coordinator. TechTrends, 53(6), 66.

Sugar, W., & van Tryon, P. J. S. (2014). Development of a virtual technology coach to support technology integration for K-12 educators. TechTrends, 58(3), 54-62.

Swan, B., & Dixon, J. (2006). The effects of mentor-supported technology professional development on middle school mathematics teachers’ attitudes and practice. Contemporary Issues in Technology and Teacher Education, 6(1), 67-86. Retrieved from https://citejournal.org/volume-6/issue-1-06/mathematics/the-effects-of-mentor-supported-technology-professional-development-on-middle-school-mathematics-teachers-attitudes-and-practice

Swart, W., & Kaufman, R. (2009). Developing performance data for making useful faculty and leadership decisions: Needs assessment as a vehicle. Performance Improvement Quarterly, 22(3), 71-82. doi: 10.1002/piq.20061

Telese, J. A. (2012). Middle school mathematics teachers’ professional development and student achievement. The Journal of Educational Research, 105(2), 102-111.

Thomas, C. N., Hassaram, B., Rieth, H. J., Raghavan, N. S., Kinzer, C. K., & Mulloy, A. M. (2012). The integrated curriculum project: Teacher change and student outcomes within a university–school professional development collaboration. Psychology in the Schools, 49(5), 444-464.

U.S. Department of Education. (2016). Future ready learning: Reimagining the role of technology in education. Retrieved from http://tech.ed.gov/files/2015/12/NETP16.pdf

U.S. Department of Education. (2017). Reimagining the role of technology in education: national education technology plan 2017 update. Retrieved from https://tech.ed.gov/files/2017/01/NETP17.pdf

Wells, J. G. (2007). Key design factors in durable instructional technology professional development. Journal of Technology and Teacher Education, 15(1), 101.

Winslow, J., Dickerson, J., Weaver, C., & Josey, F. (2016). Iterative and event-based frameworks for university and school district technology professional development partnerships. TechTrends, 60(1), 56-61.

Yıldırım, Ö., Kurşun, E., & Göktaş, Y. (2015). The factors affecting the quality of in-service training on information and communication technologies. Education & Science/Egitim ve Bilim, 40(178), 163-182.

Yu, C., & Prince, D. L. (2016). Aspiring school administrators’ perceived ability to meet technology standards and technological needs for professional development. Journal of Research on Technology in Education, 48(4), 239-257. doi: 10.1080/15391523.2016.1215168

Appendix
Interview Protocol

1. What is your current role?
2. What was your professional path that led you to this role?
3. Please describe a specific experience from the past 2 years where you designed and implemented a technology professional development activity. (Behavioral Event Interview Framework)
4. Thinking back to that experience, and other professional developments you have designed or attended, what do you view as essential components for beneficial technology professional development? Why are these components essential?
5. Do you think there are differences between what teachers value in technology professional development compared to what technology leaders value? What are those differences and why do you think they might exist?
6. Do you evaluate your PD after it has been conducted? What does this process look like?