CITE Journal - General

Kmitta, D. & Davis, J. (2004). Why PT3? An analysis of the impact of educational technology. Contemporary Issues in Technology and Teacher Education [Online serial], 4(3). Available: http://www.citejournal.org/vol4/iss3/general/article1.cfm

Abstract

This paper examines three questions regarding the use of computer technologies and education. The first question addresses the effects of computer technologies on student achievement, the second regards the effects of computer technologies on school climate, and the final question examines the cost efficacy of computer technologies in our nation’s schools. Using the most recent literature reviews, recent studies, and survey research that was not included in the most recent reviews, our synthesis of the data demonstrates an overall positive effect that computers have on student achievement and on the school environment. It also appears that using the latest computer technologies to keep the United States competitive in the global economy is cost effective.

Over the past 5 years an extraordinary amount of private and public sector funds have been spent on the equipment and training infrastructure for the K-12 teaching/learning environment. According to Johnson (2000), during fiscal years 1997 and 2000, the federal government had already spent $1.25 billion dollars on the Technology Literacy Challenge Fund. The Preparing Tomorrow’s Teachers to Use Technology (PT3) initiative had invested over $75 million alone in programming as of March 2001 (see http://www.aace.org/conf/site/pt3.htm). Future expenditures are projected to be just as great. Thus, it is no surprise that calls for accountability regarding the impact of these efforts upon student achievement continually echo throughout the country.

These calls focus on student achievement in content areas, issues related to the social impact of educational technology, and overall cost-effectiveness. Thus, the primary questions this review of the literature addresses are as follows:

The most important question seems to concern the effects of computer technologies upon student achievement. This question immediately creates a problem in that there is no consensus on what is meant by student achievement (see Kohn, 2001), which makes the determination of the effect of computer technologies upon this variable a tenuous connection at best. Generally, academic achievement is defined by student performance, as measured by standardized achievement tests. All of the reviews of the literature have focused upon research using standardized test scores, and only a few have focused upon other measures of student achievement.

The interaction between technologies and the social environment are as old as history itself. One cannot ignore the politics of this debate and, thus, the only intent of this research is to describe the phenomenon from reviewing the extant literature. For detailed critiques looking at computer technologies and educational reform movements see Cuban (1998) and Healy (1999).

How cost-effective are computer technologies in K-12 schools? Given the emphasis on accountability, the cost-efficacy of any social program has become a major issue. Cost effectiveness and cost efficacy are used synonymously throughout this paper. Again, there is reliance upon the literature to see if this question is even being addressed and, if so, how it is being covered.

There has been a considerable amount of research regarding the effects of computer technologies upon schooling, learning, and achievement. According to the Software Information Industry Association (SIIA; Sivin-Kachala, 2000), more than 3,500 studies exist examining the impact of computer technologies upon education and learning. Of course, not all of these studies were “scientific” but the volume of study does testify to the intense interest the topic has generated over the past two decades. (A redundant theme found in all the reviews of the research, as well as many of the recent studies, is the lack of sound “scientific” research. This is a common theme regarding all social science research. All social science research, no matter how well structured the study, has flaws. For an excellent discussion on why this is the case, see Campbell & Russo, 1999.)

This paper is organized into four sections. First is an examination of the most recent syntheses of the literature, 1996-2001, and an extraction of key findings from these reviews of the research. Next, is coverage on research that, as far as can be determined, is not included in any of the research syntheses covering the years 1994-2001. Finally, a report on survey research is included that indirectly examines the effects of computer technologies on student learning, school climate, and cost efficacy. The last section is a synthesis of the three data streams from the extent literature, followed by a discussion of the implications of the findings.

The primary authors of this research recruited a cadre of six graduate students, who helped to identify and retrieve articles for the study. It was decided that a multiple database search should be employed to find and secure articles. The search included the following databases: Educational Resources Information Center (ERIC), Google, Lexis-Nexus, PsychInfo, Government Documents, and Dissertation Abstracts. In addition, articles, reports, and conference presentations in the possession of the lead researchers were also included in the retrieval process. Search terms were standardized across databases, except when database parameters required alternative search terms. The basic terms included computer technology, achievement, school and classroom climate, cost, cost efficiency, and cost efficacy. In total over 400 “hits” were generated.

The first articles retrieved were the literature reviews (both meta-analytic and traditional). A master list of authors and articles was then created. Each database created a list, which was then cross-referenced against the master list references and separate research articles to remove duplicate studies. After this process was completed the remaining articles were retrieved for further study and review.

In order to summarize the extant data into manageable form a set of tables was created, based upon the research question asked, coupled with the literature that best informed the analysis. The categoration process was time consuming but did help reduce a large volume of information into the essential components for each article included in the review.

According to Vogt (1999), triangulation is, “Using more than one method to study the same thing” (p. 295). The synthesis employed in this study brings together three data streams from the extant literature to reach some general insights about computer technologies applied in K-12 educational settings. Past research reviews coupled to current research, both field based and survey, provide multiple sources of information to examine under the posed research questions. The approach to the literature synthesis relied upon identifying patterns that emerged from the extant literature. The synthesis of information, thus, follows the questions asked of the literature.

Much of the research has been reviewed in either a traditional or meta-analytic synthesis of the literature. Traditional literature reviews either “target” or “exhaust” all possible studies and articles related to a topic and include theoretical pieces, large and small empirical studies (both qualitative and quantitative), historical research, and literature covering policy and practice. Meta-analytic studies use only quantitative research, in which effect sizes can be computed and an overall effect size can be derived from the combined studies.

The current review falls more along the line of the traditional literature review. This review uses 11 recent literature reviews that have taken place since 1997 (see Table 1 for summary). Two of the studies were juried publications (Christmann & Badgett, 1999; Schacter & Fagnano, 1999). Three papers were presented at refereed conferences (Cavanaugh, 1999a, 1999b; Rampp & Guffey, 1998; Soe, Koki, & Chang, 2000). Three of the reviews were produced by the United States Government (Mann, 1999; McCombs, 2000; President’s Committee of Advisors on Education and Technology [PCAET]). Two reviews were supported by philanthropic foundations (Reeves, 1998; Schacter, 1999). The last review is a trade organization paper published by the Software Information Industry Association (Sivin-Kachala, 2000). The SIIA review is the most recent and focused on 311 studies covering all academic disciplines and levels. It appears to be the most comprehensive review of the literature to date. Unfortunately, it is a proprietary report whereby only the executive summary is publicly accessible. The details of the report are available but at a cost. As of this writing, the authors do not have access to the full report, and so we acknowledge that several of the studies included in our review already may have been included in this report.

These 11 reviews do not seem to overlap each other. They do draw upon similar studies, most notably Kulik’s (1996), Mann’s West Virginia study (Mann, 1999; Mann & Shafer, 1997), and Baker, Gearhart, and Herman’s (1993) Apple Classrooms of Tomorrow study, but do not cross-reference each other.

The overall conclusion of this synthesis of the reviews of the literature is that computer technologies generally have a positive effect on academic achievement. Within this finding there is great variance. On average, the strength of the correlation between computer technologies and student achievement varies from low to moderate. Most of the effect sizes range from .10 to .40. Rarely in the literature are there overtly strong relationships. Christmann and Badgett (1999) found a moderately strong effect size for the category General Science, .70, but also very small effects for chemistry, .08, and biology, .04. Cavanaugh (1999b), found that the academic discipline category “math” had an effect size of .76, and the category “other” had an effect size of .80. Cavanaugh also found a strong negative correlation with interactive computer technologies and foreign languages, one of the few negative findings in the literature. Ironically, Cavanaugh’s analysis found a small overall effect size for science, .07, compared to the overall effect size found by Christmann and Badgett of .26. This difference may well be the result of the types of computer technologies the researchers were examining. Cavanaugh was including only studies of interactive distance education, while the Christmann study included a variety of computer technologies. Cavanaugh’s overall achievement effect size was small, .147. Soe et al. (2000) found a small effect size of .132 for reading achievement and the use of computer technologies.

The remaining reviews did not compute statistical relationships between achievement and technology but rather deduced a positive technology achievement relationship from their reviews of the literature. An example of the types of deductions made can be found in documents like the Software Information Industry Association’s (Sivin-Kachala, 2000): “Technology can improve teaching and learning, but just having technology doesn’t automatically translate to better instructional outcomes.”

Schacter’s (1999) review of 13 meta-analyses and four large-scale studies concluded that “Computer Based Instruction (CBI), the most widely implemented and studied computer technology, moderately improves student learning” (p. 330).

Though all of the reviews of the literature indicated that the effects of computer technologies upon student achievement are generally positive, it is important to point out that none of the reviewers advocated computer technology as the solution to educational problems in the United States. In other words, computer technologies, in and of themselves, are not a panacea for improving student academic performance. Computer technologies have both positive and some negative correlations with student achievement. The research indicates that, generally, those using computer technologies have small but definite advantages over those who do not use computer technologies regarding overall academic achievement.

None of the studies ever detailed a discussion about the value of learning the technologies in and of themselves or of assessing the learning of the technology as its own discipline. Several of the reviews, McCombs (2000), Mann, (1997), and the 1997 President’s report, alluded to this technical knowledge as good, in and of itself, but ironically none of the studies provided any outcome documentation on this variable. A possible reason for this “hole” in the research is that computer technology is not itself a recognized academic discipline.

A consensus seems to be emerging that the relationship between computer technologies and student achievement are intricately tied to the social environment of the school and the larger community. Nine of the reviews examined the impact of computer technologies upon the social environment of the school and community. Two studies, Soe et al. (2000) and Cavanaugh (1999a), did not.

One of the emergent findings is that the type of learning theory used in conjunction with computer technologies has some bearing on the quality of instruction of K-12 students. Reeves (1998), Schacter (1999), Mann (1999), Sivin-Kachala (2000), the PCAET (1997), and Rampp and Guffy (1998) all reached similar conclusions. They found that traditional instruction coupled with the use of the computer as an enhancement, either as tutor, “drill and skill,” or practice, resulted in marginal improvement in student achievement levels as measured by standardized tests.

It appears that lower performing students derive benefit from this drill and skill approach but higher performing students do not. Higher academic achievement was observed in both lower and higher performing students in the few studies that used some form of either cognitive, developmental, or constructivist forms of pedagogical theory coupled with computer technologies. McCombs (2000), Schacter and Fagnano (1999), Christmann and Badgett (1999), Reeves (1998), and Rampp and Guffy (1998), cited studies that demonstrated how varying degrees of theory, constructivist, developmental, or cognitive based pedagogies (learning communities, collaborative learning, cooperative learning, multiple intelligences, and teaming) have had positive effects upon higher order learning skills. Unfortunately, none of the studies cited in the reviews used a common metric or standardized measure – probably because no measure really exists for such a pedagogy. (See Eva Baker’s, 1998, Understanding Educational Quality: Where Validity Meets Technology. In this essay Baker marshalled evidence to demonstrate that the traditional system of educational assessment is not highly valid in measuring the effects of technology upon learning, especially higher order learning.) Nonetheless, initial findings strongly suggested that more research needs to be conducted in this area.

Another emergent finding is that computer technologies have had a marked effect on the professional development of teachers and other educational leaders. Mann and Shafer (1997), National School Board Association (NSBA, 2000), Reeves (1998), Schacter and Fagnano (1999), and Christmann and Badgett (1999) all cited studies on professional development and computer technologies. The general findings were that computer applications must be relevant to the teacher, that the training must afford enough time for the educator to produce some mastery over the technology, and most important, that educators must have the technology to use in their own practice.

In addition, when teachers are presented with clearly structured lessons on how other teachers use technology in their classrooms and then are provided with opportunities to adapt and apply the technology to their own pedagogy, computer use in the classroom increases (see Schacter & Fagnano, 1999). McCombs (2000) took the idea of professional development and pedagogy one step further, arguing that, theoretically, incorporating constructivist models of learning into the professional development of computer competencies would be a good idea.

Another emergent theme on the impact of computer technologies upon student learning is that student attitudes toward learning and schooling improve. Schacter and Fagnano (1999), Sivin-Kachala (2001), and McCombs (2000) cited evidence of improved student attitudes toward schooling and learning.

Finally, Christmann and Badgett (1999), demonstrated that computer technologies seem to have a stronger impact upon science achievement scores of urban students, “e.s. .68,” than upon suburban, “e.s. .27,” or rural students, “e.s. .16” (p. 139).

Only seven reviews addressed issues concerning cost efficiency and/or accountability. None of the studies offered any firm empirical evidence or economic modeling derived from data to support their conclusions. Cavanaugh (1999a) theorized that interactive distance education will be more cost effective than traditional forms of distance and classroom education because the diversity of options improves, as well as the delivery of the content. Cavanaugh, however provided no empirical notion to support this contention.

Sivin-Kachala (2000), Mann and Shafer (1997), NSBA (2000), and Rampp and Guffy (1998) made similar arguments that cost efficacy ought not be formulated as simply time saved with a computer vs. time lost without a computer or as simple computer to student ratios. Given the dynamic nature of technology, budgeting for “technology” has to be flexible, based upon the needs of and training of the teachers and administrators, and ultimately, the goals and objectives set for the students.

McCombs (2000) argued for a value added model of student learning to improve cost efficacy. Unlike a traditional value added output model, McCombs envisioned a formula that would include the actual value of learning the computer skills in the achievement output equation. McCombs provided no empirical data to support this contention.

Finally, the PCAET (1997), advocated that 5% of the federal education budget be allocated toward technology and education, up from the 1.3% that was allocated in 1995. The PCAET projection cited the only empirically driven numbers and reasoned,

In addition, 10 recent studies were reviewed that did not appear in the previously described reviews. The reason for their exclusion may rest in the newness and availability of the reports. The Hawkins, Spielvogel, and Panush, (1996) and Johnson, Cox, and Watson (1994) studies are interesting exclusions. The Hawkins et al. study was a qualitative multiple case study conducted by the Center for Children and Technology. Although the data are not quantitative, the study does meet the criteria for rigorous qualitative inquiry and, thus, is included in our review of the literature. The Johnson et al. study is a multiple case study but employs quantitative and qualitative method and was conducted in England. Although there are many differences in the way schools operate in the US and in England, we are including the Johnson et al. study simply to compare if computer technologies are affecting academic achievement in other parts of the world. Only those studies that, at a minimum, covered multiple locations were included. Excluded were those studies that were limited to one or two classrooms or schools.

Finally, one study, Wenglinsky (1998), is included in this analysis even though his study was used in the Schacter review of the literature. We include this study, in addition, because much of the social effects reported in the individual study were not reported in Schacter’s original review.

Again, the need to address the issue of methodological purity of these selected studies should be readily apparent. All of the studies have methodological flaws that, depending upon one’s politics and dogmatic adherence to scientific method, will either be dismissed or revered. This review does not scrutinize the methodology of these studies, rather the review is only reporting upon the observed effects and assumes that the reader is fully aware of the problems that beset field-based social science research.

Generally, the recent independent studies’ results support the findings of the reviews of the literature. As is demonstrated in Table 2, nine of the studies reported positive or mostly positive results on both academic achievement and social effects: Chang, Honey, Light, Moeller, and Ross (1998), Hawkins et al. (1996), Spielvogel et al. (2001), Mann and Shafer (1997), Kusimo, Carter, and Keyes, (1998), Wenglinsky (1998), Yekovich, Yekovich, and Nagy-Rado (1999), Pisapia, Knutson, and Coukos (1999), and Johnson et al. (1994). Barron, Hogarty, Kromrey, and Lenkway (1999) theorized that academic performance would improve as a result of student computer use because of improved student behavior associated with students’ use of the computer. However, no empirical link was established in their study.

Chang et al. (1998) examined the effects of computer technologies in the Union City, NJ, school district during an approximately 10-year period. Their major findings were that first- and fourth-grade CAT scores increased in reading (45%/25%), mathematics (18%/15%), and writing (34%/14%) during the 1989-1997 time span.

In addition, eighth-grade Early Warning Test (EWT) scores improved in reading by 53%, in math by 30%, and in writing by 40% over a 5-year period from 1992-1997. Ninth-grade EWT scores also improved by 10% in reading, 4% in mathematics, and 7% in writing over a 3-year period. These scores were compared to a control group of students who had less technology available to them. The researchers found that there were eighth-grade treatment control differences where the passing rates for treatment were EWT +3% (reading), +27% (mathematics) and +32% (writing). This analysis covered a 3-year time period from 1994-1997.

Finally, the researchers also found overall positive results with 10th-grade students on the High School Proficiency Test (HSPT). There was an 11% increase in reading and a 14% increase in writing. There was, however, a 4% decrease in mathematics scores. This analysis covered a 2-year period from 1995-1997.

Hawkins et al. (1996) focused on qualitative aspects of 12 school districts using computer technologies. Their basic findings included the following:

The Spielvogel et al. (2001), study was an examination of nine locations using primarily IBM hardware and software. This study was about educational reform that uses computer technologies. It employed both qualitative and quantitative methods and found,

Although improvements were found across location in both instruction and academic achievement, the authors did note, "The solutions go beyond technology – they address process and change on an organizational level within districts and states that is a fundamental component of a reform effort, given the scope and time frame over which the changes occur" (p. 3).

Johnson et al. (1994) found that high levels of instructional technology generally favored improved academic achievement across four content areas, mathematics, science, geography, and English. A national research project that used multiple quasi-experimental design in 24 locations and then computed an overall effect size from the 24 separate studies found that there were small but statistically significant academic achievement effect sizes for those in the high computer use conditions. These effects ranged from a low of .08 (English) to a high of .31 (mathematics). The two other disciplines, geography and science, both had small but significant effect sizes of .25 and .21, respectively.

Pisapia et al. (1999) studied the effects of computer technologies upon a large metropolitan school district of some 44,000 students. The report covered only third and sixth grades and used two sets of standardized test scores, the Literacy Testing Program and the Cognitive Abilities Test. A cohort design was used, examining before and after affects of computer technologies on grades 3 and 6. ANCOVA results demonstrated significant differences in verbal, quantitative, nonverbal and reading, mathematics, and writing scores of the cohort group exposed to the computer technologies.

Yekovich et al. (1999) examined the Technology-Rich Authentic Learning Environments (TRALE) project in 20 of the District of Columbia's poorest schools. The study found that TRALE students saw grade equivalent score increases on the California Test of Basic Skills (CTBS) in reading, .83, as compared to a control group of .24, language, .7/.07, and math, 1.78/1.30. It was also found that teachers who had high rates of technological implementation had average Normal Curve Equivalent (NCE) point gains of 10.9 on the CTBS, while teachers who had low rates of technological implementation had only a 1.1 NCE point gain. Teachers who did not implement any computer technologies had a 2.7 NCE gain.

Two studies reported decidedly mixed results, Johnson (2000), and Wenglinsky (1998). These two studies are similar in that they both used the National Assessment of Educational Progress (NAEP) data sets but they differ in content area and in type of analyses. By far, Wenglinsky's study is the most comprehensive and sophisticated to date. The research employed associational methodology and used Structured Equation Modeling statistics, specifically, Pathways Analysis, to analyze a variety of variables and their relative influence upon mathematics achievement for the fourth- and eighth-grade NAEP scores. Johnson (2000) was an associational study but used multiple regression analyses upon a variety of variables for the fourth- and eighth-grade NAEP reading scores.

Wenglinsky's research on the impact of computer technologies upon mathematic achievement revealed several key findings. First, it was found that both fourth- and eighth-grade rural and urban students of lower socioeconomic status (SES) had fewer teachers with recent professional development in computers and mathematics education and had less access to home computers than did suburban students. Next, both fourth- and eighth-grade black students had less access to a home computer than did white students. Eighth-grade black students were more likely to use the computer for drill and skill activities and less likely to use the computer for more higher order learning.

For eighth graders it was also found that the professional development of teachers in computer technologies was positively related to both academic achievement and to the improvement of the social environment of the school. Results indicated that eighth graders who had frequent use of a home computer also had both positive effects on academic achievement and on improving the school’s social environment. Two negative findings for the eighth grade were that the use of computers to teach lower ordered thinking skills was negatively related to both achievement and environment, as was the frequency of school computer use.

Kirk Johnson's study (2000) on the effects of computer technologies upon NAEP reading scores used a multiple regression to analyze the effects of computer technologies and other variables, such as familial income upon student achievement. Johnson's study used a system of statistical controls that differed from Wenglinsky's research. Wenglinsky controlled for confounding effects by isolating the covariation of variables upon student mathematics achievement and hypothesized the relative weight of each covariate. His model supported his hypothesis. Johnson sought to control for confounding variables in a rather arbitrary manner, testing the effects singularly and then aggregate effects of the confounding variables upon the outcome reading variable. He did so with no a priori hypothesis of the order of each independent variable or any indication of the manner in which he proceeded with the multiple regression – stepwise, forward, or backward.

Only one study, St. Clair (1999) reported decidedly negative results on academic achievement and the use of computer technologies. This study focused on the effectiveness of using ACHIP technology in fourth and seventh grade classrooms. The ACHIP is a small hand-held device that allows students to access information about their subject matter.

A total of 655 students and 22 teachers from six participating sites were included in St. Clairs’ study. A summative and a formative assessment were carried out. The summative portion was designed to determine if students in the treatment group would score higher on the Michigan Educational Assessment Program (MEAP) and Metropolitan Achievement Test, 7th Edition (MAT7), than would students in the control group. Frequency distributions were calculated for school-wide scores on the MEAP and MAT 7 tests to determine whether students had accelerated, moderate, or low scores. The MEAP results of students from the project were then compared to the school results.

For the second testing objective, MAT 7 scores were obtained from 1997 and 1998 for those students who were participating in the ACHIP Project. An ad hoc control group of 432 students who matched the 1997 MAT 7 scores was obtained from the schools. Normal curve equivalent (NCE) gains for both control and treatment groups were calculated.

Analysis was based on a frequency distribution that revealed the number and percentage of students achieving a gain of at least three NCE units on the MAT 7, as well as the number and percentage of students achieving a net gain of greater than 0 but less than 3 NCE units. Independent t-tests were used to determine if significant gains existed between control and treatment groups.

A formative evaluation designed to measure teachers’ perceptions of the ACHIP program was also applied to this study. Sixteen of 22 teachers participating in the project completed the surveys. The survey was an almalgom of 56 questions covering a wide range of programmatic issues for the ACHIP program. There was no indication, though, that the survey was ever tied to outcome measures for the program.

Results of the summative evaluation revealed that treatment students in the Grade 4 MEAP reading and mathematics tests scored levels comparable to control group results in satisfactory, moderate, and low levels. Grade 7 MEAP results were also comparable to school results in all levels.

The second testing objective stated that students would demonstrate NCE gains on the MAT 7 that exceed those of a post hoc matched control group. Results revealed that students participating in the ACHIP Project did not demonstrate NCE gains exceeding those of the control group. On the mathematics portion of the MAT 7 mathematic and reading sections, students did not demonstrate NCE gains that exceeded that of the control group. On the science portion of the MAT 7, students lost significantly more NCE units than did the control group. Analysis of data from the 1997 and 1998 MAT 7 test scores for all three content areas revealed that students participating in the ACHIP Project lost NCE units.

Overall, these studies demonstrated support for the findings of the literature reviews. It appears that for this set of studies, most academic gains were small to modest, and in some cases, either nonexistent or, in the case of the ACHIP study, negative. The next section turns to the question of the effect of computer technologies on the social environment.

Nine of the studies reported generally positive results on the social effects of computers in the classroom. Seven of the studies — Hawkins et al. (1996), Spielvogel et al. (2001), Mann and Shafer (1997), Kusimo et al. (1998), Wenglinsky (1998), Yekovich et al. (1999), Pisapia et al. (1999), and Johnson et al. (1994) — all reported upon the importance of professional development for teachers and administrators regarding computer skills. Each study reported that the teachers’ competency in using computer technology not only influenced how often the teachers used computer technology, but also how teachers used computer technology. Kirk Johnson's (2000) research provides some evidence that there may be no effect of professional development on student achievement, at least as far as measured by NAEP reading scores and framed in limited use of computer technologies.

Four of the studies address the importance of teacher philosophy regarding using computers. Johnson et al. (1994), Pisapia et al. (1999), and Yekovich et al. (1999) all reported that teachers with more constructivist or student-centered approaches (which emphasize community or collaboration while teaching) utilized the computer more often, as well as in more varied ways, than did teachers with more traditional pedagogies. Philosophy is an important component in the frequency of computer use, but how one uses the computer is more important. Apparently, constructivist teachers tend to use the computer for more higher order learning than do traditional teachers.

One study of interest, Barron et al. (1999), reported on the effects of the computer in improving student behavior. Although only relational data was used and no causal connection was established, the researchers did observe that a relationship existed between the use of computers and improved student behaviors. The conclusion reached by this team of researchers was that in all grade levels that were related to computers in the classrooms, there were small (-.09) to moderate (-.35) effect sizes for reductions in conduct violations and disciplinary actions and improved attendance.

Six of the studies addressed some facet regarding costs and efficiency: Hawkins et al. (1996), Spielvogel et al. (2001), Mann and Shafer (1997), Wenglinsky (1998), Johnson (2000), and St. Clair (1999). All but Johnson and St. Clair advocated for more monies for computer technologies.

Hawkins et al. (1996) called for investments in all grades but said that lower expenditures should be allocated to lower grades and higher amounts for the upper grades. They also argued for coordinated school budgets throughout the district and the state. In other words, budgets need to coordinate improvements in technology and training with changes in the curriculum. This strategy would better align technology and curriculum, thereby reducing after-the-fact spending. Hawkins et al. (1996) also argued that many financing options are needed, including corporate, nonprofit agency, and low interest government loans. More grant monies also need to be available to supplement loans and traditional revenue sources.

St. Clair (1999) argued that in the case of the ACHIP program that the ACHIP hardware was not cost effective and that the manufacturer bore responsibility for fixing the problems with the hardware before the ACHIP units are used again. This is the first study that actually holds a corporation accountable for student achievement.

Spielvogel et al. (2001) argued that there is a need for more technology expenditures in education, especially in teacher and school leadership development. These expenditures focused not only on training for technological skills but on collaborative leadership and more constructivist ways of teaching.

Mann and Shafer (1997) essentially argued that schools spend less than industry spends on technology training. These authors’ logic is that if industry expects students to be prepared with high tech skills, then school spending must at least equal industry spending in order to keep pace with current market needs. Mann's research also demonstrated that teachers would be willing to forgo a raise if they could get more technology and that greater savings could be realized by increasing class size with more technology to support the increases.

Wenglinsky (1998) recommended that there should be a targeting and prioritization first by doubling professional development in the use of computer technologies. Second, he wrote, there is a need to support the teaching and development of higher order thinking skills both technologically and traditionally. Finally, he suggested that middle schools rather than elementary schools be targeted for the implementation of more technology and the use of technologies in instruction.

Chang et al. (1998) called for increases in the budget for computer technology tied to increases in student performances. This position was in keeping with the trend toward accountability of educators to parents, which has been a prevalent movement of the 1990s.

Johnson (2000) was the only author who directly argued that the cost of computer technologies are not producing superior learning results. Therefore, he asserted, schools should curtail spending on computer technologies until more dramatic results can be demonstrated.

This review also summarized the results of three national surveys examining the use of computers in schools and homes and the applications of school and home computer use to student learning (see Table 3 for summary). These surveys, the oldest of which was published in 1997, provided both context and insight into computer technology learning outcomes. In addition, all of these surveys examined self-reports of educators and students regarding the effects of computer technologies on academic achievement.

Becker (2001) used the Teaching, Learning, and Computing Survey, a survey instrument that had undergone some attempts to validate its accuracy. This survey of over 4,000 teachers, grades 4-12, used explicit sampling protocols to try to establish as representative a sample of teachers as possible. Unfortunately, a margin of sampling error was not provided. The major findings from this study included the following:

Although computers in schools by now number over 10 million, frequent student experiences with school computers occur primarily in four contexts—separate courses in computer education, pre-occupational preparation in business and vocational education, various exploratory uses in elementary school classes, and the use of word processing software for students to present work to their teachers. The one area where one might imagine learning to be most impacted by technology students acquiring information, analyzing ideas, and demonstrating and communicating content understanding in secondary school science, social studies, mathematics, and other academic work involves computers significantly in only a small minority of secondary school academic classes. (p. 3)

Thus, according to Becker, the classes most frequently using the computer were computer classes, followed by business and vocational education. Among academic subjects, English teachers used the computer more often than did any other discipline. Only one out of six science teachers, one out of eight social studies teachers, and one out of nine math teachers used the computer at least once a week in their classroom instruction.

There are three important determinants on how frequently teachers used computers in classroom instruction. First, was classroom access; that is, how many computers per student a teacher had in his or her own classroom. Next, was the school class schedule. Block scheduling was more conducive to frequent use than were 50 minute classes. Finally was the experience of teachers using the computers. Although not universal, it was generally true that a teachers' own level of computer competency was associated with how frequent and how well the computer was used in instruction.

Professionally engaged teachers, in other words, teachers who were committed to the profession by keeping an open door policy, were more likely to use the computer in their classroom instruction than were private, or closed door, school teachers. Not only that, professionally engaged teachers used more software and conducted more of a variety of computer activities than did their closed door colleagues.

Students in the lowest SES quartile used the computer more frequently than did students in any other SES band. Unfortunately, most of the time was spent on drill and skill practice programs and not on higher order thinking tasks and activities. There were two distinct tracts in the use of computers. High SES students tended to use the computer for more higher order learning activities while low SES used computers for drill and skill.

In other words, teachers with constructivist backgrounds used the computer in more higher order ways of learning than did the traditional transmission-oriented teacher. In addition, Becker also noted,

Finally, Becker conducted a multiple regression of several variables extracted from the survey and found the best predictors for frequency of use in instruction were teacher expertise, type of software, and professional engagement.

Nie and Erbring (2000) did not review school computer use but rather viewed general Internet use. Their principal findings, which may have some implications for computer use in the classroom, were that the major inhibiting demographic variables are age and education. According to Nie and Erbring (2000), race, gender, and SES, had less influence upon using the Internet than did age and education. It appeared that once connected, people stayed connected, but undereducated and older people generally lacked the access and the motivation to access the Web. One negative finding was that too much time spent on the Internet was related to social isolation. Another negative finding was that when people used the Internet they spent more time at home working. This may eventually translate to less student attendance in school.

Smerdon et al. (2000) were the researchers involved in the National Center for Educational Statistics survey of teachers’ use of technology. Findings from their study revealed that teachers with computer availability generally used the technology for preparation and administrative tasks rather than for tasks such as accessing research, modeling teaching lessons, or contacting experts, parents, or students. In addition instructional methods were mostly comprised of assigning students to work on word processors and spreadsheets, practice drills, Internet research, and a mix of problem solving and data analysis skills. One interesting observation made by Smerdon et al. (2000) was that elementary and secondary teachers differed on the use of computers.

Also of note in their report was that teachers from poor and diverse schools were less likely to use computer technologies than were their counterparts in middle to upper class and more homogeneous schools. The differences covered "a wide range of activities, including gathering information at school, creating instructional materials at school, communicating with colleagues at school, and instructing students" (p. 9).

Finally, it was found that less experienced teachers were more likely to use computer technologies than were teachers with high levels of experience. The computer activities that the average new teachers used included accessing research and best practices, as well as retrieving model lesson plans.

Some common themes seem to be emerging from the survey data. First, almost all teachers, in every type of school, have some access to a computer. It appears that there are some discrepancies based upon school location. That is, poorer districts tend to have less and poorer quality access, and richer schools have greater and higher quality access. Next, it appears that teachers who do have classroom Internet access use it during classroom instruction to either gather data or access information or communicate with students, parents, and colleagues. It also appears that teachers are using computers more now than ever to help with administrative tasks and the details of class preparation. Finally, younger teachers, teachers with professional commitment who are more inclined to use constructivist approaches to teaching, are also more likely to use computers in classroom instruction.

As mentioned earlier this synthesis relied upon triangulation of multiple data streams found in the extent literature, ever watching for emergent themes and patterns as they appeared during and after the review process. The synthesis of information thus follows the questions asked of the literature.

Does the use of computer technologies have an impact upon K-12 student achievement?

It is abundantly clear that the majority of studies conducted to date favor an affirmative answer to this question. It appears that there are, generally, small to moderate gains in overall student achievement when computer technologies are used. At worst, there is no loss in achievement, unless of course, the computer product being used is of inferior quality or the use of the computer becomes nothing more than a redundant practice and drill mechanism, ultimately conveying lower expectations to already disadvantaged students.

It is also clear that the impact of computer technology upon academic achievement is a complex relationship and that there are a myriad of variables that can impinge upon the success or failure of technological applications in the classroom. Such factors as teaching philosophy, professional engagement, and content and developmental appropriateness are but a few of the emergent variables that will provide more depth to our understanding of the effects of computer technologies upon student achievement.

However, there are some indicative patterns emerging on these factors from the literature. It appears that teaching philosophy has some bearing on how the computer is used, as well as how often the computer is used in the classroom. Teachers embracing constructivist pedagogies tend to use the computer for teaching more higher order thinking skills that are linked to curriculum standards and objectives. This tends to make the computer a tool, amongst other tools, that students use to complete higher order tasks. Teachers holding to the traditional transmittal models tend to use the computer in remedial or tutorial forms of learning. This relegates the computer use to a one-dimensional forum that resembles the earlier behavioral learning machines of years past.

Another pattern forming in the literature is the need to align academic content with the use of the computer. Perhaps one of the reasons why computers are used less frequently and effectively with math, science, and foreign languages is that the curriculum is not aligned with software and teaching strategies using computers that could promote higher levels of learning. It may also be that computers are more appropriate for some content areas than others.

Age-appropriate use of the computer also is emerging as a strong pattern in the literature. Exposing young children to the computer, as opposed to relying upon the computer to teach young children, may be the most age-appropriate strategy available to date. The grade levels that computer use seems to affect most positively are late elementary through high school.

Finally, although it was never mentioned in any of the research reviewed, the very fact that students are learning the technology itself is an important achievement that is apparently never measured. One wonders how students in highly computerized nations would compare on basic computer knowledge. Unfortunately, that achievement assessment is yet to be deployed in either domestic or international tests of educational progress.

The implications from these findings are, of course, subject to interpretation. First, since the evidence suggests that computer technologies generally improve student achievement overall, and no baleful results were found, there should be more computer use by students regardless of social class or geographic location. What is clear from the literature is that division between the “haves and have nots” will continue to grow, leaving one group of students severely underprepared for life after school. In addition to this finding, further research ought to be conducted on learning the technology itself as an academic achievement variable.

Next, the actual pedagogy used to deliver technologically enhanced content needs to be further investigated. If constructivist approaches are more suitable to integrating computers into the classroom than are pedantic approaches, then school administrators might wish to make computer technology appropriations to those who embrace constructivist methods of teaching.

Tied to the above implication is also the age and experience of the teachers. Professional development in computer technology applications may have to be tailored to the age and experience levels of any given school district.

Finally, continuing research regarding the age appropriateness of computer technologies ought to continue. The implications for this future research would not only cover the developmental issues around intellectual achievement but also could be a source for budget allocations and priorities.

Does the use of computer technologies have a social impact upon the school environment?

Computer technologies do have social impact on the school environment. Although far from a causal event, computer use has spurred more constructivist approaches to flourish in schools. It also appears that the use of computers helps to improve overall student motivation to learn and stay and behave better in school. The aggregate evidence also suggests that students are happier while in school doing meaningful work with a computer.

Computer technologies are changing the face of professional development in schools. There is a constant need to update teachers on the latest technologies, as well as the latest uses of computers in content areas. Rather than receiving "one shot" trainings, professional development in computer technologies is incremental and ongoing. In addition, with the need to improve content area applications, it is conceivable that professional development in computer technologies will also become more specialized in the disciplines.

It is clear that students emerging from the public school sector are much better equipped to handle computer technologies than were previous generations. It is also apparent that some students, themselves, are creating and evolving learning strategies with the new technologies.

Finally, there is ample evidence to suggest that computer technologies are improving communications amongst parents, teachers, students, and administrators. At least all of these parties are, in part, more accessible to each other because of e-mail, chat, and discussion groups. Even better, communications can cut across long distances at a fraction of the cost of long distance telephone or any other form of long distance communication.

One clear implication from the literature is that computer technologies can be used as a valuable tool in creating safe school environments. For example, opening lines of communications between teachers, students, parents, administrators, and other agencies involved in education could facilitate better planning and action plans for immediate interventions for individual students, as well as for the community as a whole.

If one were to measure just the small to modest improvements in overall academic achievement, there may be reason to re-evaluate the relative efficacy of computer use upon this variable. After all, billions of dollars have been spent on computer hardware, software, and training over the past two decades, and the best that can be shown are consistently small to moderate gains in effect sizes when comparing computer users to noncomputer users. This argument is exactly what conservatives like Kirk Johnson are advocating with their research.

But those who are looking at cost efficacy down this narrow lens have to remember that consistent gains are being made. Those who are familiar with educational research rarely have seen anywhere in the literature any gains at all as the result of a new technology or curriculum. Therefore, the call for even more expenditures, which seems to be the resounding theme, has more merit than the calls to keep computer technology costs in check.

One implication that can be derived from the extant literature is that the learning of the computer technologies, in and of itself, is a value added skill not currently being incorporated into the achievement and cost effectiveness models. Who knows how much more value is added to students’ education when such students are equipped with additional computer technology skills. It could be that incorporating academic standards of computer literacy into the core curriculum would be an advantageous way of furthering the goal of leaving no child behind.

Students’ gain more from using computers than merely increasing their academic achievement. The positive impact that computers are having socially is also a valuable addition to the overall education of our students, and some would argue, to society in general. The social value added also must be figured into the equation.

Baker, E.L. (1998, November). Understanding educational quality: Where validity meets technology. Paper presented at the 5^th annual William H. Angoff Lecture, Educational Testing Services, Princeton, NJ.

Baker, E.L., Gearhart, M., & Herman, J.L. (1993). The Apple Classrooms of Tomorrow: The UCLA evaluation studies. Cupertino, CA: Apple Computer, Inc. (ERIC Document Reproduction Service No. ED378219)

Barron, A.E., Hogarty, K.Y., Kromrey, J.D., & Lenkway, P. (1999). An examination of the relationships between student conduct and the number of computers per student in Florida schools. Journal of Research on Computing in Education, 32(1), 98-107.

Becker, H.J. (2001, April). How are teachers using computers in instruction. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA.

Campbell, D.T., & Russo, M.J. (1999). Social experimentation. Thousand Oaks, CA: Sage.

Cavanaugh, C.S. (1999a). The effectiveness of interactive distance learning technologies on K-12 academic achievement. Unpublished manuscript. (ERIC Document Reproduction Service No. ED428752)

Cavanaugh, C.S. (1999b). The effectiveness of interactive distance education technologies in K-12 learning: A meta-analysis. Unpublished manuscript. (ERIC Document Reproduction Service No. ED430547)

Chang, H., Honey, M., Light, D., Moeller, B., & Ross, N. (1998). The Union City story: Education reform and technology students’ performance on standardized tests. New York: Education Development Center. (ERIC Document Reproduction Service No. ED424108)

Christmann, E., & Badget, J. (1999). A comparative analysis of the effects of computer-assisted instruction on student achievement in differing science and demographical areas. Journal of Computers in Mathematics and Science Teacher, 18(2), 135-143.

Cuban, L. (1998). High-tech schools and low-tech teaching: A commentary. Journal of Computing in Teacher Education, 14(2), 6-7.

Hawkins, J., Spielvogel, R., & Panush, E.M. (1996). National study tour of district technology integration summary report. Center for Children & Technology. Washington, DC: Education Development Center, Inc.

Healy, J.M. (1999, April). The mad dash to compute. In J.W. Noll (Ed.), Taking sides: Clashing views on controversial educational issues (12th ed). Guilford, CT: McGraw Hill.

Johnson, D.C., Cox, M.J., & Watson, D.M. (1994). Evaluating the impact of IT on pupils achievements. Journal of Computer Assisted Learning, 10, 138-156.

Kohn, A. (2001). The case against standardized testing: Raising the scores, ruining the schools. Portsmouth, NH: Heinemann.

Kulik, J.A. (1996). Meta-analytic studies of findings on computer-based instruction. Meta-Analytic Studies, 1, 9-33.

Kusimo, P., Carter, C., & Keyes, M. (1998, April). Internet: Act 1 – scenes from Tennessee classrooms. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. (ERIC Document Reproduction Service No. ED428756)

Mann, D., & Shafer, E. A. (1997, July). Technology and achievement. The American School Board Journal, 22-23.

McCombs, B., L. (2000). Assessing the role of educational technology in the teaching and learning process: a learner-centered perspective. The Secretary’s Conference on Educational Technology 2000. Retrieved March 11, 2001, from http://www.edc.org/CCT/IBM/evaluation.doc

Nie, N.H., & Erbring, L. (2000). Internet and society: A preliminary report. Standford, CA: Standford University, Standford Institute for the Quantitative Study of Society.

Pisapia, J.R., Knutson, K., & Coukos, E.D. (1999, November). The impact of computers on student performance and teacher behavior. Paper presented at the 44th annual meeting of the Florida Educational Research Association (ERIC Document Reproduction Service No. ED438323)

President’s Committee of Advisors on Education and Technology. (1997). Report to the President on the use of technology to strengthen K-12 education in the United States. Washington, DC. Author.

Rampp, L.C., & Guffey, J.S. (1998, November). Technology effect: The promise of enhanced academic achievement. Paper presented at the annual Mid-South Educational Research Association Conference, New Orleans, LA. (ERIC Document Reproduction Service No. ED427771)

Reeves, T.C. (1998). The impact of media and technology in schools. A research report prepared for the Bertelsmann Foundation. Retrieved March 19, 2001, from http://www.athensacademy.com/instruct/media_tech/reeves0.html

Schacter, J. (1999). The impact of education technology on student achievement: What the most current research has to say. Santa Monica, CA: Milken Exchange on Education Technology. (ERIC Document Reproduction Service No. ED430537)

Schacter, J., & Fagnano, C. (1999). Does technology improve student learning and achievement? How, when, and under what conditions. Journal of Educational Computing Research, 20(4), 329-343.

Sivin-Kachala, R. (2000). 2000 research report on the effectiveness of technology in schools: Executive summary. Retrieved March 3, 2001, from http://www.siia.net/store/default.asp

Smerden, B., Cronen, S., Lanahan, L., Andersen, J., Iannotti, N., & Angeles, J. (2000). Teachers’ tools for the 21st century: A report on teachers’ use of technology. (National Center for Education Statistics No. 2000-102). Washington, DC: Office of Educational Research and Improvement, U.S. Department of Education.

Soe, K., Koki, S., & Chang, J.M. (2000). Effect of computer-assisted instruction on reading achievement: A meta-analysis. Washington, DC: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED443079)

Spielvogel, R., Brunner, C., Pasnik, S., Keane, J.T., Friedman, W., Jeffers, L., John, K., & Hermos, J. (2001). IBM’s Reinventing Education Grant Partnership Initiative. Retrieved April 3, 2001, from www.edc.org/CCT/IBM/evaluation.doc

St. Clair, S.Y. (1999). Evaluation of the ACHIP pilot project, Detroit Public Schools, 1997-1998. Detroit, MI: Detroit Public Schools, Office of Research, Evaluation, and Assessment. (ERIC Document Reproduction Service No. ED427133)

Vogt , W.P. (1999). Dictionary of statistics and methodology (2nd ed). Thousand Oaks, CA:Sage Publishing.

Wenglinsky, P. (1998). Does it compute –relationships between educational technology and student acceleration in math. Princeton, NJ: Educational Testing Service. (ERIC Document Reproduction Service No. ED425191)

Yekovich, F.R. Yekovich, C.W., & Nagy-Rado, A. (1999). A formative evaluation of the TRALE project. Philadelphia: Mid-Atlantic Region Laboratory for Student Success. (ERIC Document Reproduction Service No. ED429123)