Introduction
This study examines the implementation and impact of a home computer initiative for 5th grade students. The program, conducted with funding from the U.S. Department of Education grant, was designed to build upon similar efforts already in place in several Pennsylvania school districts. By presenting information about the study’s methodology, this report is also intended to inform the efforts of other states that are interested in using an experimental design to examine the impact of an educational technology initiative.
A growing body of research pointing to a positive relationship between computer use and student engagement and achievement has made the continuing digital divide a pressing concern among educators and policymakers. As a result, home computer programs intended to redress gaps in access and level the educational playing field have become more common in school districts around the nation.
In spite of the proliferation of these initiatives, few empirical studies have employed rigorous experimental designs that allow for an examination of the impact of providing students with home access to computers, and the results of existing studies that employ non-experimental designs are viewed by many as inconclusive. This report presents findings from an evaluation of a home computer intervention that used a rigorous experimental design to assess the intervention’s impact on a range of student and parent outcomes.
The evaluation of the Student and Parent Access to Recycled Computers (SPARC) program was designed to assess the impact of providing refurbished computers and dial-up Internet access to the families of 5th grade students in four urban communities in eastern Pennsylvania. The 3-year study randomly assigned 355 5th grade students across 22 schools to either a treatment or control group (all of these students resided in households that reported not having a working home computer in the months prior to the intervention). During the 2004–05 school year, students randomly assigned to the treatment group received a refurbished Dell Pentium II/III computer, a printer, a pair of speakers, free dial-up Internet access through America Online, access to toll-free technical assistance, access to a website that provided links to educational sites, and the opportunity to participate in monthly training sessions about how to make use of learning technologies. In order to provide parity, students assigned to the control group received these same benefits at the end of the 2004–05 school year.
Research Questions
A primary objective of the study was to examine whether the provision of home computers and Internet access (1) increases the frequency with which students use these tools for academic, informational, and recreational purposes, (2) improves students’ skills at using these tools, (3) increases students’ interest in school and specific academic subjects, (4) improves student performance and achievement, (5) increases parents’ frequency of use and technical proficiency, and (6) increases parental involvement in their children’s education. The study also explored whether there are conditions under which the provision of home computers is more or less likely to improve student achievement and/or increase parental involvement in their child’s education. A secondary study objective was to document the practical issues that need to be considered when using refurbished computers to bridge the digital divide. An additional, but equally important objective was to assess the feasibility of using experimental designs to study the impact of educational interventions and develop data collection and evaluation tools for use by state and local school systems.
Methodology
The use of random assignment provided a robust framework for attributing student and parent outcomes to the effects of the SPARC intervention. Within-classroom random assignment was used to ensure an equal distribution of treatment and control students in each class, thereby allowing us to control for possible confounding variables associated with the instructional practices of individual teachers. Random assignment occurred in September 2004, with 178 students assigned to the treatment group and 176 assigned to the control group. Throughout the 2004–05 school year, Westat employed a wide range of data collection activities to address the study’s research questions. These included pre- and post-intervention student and parent surveys, a teacher log, a teacher survey, case studies with students in one study school, and focus groups with a sample of treatment group parents. In addition, we obtained grades and standardized test assessment scores in mathematics and reading for individual study participants.
The recruitment process for the SPARC study was designed to identify schools with a high percentage of students who lacked access to a working home computer. It is therefore not surprising that the 22 elementary schools that elected to participate in the study were located in neighborhoods characterized by high rates of poverty and unemployment. All of the participating elementary schools were equipped with computers and Internet access. Nonetheless, it appears that the extent to which teachers made use of computers and the Internet with their 5th grade students was limited. Even more notable for the SPARC intervention, most of the teachers with students in the study were reluctant to require computer or Internet use for homework assignments because they believed that many of their students had limited access to computers outside of school.
As such, it is reasonable to characterize the SPARC initiative as a “passive” intervention—that is, treatment students took possession of their home computers without any requirement that they be used for school-related or academic purposes. On the one hand, the use of a passive intervention was not accidental in that SPARC was designed to assess the impact of a low-cost home-based technology initiative that could be replicated in other high-need communities. On the other hand, the minimal use of learning technologies in the classrooms, as well as the lack of any homework assignments that encouraged or required computer and Internet use, may have decreased the likelihood that treatment students would make use of their SPARC computers for academic purposes.
Results
- eSPARC students are more likely to use computer for school-related purposes
- eSPARC parents reported stronger computer skills than their control counterparts.
Intermediate Student Outcomes
Treatment students were more likely than their control counterparts to report using computers for recreational and school-related purposes. They were also more likely to report stronger computer skills than control students. Interestingly, control students exhibited slightly more positive attitudes about computers than treatment students, although it should be noted that nearly all students held very positive views about computers. The SPARC intervention did not, however, affect students’ overall interest or participation in their schoolwork, as reported by both study students and their 5th grade teachers.ii
Student Achievement
There were no significant differences between treatment and control students in the rescaled grades for any of the four core subject areas. Comparison of overall PSSA scale scores revealed no difference between treatment and control students for both reading and mathematics. In addition, there were no treatment subgroups that scored higher or lower on the PSSA as a result of their participation in SPARC. These findings suggest that exposure to the SPARC intervention had no impact on treatment students’ grades or performance on the PSSA.
Although the ultimate goal of SPARC was to improve student academic performance, the program was also designed to affect a wide range of intermediate student outcomes, including increased computer use, enhanced computer skills, and increased engagement in school and learning. Indeed, it was believed that the attainment of these intermediate outcomes might ultimately enhance students’ academic performance.
There are several potential explanations for the lack of impact on student achievement. First, and perhaps most importantly, the SPARC intervention lacked an academic component that directly linked the home computer to any aspect of study participants’ schoolwork. Second, few treatment households took advantage of the voluntary training in basic computer skills that was made available through the SPARC intervention. It is therefore likely that some treatment students lacked the incentive and skills needed to maximize the educational potential of their home computers. Equally importantly, they may have lacked a full appreciation of how the computers could be applied to their schoolwork. As such, their educational use of the SPARC computers appears to have been limited to typing up reports and occasionally using the Internet to look up information about a subject. Third, significant student achievement outcomes were unlikely due to the short period of the intervention. Finally, many have argued that traditional grading and current standardized assessments may not be the best ways to measure the academic growth that occurs as a result of students’ use of learning technologies (Becker and Lovitts 2002; Rockman et al. 2003; Quellmalz and Zalles 2002).
Parent Outcomes
Parents of treatment students reported stronger computer skills than their control counterparts. However, the SPARC intervention did not affect any of the three aspects of parental involvement that resulted from the factor analysis. Taken together, these findings suggest that while home access to computers and the Internet led to a significant improvement in treatment parents’ computer skills, it did not compel treatment parents to be more actively involved in most aspects of their children’s education. It is worth noting, however, that the SPARC intervention did result in greater parental involvement for specific interactions that required computer and/or Internet use—e.g., using the Internet to help their child with schoolwork.
Contact Information for the SPARC Intervention and Evaluation
Sean D. McDonough
Acting Director, Bureau of Educational Technology
PA Department of Education
333 Market St., Harrisburg, PA 17126-0333
717-705-2558 (office) / (717) 787-3148 (fax)
smcdonough@state.pa.us
Cathy Groller
Director of Curriculum and Instruction/Educational Technologies
Carbon Lehigh Intermediate Unit #21
4210 Independence Drive
Schnecksville, Pennsylvania 18078-2580
grollerc@cliu.org
Gary Silverstein
Westat
1650 Research Boulevard
Rockville, Maryland 20850
(301) 251-2244
silverg1@westat.com
