Introduction
The Technology Immersion Pilot (TIP) sets forth a vision for technology immersion in Texas public schools. Technology immersion encompasses multiple components, including a laptop computer for every student and teacher, wireless access throughout the campus, online curricular and assessment resources, professional development and ongoing pedagogical support for curricular integration, and technical support to maintain an immersed campus. The Texas Education Agency (TEA) originally directed more than $14.5 million in federal Title II, Part D monies toward funding a wireless learning environment for high-need middle schools through a competitive grant process. A concurrent research project funded by a federal Evaluating State Educational Technology Programs grant is evaluating whether student achievement improves over time as a result of exposure to technology immersion. The Texas Center for Educational Research (TCER)—a non-profit research organization in Austin—is the TEA’s research partner in this five-year endeavor.
Purpose of the Study
The overarching purpose of the study is to scientifically investigate the effectiveness of technology immersion in increasing middle school students’ achievement in core academic subjects as measured by the Texas Assessment of Knowledge and Skills (TAKS). The evaluation also examines the relationships that exist among technology immersion, mediating variables (school, teacher, and student), and student achievement. Research questions are as follows.
• How is technology immersion implemented, and what factors are associated with higher implementation levels?
• What is the effect of technology immersion on teachers and teaching?
• What is the effect of technology immersion on students and learning? and
• Does technology immersion affect student achievement?
The Theoretical Framework for Technology Immersion guides the evaluation. The experimental research design allows an estimation of the effects of technology immersion, which is the difference between the treatment and control groups. The framework postulates a linear sequence of causal relationships. Experimental schools are to be “immersed” in technology. An improved school environment for technology presumably leads teachers to create technology immersed classrooms. In turn, improved school and classroom conditions theoretically contribute to improved student learning and achievement.
Technology Immersion
As a way to ensure consistent interpretation of technology immersion and comparability across sites, the TEA issued a Request for Qualifications (RFQ) that allowed commercial vendors to apply to become providers of technology immersion packages. Vendors had to include six components in their plan: (a) a wireless mobile computing device for each educator and student on an immersed campus; (b) productivity, communication, and presentation software for use as learning tools; (c) online instructional resources that support the state curriculum in English language arts, mathematics, science, and social studies; (d) online assessment tools to diagnose students’ strengths and weaknesses or to assess their progress in mastery of the core curriculum; (e) professional development for teachers to help them integrate technology; and (f) initial and ongoing technical support. Based on an expert-review process, the TEA selected three lead vendors as providers of technology immersion packages (Dell Computer Inc., Apple Computer Inc., and Region 1 Education Service Center [ESC]). Six middle schools selected the Apple package, 15 selected the Dell package, and 1 school selected the Region 1 ESC package (Dell computer).
Methodology
Research Design
The research design is quasi-experimental. In the first year (2004-05 school year), the study included 22 experimental and 22 control schools. Interested districts and associated middle schools responded to a Request for Application (RFA) offered by the TEA in spring 2004 to become technology immersion schools. Applicants had to meet eligibility requirements for Title II, Part D funds (i.e., high-need due to children from families with incomes below the poverty line, schools identified for improvement, or schools with substantial need for technology). Technology immersion schools were matched by researchers with control schools on key characteristics, including eligibility for Title II funds, size, regional location, demographics, and student achievement. In the project’s second year, two middle schools in one district (one experimental and one control) were lost due to damage caused by Hurricane Rita on the Texas Gulf coast. Thus, second-year results (2005-06 school year) are for 21 treatment and 21 control schools. A re-analysis of baseline data revealed no statistically significant differences between the comparison groups.
Setting and Participants
The study includes grades 6-8 middle schools drawn from rural, suburban, and urban locations in Texas. Middle schools are typically small (402 students, on average), but enrollments vary widely (from 83 to 1,447 students). In the second project year, 1,257 teachers participated (604 at immersion and 653 at control campuses). The study focused on two student cohorts. Cohort 1 included 5,538 seventh graders (2,627 immersion, 2,911 control) who completed their second project year; Cohort 2 included 5,507 sixth graders (2,685 immersion, 2,822 control) who finished their first year. Nearly three-quarters of students are economically disadvantaged (about 75%) and represent minority groups (roughly 70% Hispanic and 7% African American).
Data collection and Analysis
Data collection involved a mix of qualitative and quantitative data sources. Researchers conducted site visits involving interviews, focus groups, and classroom observations at each of the middle schools in fall 2004 and spring of 2005 and 2006. Additional measures included annual online teacher surveys and student paper-and-pencil surveys. We also gathered school and student demographic, attendance, and achievement data from the Texas Public Education Information Management System (PEIMS) and Academic Excellence Indicator System (AEIS), and data on student disciplinary actions from schools. Researchers used either two- or three-level hierarchical linear models (HLM) to estimate the effects of immersion on teacher and student mediating variables and student TAKS achievement in reading, writing, and mathematics.
Study Limitations
Generalization of findings to a broader population is a primary study limitation. Compared to Texas middle-school students as a whole, students in the sample schools are substantially more Hispanic and less White and African American. Middle schools are also smaller than the statewide average. Additionally, the study relies on self-reported data from students and teachers for many outcome variables. Nonetheless, the triangulation of evidence from multiple sources (surveys, classroom observations, state demographic and test databases, student cohorts) verifies the robustness of findings.
Results
The evaluation of technology immersion is a four-year longitudinal study. Schools began planning and initial implementation during the 2004-05 school year; year two (2005-06) was the first full year of implementation. Implementation will continue through the 2007-08 school year. Our first-year report—Evaluation of the Texas Technology Immersion Pilot: First-Year Results (Shapley et al., 2006)—revealed positive effects of technology immersion on schools, teachers, and students.
Findings for the second year relative to these same variables are generally consistent with first-year results. Steadfast outcomes across two evaluation years and two student cohorts show that immersing a middle school in technology produces schools with stronger administrative leadership for technology, greater teacher collaboration and collective support for technology innovation, and stronger parent and community support for technology. Additionally, teachers in immersion schools are more technically proficient and use technology more often for their own professional productivity, their students use technology more often in core-subject classrooms, and teachers adopt more integration-oriented and learner-centered ideologies. Students in immersion schools are more technically proficient, use technology more often for learning, interact more often with their peers in small-group activities, and have fewer disciplinary problems than control-group students.
Also consistent with first-year results, we found no significant effect of technology immersion in the second year on student self-directed learning, and we found a significantly negative immersion effect on student attendance. Moreover, the availability of technology across two years provided no significant increase in the intellectual challenge of immersion teachers’ core-subject lessons.
First-year findings on academic achievement revealed no statistically significant immersion effects on TAKS reading or mathematics scores for Cohort 1, sixth graders. Similarly, second-year results for Cohort 1 students (as seventh graders) showed no significant effects of immersion on TAKS reading, mathematics, or writing achievement. Likewise, achievement results for Cohort 2 students (sixth graders involved in the project for one year) revealed no significant effect of immersion on TAKS reading achievement. However, for TAKS mathematics, students in immersion schools who began the year with higher math pretest scores had significantly higher mathematics achievement than their control-group counterparts. The math achievement gap favoring immersion students over control widened as students’ pretest scores increased. Although TAKS score differences between immersion and control schools usually did not differ by statistically significant margins, second-year achievement trends, in contrast to first-year results, generally favored technology immersion schools.
Findings for the first two years provide preliminary outcomes. In designing the study, we thought that some effects might emerge during early implementation, but we also believed that changes in longer term outcomes, such as student achievement, might require at least three years to surface (i.e., time for Cohort 1 students to progress from sixth to eighth grade). Additionally, there are other outcomes for immersion students that may contribute to their long-term success. Certainly, technology immersion has narrowed the technology equity gap for economically disadvantaged students. Many students who previously had no technology in their homes are becoming computer literate through their experiences with laptops. Administrators, teachers, and students alike believe that middle school students at immersion schools are better prepared for future educational and workforce requirements and for 21st Century expectations, such as communication skills, and information and media literacy.
Contact Information:
Anita Givens
Senior Director, Instructional Materials and Educational Technology
anita.givens@tea.state.tx.us
Link to Websites
www.tea.state.tx.us/technology/tip
www.txtip.info
www.etxtip.info
