Major Second-Year (2005-06) Findings
Effects of Immersion on Teachers and Teaching
Immersion teachers grew in technology proficiency and in their use of technology for professional productivity at significantly faster rates than control teachers. In a self-assessment of their technology proficiency across three time points, immersion teachers considered themselves to be increasingly more technology literate than control teachers in areas involving technology operations and pedagogical skills. Similarly, teachers in immersion schools used technology significantly more often for administrative and classroom management purposes.
Teachers in immersion schools expressed stronger ideological associations across time with technology integration and learner-centered practices. Immersion teachers changed their instructional beliefs at a significantly more positive rate than control teachers. Immersion teachers increasingly employed technology integration actions, such as promoting students’ authentic problem solving or critical thinking through technology. They also expressed increasingly stronger affiliations with constructivist or learner-centered practices, such as having students establish individual learning goals, emphasizing experiential learning, and providing real-world experiences.
Teachers at schools with higher concentrations of student poverty grew in technology proficiency and adopted new ideologies at slower rates. Teachers who taught at schools with higher student poverty levels grew in technology proficiency and embraced technology integration and learner-centered practices at slower rates than their peers in more advantaged schools. Weaker supports for implementation at more impoverished immersion schools as well as the characteristics of teachers employed in those schools (proportionately more male teachers who were less likely than females to embrace innovative methods) may at least partially explain immersion teachers’ progress.
Given greater abundance of technology, teachers in immersion schools collaborated more often with their peers on technology-related issues than control teachers, and students used technology more often in immersion classrooms. Teachers at immersion schools compared to control had a significantly steeper growth rate for collaborative interactions with colleagues that supported improvements in instructional practices (e.g., developing lesson plans, exchanging information about students), as well as for the frequency of their students’ classroom activities involving technology. Despite their positive growth trend, statistics indicated that by spring 2006 teachers in immersion classrooms had students use various technology resources infrequently (i.e., about once or twice a month). While the overall level of classroom technology use was low, practices varied across teachers and core-subject areas.
Availability of technology resources had little, if any, effect on the intellectual challenge of immersion teachers’ lessons. Technology immersion’s theorized impact on student achievement hinges on technology’s facilitation of more rigorous and authentic learning experiences. Observations of core-subject teachers in fall 2004 and spring of 2005 and 2006 revealed no statistically significant differences between the intellectual demand of immersion and control teachers’ lessons. Across classrooms, lessons generally failed to intellectually challenge students. Observed activities most often focused on student acquisition of facts, definitions, and algorithms, and less often centered on writing lesson-related communication, constructing knowledge (e.g., synthesizing, explaining), or engaging in disciplined inquiry (e.g., investigation, experimental inquiry).
Effects of Immersion on Students and Learning
Technology immersion significantly increased students’ technology proficiency and narrowed the gap between economically advantaged and disadvantaged students. Estimated yearly growth in proficiency for economically advantaged and disadvantaged immersion students in Cohort 1 were nearly twice the rates for their control-group counterparts. Consequently, by the end of seventh grade, economically disadvantaged students in immersion schools surpassed advantaged control students in proficiency. Similarly, for Cohort 2, sixth graders, immersion had a significantly positive effect on students’ technology proficiency (ES = 0.30).
Students in immersion schools used technology significantly more often in core-subject classrooms and interacted more frequently with their peers in small groups. Cohort 1 students at immersion schools had a significantly steeper growth trend for the frequency of classroom activities with technology than control students. Results for Cohort 2 students, similarly, revealed significant and practically important differences in classroom activities favoring immersion schools (ES = 0.83). Students in immersion schools also had more frequent opportunities to learn with other students in small groups and to take a more active learning role.
Although immersion students used technology more often, classroom observations showed that they used technology in rather conventional ways. Observed students most frequently used a word processor for writing, learned and practiced skills (typically multi-choice exercises or digitized worksheets), created or made presentations (using PowerPoint or Keynote), or conducted Internet searches for information on an assigned topic. In general, changes in classroom activities and organizational structures in immersion classrooms did not necessarily alter the rigor or relevance of students’ experiences with core-subject content.
Technology immersion had no significant effect on student self-directed learning. We theorized that opportunities for independent and self-guided learning afforded through one-to-one technology would positively affect students’ personal self-direction. Findings in the second year replicated first-year results showing there was no significant immersion effect on self-directed learning. As both immersion and control students in Cohort 1 progressed from sixth to seventh grade, their responses to statements measuring self-direction revealed a significantly negative growth trend. Results for Cohort 2 students, similarly, revealed no significant immersion effect (ES = 0.03).
Outcomes for student engagement varied. Students in immersion schools had significantly fewer disciplinary actions, similar levels of school satisfaction, and significantly lower school attendance rates than control-group students. Disciplinary Action Reports for the 2005-06 school year showed that immersion students had proportionately fewer behavioral and disciplinary problems than their counterparts in control schools (ES = 0.14 and 0.16 for Cohorts 1 and 2, respectively). Conversely, surveys of students’ school satisfaction showed no significant differences between immersion and control students’ satisfaction with the kinds of work they do in classes or with the relevance of their schoolwork.
Unexpectedly, technology immersion had a significantly negative effect on school attendance. For Cohort 1 students, school attendance rates declined across years, and by the end of seventh grade, the estimated average attendance rate for economically advantaged immersion students was 95.9% compared to 96.4% for control students (rates were lower for disadvantaged students). Results for Cohort 2 students, similarly, showed statistically significant but small differences in attendance rates favoring students in control schools (ES = 0.07).
Effects of Immersion on Academic Achievement
Technology immersion’s ultimate goal is increasing students’ achievement in core academic subjects as measured by state assessments. For analyses reported below, students’ TAKS scale scores were standardized and then normalized as T scores with a mean of 50 and a standard deviation of 10.
Technology immersion had no statistically significant effect on Cohort 1, seventh graders’ achievement in reading, mathematics, or writing. For Cohort 1 students, we used three-level HLM growth models to estimate mean rates of change in TAKS reading and mathematics scores and a two-level HLM model to estimate the effects of immersion on TAKS writing scores.
• Reading. Controlling for student and school poverty, there was no significant effect of immersion on students’ growth rate for TAKS reading. The immersion effect was positive but not by a statistically significant margin. Economically disadvantaged students in both immersion and control schools grew in reading achievement at a significantly faster rate than their more advantaged peers. Combined with the positive immersion result, this yielded a positive boost in reading achievement for disadvantaged immersion students.
• Mathematics. After controls for student and school poverty, there was no significant effect of immersion on students’ growth rate for TAKS mathematics. The immersion effect was positive but not by a statistically significant margin. In contrast to reading, economically disadvantaged students at both immersion and control schools grew in mathematics achievement at a significantly slower rate than their more advantaged peers.
• Writing. After adjusting for Cohort 1 students’ initial TAKS writing scores (as fourth graders in 2003), student demographic characteristics, and school poverty, there was no statistically significant difference in the 2006 writing scores for students in immersion and control schools. The immersion effect was negative but not by a statistically significant margin.
Technology immersion had no statistically significant effect on Cohort 2, sixth graders’ reading achievement. However, immersion had a significantly positive effect on mathematics scores for higher achieving students. We analyzed the effects of immersion on Cohort 2 students’ TAKS reading and mathematics scores using two-level HLM models.
• Reading. Controlling for students’ prior achievement (as fifth graders in 2005), demographic characteristics, and school poverty, there was no statistically significant difference in the 2006 TAKS reading scores for students in immersion and control schools. The immersion effect on reading was positive but not by a statistically significant margin.
• Mathematics. After controls for students’ prior achievement (as fifth graders in 2005), demographic characteristics, and school poverty, there was no overall significant difference between immersion and control students’ TAKS mathematics scores. The immersion effect was positive but not by a statistically significant margin. However, there was a statistically significant immersion effect on mathematics achievement that acted through students’ pretest scores. Other factors being equal, having higher pretest scores predicted larger gaps in 2006 math scores favoring immersion students. Thus, immersion had a significantly positive effect on mathematics achievement for higher achieving sixth graders.
Second-year achievement trends generally favored technology immersion schools. Although TAKS scores for immersion and control students usually did not differ by statistically significant margins in the second year, noteworthy achievement trends emerged. In the first project year, TAKS reading and mathematics achievement trends favored control schools. Conversely, in the second year, immersion schools had more positive achievement trends than control schools across both Cohorts 1 and 2 and for both reading and mathematics subject areas. Outcomes for TAKS writing, in contrast, favored students in control schools. The analysis of writing achievement, however, differed from other subject areas in the wider span of time between the pretest (4th grade) and posttest (7th grade). The testing mode for writing could also have affected outcomes. Immersion students who regularly use word processors for writing may be at a disadvantage when completing a writing assessment in traditional paper-and-pencil format.
Second-year findings provide formative evaluation outcomes. The evaluation of technology immersion is a four-year, longitudinal study, and findings from the second year provide preliminary outcomes. In designing the study, we thought that some effects might emerge during early implementation, but we also believed that changes in longer term outcomes, such as student achievement, might require at least three years to surface (i.e., time for Cohort 1 students to progress from sixth to eighth grade). Moreover, while student achievement results as measured by TAKS scores are extremely important, there are other outcomes for immersion students that may contribute to their long-term success. Certainly, technology immersion has narrowed the technology equity gap for economically disadvantaged students. Many students who previously had no technology in their homes are becoming computer literate through their experiences with laptops. Administrators, teachers, and students alike believe that middle school students at immersion schools are better prepared for future educational and workforce requirements and for 21st Century expectations, such as communication skills, and information and media literacy. In the sections to follow, we describe how the generally low levels of implementation may have contributed to second-year results.
Nature of Second-Year Implementation
Most of the middle schools had difficulty in the second year implementing the prescribed components of technology immersion. Full implementation of the immersion model requires support in several ways: Leadership, Teacher Support (buy-in), Parent and Community Support, Technical Support, and Professional Development. Given adequate supports, teachers are expected to reach high levels of Classroom Immersion, and Student Access and Use of technology is expected to be robust. The Implementation Index, a composite campus z score measuring the strength of immersion components, showed that some middle schools had a stronger presence of immersion components that nearly approximated expected standards. Mean immersion standard scores (ranging from 2.48 to 3.06 on a 0 to 4.00 scale) indicated that supports for immersion generally failed to meet full implementation standards (3.50 to 4.0). Given generally low-to-moderate supports for immersion, the average levels of Classroom Immersion (2.48) and Student Access and Use (2.17) were below expectations. Major concerns included students’ inconsistent use of laptops across classrooms and subject areas, uneven provision of professional development supporting the design of effective technology-infused lessons, and variability in students’ access to laptops during the school day and at home.
The strength of professional development and other supports were associated with higher levels of classroom and student immersion. Variability in the quality of professional development provided by schools was a major obstacle to teachers’ growth in creating technology-immersed classrooms. While the immersion model requires that a quarter of grant funds be expended for professional development, the design rested largely with individual districts and campuses and their selected technology vendors (mainly Apple or Dell). Our measure of the strength of the campus professional development component was significantly correlated with teachers’ reported levels of classroom immersion. Leadership for immersion also emerged as an important factor in advancing change. Administrators appeared to influence teachers’ attitudes toward technology through their provision of supports for changed practice. Similarly, students’ access to and use of technology for learning was significantly related to their teachers’ greater involvement in professional development and the strength of other school supports for immersion.
A continuing challenge in the second year was the consistent provision of laptops for students both within and outside of school. Student laptop access varied widely both across and within schools. The average number of laptop access days reported by students ranged from 42 to 178 days, with only a few campuses achieving full access (the targeted 170 to 180 days per student). Student laptop access was limited by factors such as disciplinary infractions, technical issues, time for repairs, and in a few cases, parent resistance. Additionally, some immersion schools allowed students to have unlimited access to laptops outside of the school day, while others restricted students’ out-of-school access to a series of days or to laptop check-outs for teacher-assigned schoolwork. Overall, laptops’ potential influence on learning varied across students and schools.
Schools with a greater proportion of economically disadvantaged students had lower implementation levels. Schools with larger concentrations of student poverty had significantly lower levels of implementation. Accordingly, teachers at these schools grew in proficiency and created immersed classrooms at significantly slower rates than teachers in more advantaged schools. Schools serving predominantly disadvantaged and often low-performing student populations faced special challenges in implementing a project requiring profound school and classroom change.
