Research Design

The research design is quasi-experimental. Twenty-two technology immersion schools, selected through a competitive grant process, were matched by researchers with 22 control schools on key characteristics, including size, regional location, demographics, and student achievement. Selection of control schools involved the generation of a pool of grades 6-8 middle schools eligible to receive federal Title II, Part D funds. Next, researchers used statistical procedures to identify schools that matched treatment campuses as nearly as possible on factors, including (a) district and campus size, (b) regional location, (c) the proportion of economically disadvantaged and minority students, (d) percentage of students passing all TAKS tests, and (e) the gaps between the percentage of White students and African American and Hispanic students passing TAKS (all tests). Researchers selected control schools from a list of “best matches” for each treatment school. All of the selected control schools agreed to participate, except for one. The first alternate was selected for the school that declined participation.

Given the limitations of the quasi-experimental design, we analyzed extensive baseline data to establish the comparability of treatment and control groups. In the project’s second year, however, two middle schools in one district (one immersion and one control) were excluded from analyses due to damage caused by Hurricane Rita on the Texas Gulf coast. Thus, second-year results are for 21 treatment and 21 control schools. A re-analysis of baseline data revealed that differences between group characteristics remained statistically insignificant. Thus, the integrity of the research design appears sound. Since immersion schools have somewhat larger proportions of economically disadvantaged and limited English proficient students, we used statistical methods to adjust for differences that arise from sampling variability. Generalization of findings to a broader population is a primary study limitation. Compared to Texas middle-school students as a whole, students in the sample schools are substantially more Hispanic and less White and African American. Middle schools are also smaller than the statewide average (402 students versus 667).

Setting and Participants
The study includes grades 6-8 middle schools drawn from rural, suburban, and urban locations in Texas. Middle schools are typically small, with about three-quarters of schools enrolling 600 students or less, and are located in either small or very small public school districts (enrolling 2,999 students or less) or large districts (enrolling 10,000 students or more). In the second project year, 1,257 teachers participated in the study (604 at treatment schools and 653 at control). Teachers are remarkably similar in terms of gender, ethnicity, advanced degrees, and average teaching experience.

The second-year study centers on two student cohorts. Cohorts 1 (sixth graders in 2004-05 and seventh graders in 2005-06) included a total of 5,538 students, with 2,627 treatment-group and 2,911 control-group students. Cohort 2 (sixth graders in 2005-06) included a total of 5,507 students, with 2,685 treatment-group and 2,822 control-group students. Comparison groups have nearly equal proportions of economically disadvantaged students (73-76%), minority students (6-8% African American and 67-73% Hispanic), and male and female students (about 50% each). The main difference between groups is the greater proportion of limited English proficient students in treatment schools (23%, 30%) compared to control schools (16%, 19%).

Data Collection
Data collection involved a mix of qualitative and quantitative data sources. Researchers conducted site visits at each of the middle schools in fall 2004 and spring of 2005 and 2006. Data gathered during site visits included an in-depth examination of campus conditions, school and classroom activities, and educational roles and processes through interviews with key administrators and technology leaders, focus groups with teachers and students, and observations in core-subject classrooms (English/language arts, mathematics, social studies, and science).

Additional measures included annual online teacher surveys and student paper-and-pencil surveys. The Teacher Questionnaire includes measures of school technology (Leadership, Technical Support, Innovative Culture, and Parent and Community Support) as well as other teacher mediating variables (Technology Proficiency, Professional Productivity, Student Classroom Activities, and Collaboration). Each teacher completed a baseline survey and then completed surveys in the spring of each project year. Measures of internal consistency (Cronbach’s alpha) ranged from 0.66 to 0.99. Response rates ranged from 87% to 98% across time periods. Classroom observations in sixth- and seventh-grade classrooms documented student and teacher activities, including technology use, as well as the intellectual challenge of classroom work.

Students completed a Technology Survey as sixth graders in fall and then completed the survey again every spring. The survey measured Technology Proficiency, School Satisfaction, Classroom Activities, Technical Problems, and Small-Group Work. Cronbach’s alpha reliability coefficients for scale scores ranged from 0.77 to 0.94; response rates ranged from 80% to 90% across time periods. Students also completed the Style of Learning Inventory (SLI), a measure of self-directed learning. The SLI has a reliability coefficient of 0.89; response rates ranged from 77% to 82%. Students completed the Texas Assessment of Knowledge and Skills (TAKS) in the spring of each year. Students had baseline and annual measures for TAKS Reading and Mathematics. Seventh graders completed TAKS Writing. Since TAKS scale scores are not equated across grade levels, researchers generated standard scores (T scores) for analyses. We also collected students’ school attendance rates from the TEA, and students’ Disciplinary Action Reports from each campus.

Data Analysis

Researchers used hierarchical linear modeling (HLM) to estimate the effects of immersion on teacher and student mediating variables and student academic achievement (TAKS scores). For teachers and Cohort 1 students, we used three-level HLM growth models to examine the effects of technology immersion on individual growth rates for various measures. Level 1 is a repeated-measures model (i.e., survey time within teachers, survey or assessment time within students), Level 2 is the between-subjects model, and Level 3 is the school level. Analyses contrasted the growth trajectories for teachers and students at immersion and control schools. We analyzed the effects of immersion on Cohort 2 students’ scores using two-level HLM models. Level 1 is the student-level model and Level 2 is the school-level model. For two-level HLM models, we calculated effect sizes (ES) in standard deviation units (usually Cohen’s d).