Project Design
In order to conduct the evaluation of the IMPACT model, the central evaluation team at NCSU and the Friday Institute designed a quasi-experimental (matched subjects) longitudinal evaluation looking at multiple outcomes, which included teacher, student, and administrator skills, attitudes, and behaviors.   Each IMPACT school was carefully matched with a similar school that could serve as a comparison.  Schools were matched based on factors such as geographical proximity, demographics, and school configuration, and multiple measures were used to assess outcomes at each level. 

Within the experimental component of the study, IMPACT schools were randomly assigned to IRCMS treatment or comparison condition, and recruits from geographically proximal schools were also assigned to condition. Multiple measures were used to assess outcomes for teachers and students.

Research Questions: Quasi-Experimental Design
For the quasi-experimental (IMPACT) component of the study, multiple variables were examined through a longitudinal repeated-measures approach.  The hypotheses used took the following form:

Ha:  Students or teachers participating in the IMPACT program will show greater change from the baseline through the third year than matched schools not participating in IMPACT
    μ1d - μ1a  > μ2d - μ2a
Ho:  Students or teachers participating in the IMPACT program will not show greater change from the baseline through the third year than matched schools not participating in IMPACT
μ1d - μ1a  = μ2d - μ2a

Variables that were addressed within the study included teacher technology use, teachers’ attitudes toward technology, teacher retention, student reading, math, and writing achievement, and student technology skills.

Experimental Design: Research Questions
Within the experimental design component (IRCMS), separate research questions were posed.

Students

1.    How changes in reading comprehension varied across condition?
2.    How changes in self-efficacy for reading varied across condition?
3.    How changes in metacognitive awareness varied across condition?
4.    How changes in strategy knowledge varied across condition?

Teachers

1.    How changes in self-efficacy for teaching reading comprehension varied across condition?
2.    How changes in metacognitive knowledge varied across condition?
3.    How did teachers in the two IRCMS conditions vary in terms of treatment integrity?