Introduction

This study is a three-year analysis of West Virginia's Technology Model Schools program documenting the multiple effects of professional development on the integration of technology into classroom instruction by elementary school teachers is the chief purpose of this analysis.

The Technology Model Schools program dramatically increased the use of technology by teachers and students in classrooms and that increase is associated with gains in Mathematics and Reading/Language Arts.  This is a detailed analysis of the ability of a professional development program to impact the classroom integration of technology and, through that, the achievement of students.  The study is unusual in its:  (1) multiple, independent and corroborating data sources; (2) statewide scope; and (3) multi-year horizon.

Methodology

This evaluation answers the following questions.  As a result of the Technology Model Schools program:
1.    Did student achievement go up and for whom?
2.    How did students use their computers?
3.    Were teachers benefited in their instructional and other duties?
4.    Was the turnkey expectation (that teachers would train other teachers) realized?
5.    Did the effects of the TMS program continue beyond the departure of the TISs?
6.    Did the program---as a whole---work?
7.    Can the Technology Model Schools program be replicated?
8.    Was student achievement in TMS schools higher than in other schools?

Over the two years of this study, teacher professional development from the Technology Model Schools program was delivered in three configurations.  In general, the achievement of students whose teachers had any of those three versions of the program's help was higher than students in classes without the program.  And, in general, two years of TMS help is associated with higher achievement than one year or no years of TMS help.

To determine the effectiveness of the EETT program, West Virginia contracted with Interactive Inc., a company that specializes in documenting the outcomes of learning technology. Interactive will use technology to measure the technology outcomes from the EETT program. Specifically, Interactive will use desktop meters and random interval data collection pop-up screens to document the use, time, topic and function budgets of teachers and students. The analysis will give direct, detailed and objective measures about two common interventions---professional development and turnkey training.

This is a quasi-experimental evaluation research design applied to an EETT-funded program and to control schools. These new documentation methods will, for the first time, yield objective, detailed information about classroom integration of technology. In addition to precise measurement of the outcomes of professional development, Interactive will be able to test amounts of classroom integration related to student performance on WV's new tests of standards-based content.

Results

For Math, the performance of students in different configurations of the Technology Model School program was consistently higher than in the control schools.  That was not the case for Reading/Language Arts.

We examined whether or not the TMS program was helpful to students from low-income, Title I-eligible backgrounds.  For Math achievement, the estimate for the two-year treatment group (TIS both years) and the control and treatment group (TIS 2nd year only) is higher than for the control group (no TIS either year).  Students eligible for Title I services had higher math achievement in both the one-year and the two-year TMS schools than their counterparts in the control schools.   

For Reading/Language Arts achievement the estimate for one-year treatment/turnkey group (TIS in the first year only) and the control-plus-treatment group  (TIS 2nd year only) is higher than for the control group (no TIS either year).  In two of the three TMS-service delivery groups, there were no statistically significant differences in Reading achievement between students who were and were not eligible for Title-1 assistance.  To that extent, the TMS program may be contributing to closing the achievement gap.

Contact Information

West Virginia Department of Education
            Division of Curriculum & Instruction
            Office of Technology and Information Systems
            Charleston, West Virginia

Dale Mann, PhD., Managing Director
Jonathan Becker, LLD & Ph.D., Research Director
            Interactive, Inc.

http://wvde.state.wv.us/evaluation/