Introduction
The State of Tennessee Department of Education has developed and implemented a number of educational interventions that incorporate technology applications to improve student outcomes, particularly in core academic areas. In conjunction with these programs the state conducted a three year evaluation of the systematic educational technology intervention. The Tennessee EdTech Accountability Model (TEAM) is a rigorous, scientifically based high quality evaluation of those interventions.
The TEAM evaluation began during the 2003-2004 school year using a rigorous quasi-experimental method (matched treatment-control schools). Data collection took place at 26 matched pair schools, 13 of which were grant recipients and 13 which served as control schools. In 2004-2005, 14 additional treatment and control schools were added to the study. A vast amount of information concerning the extent and quality of instructional strategies, student and teacher technology use, program implementation, stakeholder perceptions, and student achievement has been gathered.
Methodology
The Tennessee EdTech Accountability Model (TEAM) is a rigorous, scientifically based high quality evaluation of those interventions. Three research questions were formulated to help guide the study. The research questions asked the following:
- Does implementation of the TnETL model raise student achievement?
- Does implementation of the TnETL model increase the use of research-based classroom practices?
- Does implementation of the TnETL model improve students’ ability to use technology as a tool?
To answer the research questions, comparative analyses were conducted on classroom practices, teacher attitudes, student performance outcomes, and student achievement. Descriptive analyses were used to describe reactions to TnETL Program by teachers, principals, and technology coaches. The evaluation methodologies involved using the Formative Evaluation Process for School Improvement – Technology Package (FEPSI/TP). Data gathered through the FEPSI/TP were derived from direct classroom observation, teacher surveys, interviews/focus groups, technology implementation benchmarks, and analysis of student achievement.
Results
- TnETL helped increased student engagement.
- TnETL students and teachers increased the use of technology tools.
A promising trend emerged as the Program students out-performed or performed as well as Control students in all instances except with regard to Launch 2 5th grade math and language arts, however, they also emerge with more experience using technology as a learning tool in meaningful computer activities. Students in Program classrooms were significantly more engaged in student-centered learning activities and were better able than the Control students to demonstrate the application of critical thinking skills, which for some students resulted in superior or comparable TCAP math and language arts performance.
Overall, both the randomly conducted whole school and targeted observations revealed that the instructional strategies implemented in TnETL Program schools were more reflective of research-based practices that accommodate technology integration that those observed in Control classes. The Program teachers were better able to integrate greater and higher-quality use of computers as a learning tool and for instructional delivery, as compared to Control teachers. The students’ attention and interest was more frequently observed in Program classes than in Control classes. Though these results are positive and certainly reflect the goal of TnETL, continued professional development is needed to better prepare teachers to increase the frequency and intensity of implementation, which could yield greater and more consistent improvement in student learning.
Overall, the project was successful. Anytime that technology flows seamlessly through instruction, there is success. Teachers’ attitudes grew more positive toward technology. Students were more engaged in their learning, which was evidenced through the outcomes of the problem based learning situations. The school climate was perceived to be better during the course of the study. The observations and surveys supported the hypothesis that when technology is used as a tool, student performance increases and improves.
Contact Information:
Tennessee Department of Education
Dr. Barbara Denson
710 James Robertson Pkwy.
Andrew Johnson Tower – 5th Floor
Nashville, TN 37243
