SETDA - TAPP
                                                      



 
Arkansas
Iowa
North Carolina
Pennsylvania
Tennessee
Texas
West Virginia
WV: Ed Pace
Wisconsin
TAPP

 

 SETDA.org

 
Methodology

The EAST evaluation involved two parallel and interdependent studies: a three year implementation study designed to provide insights into the variations in the implementation of the EAST program in different contexts throughout the state, and a two year outcomes study that sought to determine the impact of the initiative on participating facilitators and students by comparing eight schools who were randomly assigned from a larger pool of applicants to begin a new EAST program during the 2004-2005 school year, with a matched group of control students in eight schools that were assigned from the applicant pool as delayed implementation schools.  Following are the major research questions that the EAST evaluation study sought to answer:

Implementation Study—conducted in all existing EAST schools statewide throughout all three years of the project (2003-2006)

•    What recruitment strategies are being used? 
•    What are the characteristics of EAST schools, students, and teachers? 
•    What are the nature, quality, and intensity of training activities for EAST facilitators and students? 
•    What are the nature, quality, and intensity of the EAST instructional practices and how do they differ in the various program settings? 
•    What are the nature and quality of the measures currently being used to document program implementation? 
•    What are the nature and quality of the assessments currently being used to measure teacher and student outcomes? 
•    What are the nature and quality of participating schools’ partnerships with local industry and community organizations, and what factors influence these partnerships at the local level? 
•    What are the nature, quality, and intensity of EAST parent involvement programs and activities? 
•    What are the nature and quality of administrative support for the program? 
•    In what ways and to what extent do the EAST National Center and the ADE support implementation of programs and practices in the participating schools? 
•    What are the nature and quality of dissemination activities offered to participating sites, and what is the level of participation in these activities?
•    What steps have participating schools taken to sustain EAST beyond the three-year start-up period? 
•    What are the costs associated with the EAST program?
•    What obstacles, if any, have been encountered to program implementation, and what steps were or can be taken to overcome these obstacles? 

Outcomes Study—conducted during Years 2 and 3 of the evaluation (2004-2006) at 16 schools that were randomly assigned as new implementers (target) or delayed implementation (control) schools prior to the beginning of the study

•    What is the impact of exposure to the EAST program on students’ attitudes towards learning and school?
•    What is the impact of exposure to the EAST program on students’ problem solving skills and content knowledge in math and English language arts?
•    What is the initiative’s impact on facilitators’ attitudes towards teaching, classroom practices, and content knowledge?
•    What influence do additional factors, including environmental factors, participant demographics, training conditions, instructional practices, and varying degrees of program integrity, appear to have on the impact of the EAST program on student and teacher outcomes? 

Results from the Implementation Study
Data for the implementation study were obtained from facilitator and student focus group interviews, principal interviews, observations of EAST classes, and on-line principal and facilitator surveys that were administered in the spring of each year. 

On the principal and facilitator surveys, schools reported using a combination of student recruitment and selection strategies that promoted general awareness of the program and those that targeted specific students. Comparisons of demographics of EAST students and non-EAST students in the same schools were conducted as part of the outcomes study and are discussed below.

Surveys also asked facilitators and principals to report on their perceptions of the impact of program training activities.  Both groups have expressed very positive feelings about the training process throughout all three years of the study.  When asked to rate the adequacy of EAST training for bringing their skills to the level they needed as a facilitator, facilitators reported that the training was best for raising their skills in the instructional methods advocated by EAST, and also provided favorable ratings to training they received in assessing their own progress in facilitating EAST. 

Facilitators were consistently positive about the support available from their school administrations, with large majorities agreeing that their principals supported class scheduling in their schools in ways that benefited the EAST program.  Principals also echoed the facilitators’ impressions about their willingness to provide flexibility and facilitate class scheduling.  Principals also expressed strong agreement with almost all EAST philosophies, such as the value of learning in a real world context, encouraging self-directed learning, and the value of group work.