If a study such as this one was to be replicated in another state, the first recommendation would concern the methodology of choosing the Program Schools and the Control Schools. It’s difficult to conduct scientific research in a humanistic environment. A provision to reward the Control Schools for their patience and participation should be built into the model on the front end. Also, the participating schools should create and submit a sustainability plan that would go into effect as the grant monies were ending.

I would share this advice with anyone conducting rigorous evaluations at the State Level: 1.) Have a great Research and Evaluation team and communicate with them regularly. 2.) The outside evaluators should be involved from the very beginning of the project. 3.) The Research and Evaluation team should be able to listen to what the research design requires as well as be aware of the nuances that indicate the need for a tweak or change along the way. 4.) The Research and Evaluation team should also be able to write meaningful reports appropriate for various audiences to supply and disseminate results of the project. 5.)

In summation, the Research and Evaluation team should be like great automobile mechanics, in that, they should know all of the intricacies of the engine and be able to diagnose and fix any issues that may present themselves along the way.

If applicable, describe your plans for continuing the project (sustainability; capacity building) and/or disseminating the project results. What tools, methods, or products have been verified, for example, through this process that might be useful to others?

Through this project, the state greatly enhanced the capacity for rigorous research evaluations. The districts and schools now understand that evaluation, much like formative assessment, must occur and when necessary corrective actions must be taken. The resulting data from the project has been and continues to be greatly appreciated at all levels. The districts and schools have reported that many decisions are made based upon data that are collected through the TESS (Technology in Education Survey System) survey. The dissemination of the project results has been ongoing. The research and evaluation team have published the results in several academic journals, as well as, presented the project at conferences across the United States.

Face validity of the school climate items and logical ordering of the items by scales were established during the development of the inventory (Butler & Alberg, 1991). Subsequent analysis of responses collected during this project substantiated validity of the items. Internal reliability coefficients on all scales of the inventory were determined using Cronbach’s alpha. The lowest coefficient was α = 76, and the highest was α = 87.

Therefore, these tools would be quite useful to others replicating this study in that they revealed a strong internal reliability rating for all scales.

The methods of the research project were clearly designed and implemented. In a replication activity in similar environments, the methodology should provide a suitably sturdy framework by which to begin. These tools and methodology would be most useful to anyone wanting to conduct a similar study.

Butler, E. D., & Alberg, M. J. (1991). The Tennessee School Climate Inventory: Resource Manual. Memphis, TN: Center for Research in Educational Policy, The University of Memphis