Fullan (2005) suggests that in order for capacity building to be sustainable, it needs to be tri-level, i.e., to take place at the school, district, and state/federal levels. The lessons learned from the IMPACT schools suggest that there are certain things that policymakers and administrators at the state and federal and state levels can do that will ultimately improve their capacity for planning, conducting, and communicating the results of project evaluations.

Program: Setting the Evaluation in a Context. Perhaps the most important thing that federal and state program planners can do for grantees is to make every effort to ensure that the grantees understand the purpose of the overarching program (beginning with the federal statute and the U.S. Department of Education’s priorities), see how their particular project fits into the overall program, and appreciate the importance of their formative and summative project evaluations. It would save the would-be evaluation team a great deal of time, frustration, and angst if evaluation were built into the overall program design and if all of the varying purposes for the evaluation were identified and addressed at the outset, i.e., as the program is being designed. It would also facilitate matters if the program design were to include (a) the purpose(s) of the evaluation; (b) expectations for the scope and nature of the evaluation, i.e., formative and/or summative, internal and/or external; (c) components of the project evaluations, such as evaluation questions, data sources, methods for data analysis, timeline; (d) roles of various stakeholders in planning and conducting the evaluation; (e) expectations for reporting the results; and (f) anticipated uses of the results.

Expectations: All too often, funding agencies are disappointed with the quality of the evaluation plans in grant proposals and the quality of evaluation reports. This problem can be ameliorated somewhat if the Request for Proposals (RFP) or Grant Application Package outlines the agency’s expectations for the evaluation and provide enough structure such that inexperienced evaluators can develop a sound plan, but allow flexibility for the applicants to tailor the evaluation to their particular projects. A lot of time and energy could be saved if the RFP referred to specific data collection instruments that grantees can use for their project evaluations - so they won’t spend time developing their own instruments.

Communication: Evaluation guidance or requirements are commonly passed downward from federal program officers, through SEA administrators (often multiple levels of them), to district administrators, and ultimately to school staff members. Project goals and objectives, management expectations, accountability requirements, and expectations of dissemination must be effectively communicated through all these levels. Similarly, evaluation data and findings are typically reported upward through those same levels and must be communicated consistently and in ways that meet the needs of each successive level of management. Communication among all these various levels must take place in ways that are timely—and timed—to effectively engage stakeholders in the evaluation and thus ensure the effective implementation of the evaluation. Developing a plan for communicating with all stakeholder groups before project implementation begins and that encompasses new knowledge, roles and responsibilities, pathways and preferred methods for communication will go a long way in reducing confusion and unmet expectations, and toward gaining buy-in, understanding, and engagement.