What is the relationship between scientifically based research and evaluation?
Program evaluation and research are closely related and should be synergistic. However, program evaluation and research serve different purposes and need to meet some different requirements. Bob Spielvogel of EDC has described the difference as "evaluation is designed to improve something while research is designed to prove something." That is, evaluation is specific to a project or program, asking whether how it can best accomplish its goals (often called formative research) and documenting whether those goals were accomplished in the end (often called summative research). Research is designed to provide results that go beyond the individual project and can be generalized to other contexts and populations. This places additional requirements on research. For example, an evaluation considers the results on the population being served by the program within the context in which the program is provided. Research has to ask whether that population has the same characteristics as other groups and whether the results can apply in other contexts.
If we consider the criteria for SBR presented in question 2, we find that some are central to both evaluation and research:
While others are more central to research than to evaluation:
Implicit in these definitions is the idea of variation and differences. The way empirical studies demonstrate the effectiveness of a program is by relating differences among outcomes (e.g., student performance measures to differences between entities (e.g., schools or classrooms). Typically, the between entities is embodied by the application of a particular program. Groups of entities are distinguished by the presence or absence of program services or resources. When the differences among outcomes can be related to the differences between entities, one can conclude that the differences in outcomes are due to the presence of the program. One demonstrates the relationship between outcomes and entities by conducting systematic comparisons.
Rigorous program evaluation often results in the identification of promising practices that should be subjected to more intense scrutiny and examination under the umbrella of research. This research extends beyond evaluation to yield generalizable findings and results that inform theory development and practice in contexts other than that of the original project.
For example, program evaluation studies often rely upon naturally occurring differences as sources of variation, for example, the ways individual schools choose to implement a single program. In this case, investigators seek to identify and classify the differences they discover and relate the differences to differences in outcomes. On the other hand, research studies attempt to control sources of variation in the study's design, e.g., by requiring that a program be implemented in different ways among a set of similar schools in order to understand how these planned differences produce differences in outcomes.
 |
 |
 |
Chapter Summary Frequently Asked Questions Steps to Implementing SBR Recommendations |
||
 |
 |
 |
|
|
 |
|
|
 |
|
|
 |