I. Chapter Summary

II. eLearning Evaluation Lens [Resource Document]

III. State eLearning Initiatives

IV. Next Steps

V. Work Group Members

II. eLearning Evaluation Lens

Participation in the Program Evaluation Work Group was an intense, but rewarding experience. Descriptions of the work group process at the National Leadership Institute 2004 (NLI) indicate that it was a time for “synergy of discussion,” “one-stop shopping for quality resources,” “breaking out of state isolation,” “refining thinking about program evaluation,” and “realizing that the sum is greater than all of the parts.”

The Program Evaluation Work Group aspired to address core goals outlined in the No Child Left Behind, Title II, Part D, Enhancing Education Through Technology (NCLB II D) program during the State Educational Technology Directors Association’s (SETDA) 2004 meeting by determining how to support and guide states in conducting successful eLearning evaluations. Program evaluation is a vital component of eLearning initiatives with the ultimate aim of improving student learning and overall quality of education through technology.

Aware that evaluation is not research, and that the purpose of an evaluation is to find out how a program is working and to make decisions about improvement and capacity-building, the Program Evaluation Work Group intended for the following tool to be used as a lens or framework by states when describing and defining the components of a successful eLearning program. States can use this lens when:

1. developing RFPs;
2. working with school district grant awardees;
3. providing projects with criteria to consider in hiring outside evaluators; and
4. identifying and collecting project data elements that could be aggregated and used in a statewide evaluation.

^ Return to Top

eLearning Evaluation Lens

Program evaluation is often seen as a measurement tool and nothing more. However, carefully planned and implemented program evaluation efforts actually contribute to student achievement and are closely aligned with instructional practices in individual classrooms.

As we move into the age of ubiquitous computing, technology use must be transformed into a support that is inherent in school instructional practice. Vital to the scientifically-based-research approach is the evaluation of technology programs that have been infused into districts and classrooms. We have moved beyond simple implementation and must now establish proven track records for success and the improvement of student achievement through technology.

As states consider eLearning program evaluations, it is helpful to have a starting point from which they can work and think practically about questions to be asked and issues to consider. The purpose of the eLearning Evaluation Lens is to help states understand program evaluation and assist them in their various approaches to evaluation. The framework takes the user into an early visioning stage, sets the tone for evaluation, offers points and goals on which to reflect, provides case studies and examples from which to learn and ultimately leads to readiness for state eLearning program evaluation action.

The Program Evaluation Work Group felt that it was important to create a list of guiding questions for states to consider before engaging in program evaluation and, in turn, to refer back to the eLearning Evaluation Lens:

• What is the purpose of the evaluation?
• How will results of the evaluation be used?
• Who is the audience for the evaluation?
• By what criteria will the success of the program be judged?
• What forms of evidence are acceptable given both the purpose and audience?
• What does recent research say about successful e-learning programs?

After careful consideration of the above guiding questions for conducting eLearning program evaluations, states may move forward and explore the eLearning Evaluation Lens provided below.

The Lens is presented in two sections for ease of the reader. Both sections are separated into the three identified eLearning categories:

1. educator-centered;
2. student-centered; and
3. resource-centered.

The first provides goals, indicators, measures, pitfalls, and models for states to review. The second section supplies users with crucial information on current state eLearning initiatives.

Download eLearning Evaluation Lens [181 KB]

To download this page, this chapter or the entire toolkit, go to "Downloads" section located in the upper-right side on each page of each chapter.

^ Return to Top

#1: eLearning Evaluation Lens - Program Evaluation
[181 KB]

#2: Program Evaluation Summary & Tools
[201 KB]

Download the entire toolkit & more.