I. Chapter Summary

II. Core Beliefs about the Assessment of Student Achievement [Framework]

III. Evaluation of Online Learning [Policy Statement & A Buyer’s Guide]

IV. Assessment of the 8th Grade Technology Literacy Requirements [Policy Statement]

V. Pros and Cons of Paths To 8th Grade Technology Literacy [Assessment Charts]

VI. State of States

VII. Resources & Readings

VIII. Next Steps

IX. Work Group Members

III. Evaluation of Online Learning

• Policy Statement
• A Buyer’s Guide to Evaluate Online Programs and Courses

Policy Statement

While many states do not currently have statewide programs for online learning, state legislatures and boards of education are charging state educational agencies (SEAs) and local educational agencies (LEAs) with the creation of policies, programs or courses in this area. For LEAs, online learning opportunities can provide courses to students that are not available locally, and can provide alternative methods for all students to access high quality curriculum taught by highly qualified teachers. Furthermore, online learning has the potential to support schools in their efforts to meet Adequate Yearly Progress (AYP) under No Child Left Behind (NCLB) requirements.

The evaluation of online programs and courses helps ensure that programs achieve and maintain a high quality and reach intended goals. To this end, there is a growing body of literature that addresses the essential components of online learning programs and courses. However, a universal set of quality guidelines is needed so that program and course developers (educators and vendors) can create products that will support success for every student who takes an online course or enrolls in an online program. The NLI 2004 Curriculum Development Work Group began to address this need.

The success of online programs and courses hinges on the understanding that one must evaluate the implementation process before the impact of the program and the intended outcomes can be evaluated. This requires ongoing, purposeful, planned program evaluation, which must occur before assessment of the impact on students can be determined. Furthermore, any evaluation instruments should be based on the Standards for Program Evaluation*,particularly those standards related to feasibility and utility.

In the interest of developing a set of standards that measures the quality of online programs and courses, and to aid the assessment of such programs, we have developed a “Buyer’s Guide” to serve as a first-step in this process of evaluating online programs and courses. The goal of the Buyer’s Guide is to provide assistance in the development and selection of successful online programs and courses. Developing this Buyer’s Guide, as well as other evaluation mechanisms, is critical to the success of online programs and courses. If online courses and programs do not fit the needs of students or schools, then the concept of online learning is at risk of being tried once and abandoned without its potential being realized. We hope that the Buyer’s Guide will help state and district leaders in the ongoing process of improving online courses and programs.

* The Joint Committee on Standards for Educational Evaluation (1994). The Program Evaluation Standards. Thousand Oaks, CA: Sage Publications, Inc.

^ Return to Top

A Buyer’s Guide to Evaluate Online Programs and Courses

This Buyer’s Guide is aimed at providing an easy first-step tool to help SEAs and LEAs sort out quality courses and programs—whether homegrown or commercially developed. Unfortunately, in the absence of a Buyer’s Guide to help assess online courses and programs, states and districts are often forced to make uniformed decisions, or decisions that are based on a product’s marketing or cost rather than on its quality.

There is a growing body of literature on the effectiveness of online courses and programs, but it needs to be distilled into a form that state directors, school officials, and district leaders can use. An evaluation tool, such as a Buyer’s Guide, should have two components: evaluation of online learning programs (at the state or district level); and an evaluation of online courses as shown through assessments of student performance.

SETDA’s Profiling Educational Technology Integration (PETI) criteria and framework can be used to analyze the quality of online learning courses and programs. * Each number in the document (i.e. S1-1) corresponds to a PETI Guiding Question.

* For more information, see SETDA’s PETI framework.

A Buyer’s Guide to Evaluate Online Programs and Courses [90 KB]

^ Return to Top

#1:
Evaluation of Online Learning
[86 KB]

#2:
Evaluation of Student Achievement Summary & Tools
[774 KB]

#3:
Entire Toolkit 2005
[-- MB]