I. Chapter Summary

II. Core Beliefs about the Assessment of Student Achievement [Framework]

III. Evaluation of Online Learning [Policy Statement & A Buyer’s Guide]

IV. Assessment of the 8th Grade Technology Literacy Requirements [Policy Statement]

V. Pros and Cons of Paths To 8th Grade Technology Literacy [Assessment Charts]

VI. State of States

VII. Resources & Readings

VIII. Next Steps

IX. Work Group Members

I. Chapter Summary

• Objectives
• Key Questions
• Assessment of Student Achievement in Core content areas
• NLI Work Group Process
• SETDA Tools Developed to Assist States
• Recommended Next Steps

Objectives

To demonstrate how the informed use of data and technology can improve student learning, to identify the importance of rich formative and summative assessments to enhance the teaching and learning process, and to determine ways to assess twenty-first century skills within the eLearning context.

Key Questions

The Evaluation of Student Achievement Work Group focused on three areas with the following guiding questions for each section:

Evaluation of Online Learning

1. How should technology be used to evaluate online student learning?
   
   
2. What new metrics are required to capture the full range of what students are learning online?
   
   
3. What makes a successful online course? What type of student is most successful in an online environment?
   
   
4. By what metrics is online learning being evaluated?
   
   
5. What standards for eLearning are in use, under development or needed?
   

8th Grade Technology Literacy

1. How should technology be used to assess technology literacy (i.e. performance assessment, portfolios, peer reviews)?
   
   
2. Should technology literacy be measured within state and local tests for academic content standards, or through separate assessments? What are the implications of doing so?
   
   
3. What new metrics are required to capture the full range of what students’ technology literacy?
   
   
4. By what metrics is technology literacy being evaluated?
   
   
5. What assessments are currently in use, available or under development?

Assessment of Student Achievement in Core content areas

1. How should technology be used to evaluate student achievement in core content areas? Does the quality of the test dictate the extent to which we understand what students know and are able to do? Can technology help us address those issues?
   
   
2. How can districts use technology for local assessments (benchmark testing) that align with state tests and provide benchmarks along the way for teachers to mark their progress, and differentiate or adjust teaching?
   
   
3. How does technology contribute to or affect student achievement across content areas and grade bands? How do states “make the case” that technology improves student learning?
   
   
4. By what metrics is AYP being evaluated? Do some state standards include what the cognitive sciences say about how people learn (e.g., importance of: dealing with students’ prior misconceptions, metacognition, and learning for understanding)?
   
   
5. What new metrics are required to capture the full range of what students are learning online or through electronic media?

^ Return to Top

NLI Work Group Process

The Evaluation of Student Achievement Work Group broke into three distinct subgroups to create their deliverables:

• Evaluation of Online Learning
• Assessment of the 8th Grade Technology Literacy Requirements
• Assessment of Student Achievement in Core Content Areas

Once the participants had selected the subtopic of most interest to them, each subgroup first focused on producing a policy brief related to their topic.

After crafting a policy brief on the importance of mechanisms to evaluate the effectiveness of online courses and programs, the Evaluation Online Learning subgroup created a Buyer’s Guide for online programs and courses.

This Guide is aimed at helping states determine the quality of their programs and courses—whether the programs are commercially developed or developed in-house. The Buyer’s Guide can serve as the first step towards filling a gap in the information that is currently available on the relative quality and success of various online courses and programs.

The Evaluating Student Achievement in Core Content Areas subgroup produced a policy statement laying out a set of core beliefs about the evaluation of student achievement that places students and their learning at the center of the evaluation processes.

This subgroup also created a visual representation of an Assessment for Learning framework, which can be used to operationalize these core beliefs as a framework for assessing student achievement.

The Assessment of 8th Grade Technology Literacy Requirements subgroup created several deliverables. This subgroup’s policy statement addresses the issue of reporting on the Consolidated State Performance Report, and makes the case for including the 8th Grade Technology Literacy Requirement, but with specific guidelines and room for flexibility clearly spelled out ahead of time.

The subgroup’s other deliverables include matrices of the pros and cons of embedded versus separate assessments and a “state of the states” matrix that outlines how each state is currently dealing with assessing the 8th Grade Technology Literacy Requirements. This data was collected by soliciting information from their peers at the NLI.

^ Return to Top

SETDA Tools Developed to Assist States

• Integral Role of Technology in Assessing Student Achievement & Assessment for Learning Framework
  The policy document outlines core beliefs about the processes of evaluating student achievement in core content areas, and also visually diagrams an Assessment for Learning framework that can be used to operationalize the core beliefs about assessing student achievement.
  
  
• Evaluating Online Learning Policy Document and a Buyer’s Guide for Online Programs and Courses
  This policy document outlines the importance of evaluating the effectiveness of online courses and programs, and the Buyer’s Guide is a tool for sorting out quality online courses and programs.
  
  
• Assessment of the 8th Grade Technology Literacy Requirements of No Child Left Behind Act of 2001 (NCLB)
  This policy document addresses the absence of the 8th Grade Technology Literacy Statutory Requirement in the Consolidated State Performance Report.
  
  
• Assessment Consideration Chart
  This chart provides some of the pros and cons of embedded versus separate assessments, defines the available types of assessments, and organizes these assessment types into a continuum.
  
  
• State of the States Matrix
  This chart catalogs the “state of the states” in terms of meeting the 8th grade technology literacy requirement, and includes states’ definitions, policies, assessment processes, reporting strategies and supporting URLs.

^ Return to Top

Recommended Next Steps

1. Evaluation of Online Learning

• Create an end-of-course assessment and quality of online teaching tool;
• Create mechanisms that incorporate student voices into the evaluation of online courses; and
• Develop a comprehensive list of what criteria constitute a well-designed course and a rubric for evaluating the interactive quality of student learning.

2. Evaluation of Student Achievement in Core Content Areas

• Identify ways to build the capacity of administrators who do not have the knowledge of how to incorporate technology into assessment mechanisms; and
• Identify successful methods of creating classroom teacher buy-in to the core beliefs.

3. Assessment of the 8th Grade Technology Literacy Requirements

• Gain approval of the recommendation from SETDA’s federal policy group that states begin reporting on their progress towards meeting the 8th Grade Technology Literacy Requirement in the Consolidated State Performance Report;
• Identify ways to build the capacity of administrators to support the teaching and assessment of the 8th Grade Technology Literacy Requirements; and
• Complete and analyze the state of the States matrix.

To download this page, this chapter or the entire toolkit, go to "Downloads" section located in the upper-right side on each page of each chapter.

^ Return to Top

#1:
Chapter Summary - Evaluating Student Achievement
[86 KB]

#2:
Evaluation of Student Achievement Summary & Tools
[774 KB]

Download the entire toolkit & more.