PETI Evaluating Educational Technology Effectiveness

In the Fall of 2002, the Metiri Group was commissioned by SETDA to develop an assessment framework for effective technology use in schools. That framework, available on the SETDA site 2002 Toolkit, is based on a set of key questions aligned to indicators and data elements. (Note: A subset of the identified data elements could be used to assess states’ progress with NCLB, EETT.)

The Common Data Elements Committee recommends a state-by-state data collection methodology that includes a combination of surveys: a single point of contact at schools, districts, and the state, combined with a stratified random sampling of schools/classrooms. The workgroup at the NLI 2003 strongly concurred with the Committee’s recommendation for site visitations at the classroom/school level in order to fully assess the goals of NCLB, Title II, Part D.

Why are the profiling tools important?
The tools were developed to assist states in making sound, data-driven decisions about educational technology. Metiri Group, which developed these resources under the advisement of the SETDA Common Data Elements committee, has worked over the last year with five states to pilot and revise the instruments and protocols based on data analysis to ensure reliability and validity. Used as outlined here, PETI offers valid, reliable data collection instruments and methodologies that answer the questions in the SETDA framework. Used over time, these tools can provide valuable baseline and trend data for accountability purposes; see Suite of Tools.

How might states, districts, schools, and researchers use these tools?
States, districts, or researchers with sophisticated profiling tools already in place may want to analyze the toolset in light of SETDA’s new resources, updating where necessary.

States, districts, or researchers currently using surveys only to determine their school districts’ technology readiness may want to a new element–site visitation data. Although such visits are not required for valid and reliable profiles, they do seem to increase the credibility of resultant reports to policy makers.

States, districts, or researchers currently using hardware/software inventories only may want to continue collecting such data, adding the SETDA/Metiri surveys and/or site visitations to get more complete baselines and trend patterns on their school districts’ technology readiness over time.

States, districts, or researchers currently not collecting data state wide or school system wide may want to adopt the SETDA/Metiri tools, using only the three surveys, or a combination of the surveys and the site visitation protocols.

What methodology is recommended for data collection and analysis using these tools?
While some states will want to survey every school district and every school building, others will want to reduce the burden of data collection by surveying representative samples from those populations. Some states will include site visitations (in a sampling of schools representative of the state), while others will rely on the surveys only to determine their school districts’ technology readiness.

The SETDA/Metiri Assessment Framework is based on a set of questions that provide important data for tracking states’ progress with technology components of NCLB (Title II, Part D). The full framework, which includes indicators of success and common data elements, can be accessed at: SETDA/Metiri.

  • Impact on Learners: 
    • Improvement of academic achievement through effective technology use
    • Assurance that students acquire 21st century skills through effective technology use in the context of high standards and high quality learning
    • Engagement of students in learning through effective technology use
  • Effective Practice: Is the vision being translated into practice through learning environments characterized by powerful, research-based strategies that effectively use technologies?
  • Educator Proficiency: Are educators proficient in implementing, assessing and supporting a variety of effective practices for teaching and learning?
  • Robust Access, Anywhere, Anytime: Do students and school staff have robust access to technology-anytime, anywhere-to support effective designs for teaching and learning?
  • Digital Equity : Is the digital divide being addressed through resources and strategies that ensure that all students are engaging in an educational program aligned to the vision?
  •  Vision, Systems and Leadership: Has the education system reengineered itself into a high-performance learning organization?

This framework was grounded in the work of NCREL and Metiri Group on EnGauge – A Framework for Effective Technology Use initially published in 2000.

COPYRIGHT. While SETDA retains rights to the toolsets, broad use is encouraged. However, STEDA respectfully requests that such use acknowledge the developers of these tools: SETDA and Metiri Group.

NOTE: SETDA and Metiri recommend that the suite of tools be used as a system to maintain validity and reliability.

© 2024 SETDA, All Rights Reserved