nh responds evaluation component
DESCRIPTION
NH RESPONDS Evaluation Component. Pat Mueller David Merves October 6, 2008. Why We Collect Data/Evaluate?. Somebody said you had to Inform instruction School improvement Local, state, and federal accountability Public information Choose/set policy Marketing - PowerPoint PPT PresentationTRANSCRIPT
Pat MuellerDavid Merves
October 6, 2008
NH RESPONDSEvaluation Component
Why We Collect Data/Evaluate?
Somebody said you had to
Inform instruction
School improvement
Local, state, and federal accountability
Public information
Choose/set policy
Marketing
Because that’s what all the cool kids are doing…
State Personnel Development Improvement Grants (SIG/SPDGs)
SIG/SPDGs are measured against…
OSEP program performance measures
NH RESPONDS performance measures
SPDG Program Performance Measures
% of personnel receiving professional development (PD) on scientific-or evidence-based instructional practices.
% of projects that have implemented PD/training activities that are aligned with improvement strategies in the State Performance Plan (SPP).
% of PD/training activities provided that are based on scientific-or evidence-based instructional/behavioral practices.
SPDG Program Performance Measures
% of PD/ training activities that are sustained through on-going and comprehensive practices (e.g., mentoring, coaching).
% of SPDG projects that successfully replicate the use of scientifically based or evidence-based instructional/behavioral practice in schools
• SELECT THE CRITERIA AND STANDARDS
• DEVELOP LOGIC MODEL
• PREPARE AN EVALUATION PLAN
• COLLECT DATA
• ANALYZING DATA & UNDERSTANDING RESULTS
• COMMUNICATE THE FINDINGS
Steps for Conducting an Evaluation
1. Define the Criteria to Be Evaluated
Terminology Goals Long-Term Outcomes or Impact
Objectives Short-term & Intermediate Outcomes
Activities Outputs
Two Types of Evaluation Standards
Process/Formative: Assesses ongoing project activities
Begins at program implementation and continues throughout program
Is the program being delivered as planned?
Is the program progressing towards its goals and objectives?
Two Types of Evaluation Standards
Outcome/Summative: Assesses the program’s success and whether the program or initiative had an impact.
Compares the actual results to projected goals/objectives.
Typically used for decision making purposes
Important to look for unanticipated outcomes
2. Logic Models
A conceptual model that links an initiative’s goals and objectives, with expected outputs and/or outcomes.
Numerous types of logic models.
There are many other methods of illustrating the conceptual framework of an initiative.
3. Writing an Evaluation Plan
Components to include: Program goal/objectives
Evaluation questions
Performance indicators
Data collection procedures
Data analysis method
Person responsible
Timeline
4. Collecting Data
Process/formative data Amount and type of PD provided Satisfaction and utility of PD provided Products developed These data tend to be gathered by those providing
PDOutcome/Summative data
Reduced office discipline referrals Reduced suspensions/expulsions Improved reading scores These data tend to be collected from the LEA or SEA
NH RESPONDS Data Collection Tools
PD Activity Log completed by TA/PD providers
Minutes Leadership Team meetings
Workgroup meetings
School & District Improvement Team minutes/products
Surveys/Interviews/Focus groups Annual Participating Personnel Survey (March/April)
Workshop surveys
NH RESPONDS Data Collection Tools
Fidelity instruments (for PBIS & Literacy) Benchmarks of Quality
School-wide Evaluation Tool
Existing data Office Discipline Referrals
Suspension/expulsion data
Reading scores
5. Analyzing Data & Understanding Results
SAU/School-Level Analysis of student performance to improve
instruction (i.e. reading scores)
Analysis of school-level data to improve safety and/or climate (i.e. SET)
Project/Grant Level Analysis of formative data for program improvement
purposes
Aggregate analysis of all outcome data to describe impact of NH RESPONDS
6. Communicate the Findings (Reporting)
Provide on-going feedback to project management (what’s working/what’s not).
Provide on-going data to the NH Bureau related to completion of objectives and success of project efforts.
Provide annual report to the U.S. Department of Education related to completion of objectives and success of project efforts.
