Evaluation: WVDE Innovation Zone and Drop-Out Grant

Andu MeharieOffice of Research

Evaluation• It involves looking at your program and

asking– Is what we’re doing working?– How do we know it’s working?– Under what conditions does it work best?

• It is part of what you put in place beforehand to help you run your programs. – It should be about looking ahead not backwards.

• It is part of the continuum of planning (rolling evaluation).

EvaluationIt is a systematic process of asking questions, collecting information, and using the answers to

• Measure progress and identify areas of improvement.• Set realistic goals and fine tune strategic program

decisions.• Identify training and technical assistance needs.• Be accountable and be credible to your constituents.• Motivate by providing documentation of your

achievements.• Guide budget and resource allocation.• Generate support for programs.• Be an efficient organization that works toward its

mission.

Evaluation• If (process) or then (outcome), which to

evaluate?Process evaluation–How are resources allocated and used for activities?–How is the action plan implemented (fidelity), who participated, and what did they think about the activity? –What obstacles were encountered?–How are preliminary evaluation findings used to improve implementation of activities?Outcome evaluation–To what extent has the program achieved its desired results?

• Is a control group possible?• To effect change (Theory of change logic model)

Evaluation• Are objectives S.M.A.R.T. with respect to the

overarching goal/s of your project. – Specific?

• What exactly do you want to accomplish (desired outcome)?

– Measurable?• How will you assess your progress and accomplishments?

– Attainable?• What types of resources do you need to attain your

objective?

– Relevant?• Does it align with other objectives and overarching goal?

– Time-bound?• When do you expect to see some outcomes?

Identifying Outcomes of your Objectives

• Take 5 minutes at your table:– Choose one objective from your grant

application. Make sure to choose an Objective you think is S.M.A.R.T.

– With your team, answer the question: “What are one or more of the potential outcomes that would show your program has been successful in accomplishing this objective?”

Activity

Take 5 minutes to Identify one or more outcomes for an objective listed in your grant application.

Evaluation

• Is your outcome measurable? If your outcomes are measurable, you should be able to establish concrete criteria for gauging progress toward the attainment of each.

Activity

Take 5 minutes to Identify one or more measures/data sources for the outcome(s) you identified previously. Consider:

1.How will you know when the outcome is accomplished? 2.What indicators will you look for to measure progress and success? 3.Are data to measure your progress and success readily available? Or do you need to develop new measures, e.g., surveys, focus groups, etc…?

Evaluation

• Is your outcome time-bound? Outcomes should have starting points, ending points, and fixed durations.

– When will you achieve this outcome?– When will you undertake activities to achieve

our outcome? – When can you expect to see some short-term

outcomes?

Take 5 minutes to Identify the Time Frame for each of the outcome(s) you identified previously. Think about time lines for the following:

1.Instrument development (if applicable)2.Data Collection3.Data Analysis4.Reporting

Activity

1. Choose another of the objectives listed on your grant application.

2. Using the worksheet provided, identify an outcome, data sources; assign responsible parties for designing new measures (if necessary), data collection, data analysis, and reporting; and define a timeline.

Evaluation

Objective OutcomeMeasure/

Data Source

Responsible Person (s)

Evaluation Timeline

Instrument development

Data collection

Data analysis

Reporting

Evaluation

• Put evaluation on your upcoming meeting agendas. Always take some time to talk about where you are in your project, what data you have collected, what it is telling you and what you still need to know.

Thank You

Andu MeharieCoordinator – Research and Evaluation

WVDE Office of Researchhttp://wvde.state.wv.us/research

ph: 304.558.2546fax: 304.558.1613

ameharie@access.k12.wv.us