key performance measures, evaluation plans, and work plan

Key Performance Measures, Evaluation Plans,

and Work Plan

Shelley Kuklish

Epidemiologist

Arizona Nutrition Network

Session Objectives

By the end of this session, participants will be able to:

1. Complete the Key Performance Measures, Evaluation Plans, and Work Plan Sections of the application

2. Write S.M.A.R.T. objectives

3. Write succinct action steps

4. Identify an evaluation plan for each objective

Application

• It is recommended that you:– Complete Work Plan (Appendix B) before

completing Key Performance Measures and Evaluation Plans on the “Project Overview” sheet

Housekeeping

• We will be working on a ‘sample’ work plan one step at a time.

• Goal is that you leave here with at least one objective, one action step and one evaluation plan for that objective.– Can be used as an example for writing the rest of your FY10

work plan

• During the practice, Network staff will walk around to help as necessary.

• Be polite and tolerant of other’s opinions. – The goal of this training is to teach you HOW to write your work

plan, not WHAT you are doing to teach SNAP-ED.

Work Plan

Writing Objectives

Goals vs. Objectives

• A GOAL is:– A broad statement of intent– Non-specific, non-measurable

– Example:• Increase the proportion of SNAP recipients and

eligibles aged two years and older who meet dietary recommendations for fruit and vegetable consumption.

Goals vs. Objectives

• An OBJECTIVE is:– A realistic target for the program– Answers: Who will do how much of what by

when?

– Example:• By the end of the class series, 50% of low income

3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

S.M.A.R.T. Objectives

• S.M.A.R.T. objectives are:– Specific

– Measurable

– Achievable

– Relevant

– Time Bound

• Specific • Measurable• Achievable• Relevant• Time Bound

• What are you going to do, with or for whom?– Focus on results, not

how they are achieved.

• Specific

• Measurable• Achievable• Relevant• Time Bound

• Is the objective measurable?– Quality– Resources– Quantity– Time

• Specific • Measurable

• Achievable• Relevant• Time Bound

• Can you achieve this objective? – Set objective after

considering the baseline

– Keep objectives within reach

• Is this objective realistic?– Can we do this?– Why are we doing

this?

• Specific • Measurable• Achievable

• Relevant• Time Bound

• Is this objective logical as it relates to SNAP-Ed goals?

• Specific • Measurable• Achievable• Relevant

• Time Bound

• What is the timeframe for this objective? – For the Network

purposes, the time frame generally is by September 30, 2010

Objectives Apply to 3 Areas

• Process (Operational)

• Impact (Knowledge)

• Outcome (Behaviors)

Process Objectives(Operational)

• Tells what you are doing and how you will do it– Tracks participants, interactions and activities– Example:

• By September 30, 2010 at least 5,000 Fun Food News will be distributed to SNAP-Ed participants at community events.

Impact Objectives(Knowledge)

• Tells how you will change attitudes and knowledge– Example:

• By the end of the class series, 80% of low income 3rd grade students participating in the BBB classes will increase their knowledge of activities that help build strong bones.

Outcome Objectives(Behavior)

• Tells what the long term implications of your program will be. – Example:

• By the end of the class series, 50% of low income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

Easy Way to Write a SMART Objective

• Answer the following question:– Who will do– How much– Of What– By When?

• Example:– By the end of the class series, 50% of low income 3rd

grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

Your Turn!Write An Objective Statement

• Make sure that it meets the SMART requirements– Specific – Measurable– Achievable– Relevant– Time Bound

• Answer the following questions:– Who will do– How much– Of What– By When?

By the end of the class series, 50% of low income 3rd grade students

participating in the FVMM classes will report an increase in fruit and vegetable consumption.

(FVMM survey baseline 48%)

By September 30, 2010, teach the importance of eating fruits and

vegetables through the FVMM classes to at least 500 low-income 3rd grade

students.

Writing Action Steps

Action Steps vs Objectives

• Action Step– Focus on the activities– Provide the ‘How’

• Objective– Focuses on the results– Provide the ‘What’

Action Steps

– Should work towards the objective.– Should provide how you will attain your

objective.– Be brief and succinct.

ObjectivesObjectives

Act

ion

Ste

ps

Act

ion

Ste

ps

Example:

Objective: – By the end of the class series, 50% of low

income 3rd grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

Action Step: – 1.Teach FVMM classes to low income 3rd

grade students in 18 schools.– 2. Conduct pre-/post-surveys

Your Turn!Write an Action Step

• Using the objective statement you wrote in section 1, write at least 1 action step for this objective.

• Remember:– Action steps should work towards the objective.– Should provide how you will attain your objective.– Be brief and succinct.

By the end of the class series, 50% of low income 3rd grade students

participating in the FVMM classes will report an increase in fruit and vegetable consumption.

(FVMM survey baseline 48%)

Coordinate with 18 schools to conduct FVMM classes.

Meet with teachers to schedule FVMM classes.

Teach FVMM classes to low income 3rd grade students in 18 schools.

Conduct pre/post-test evaluations.

By September 30, 2011, teach the importance of eating fruits and vegetables

through the FVMM classes to at least 500 low-income 3rd grade

students.

Teach FVMM classes to low income 3rd grade students in 18 schools.

Writing Evaluation Plans

Evaluation Measures

• Formative – Collects data for improvement of

program/processes

• Process – Evaluates that the program is being delivered

as planned

• Impact/Outcome – Measure change in knowledge, attitudes or

behaviors

Formative Evaluation

• Formative research is conducted before a program is designed and implemented, or while a program is being conducted.

• It helps you:– Decide on and describe your target audience, – Understand the needs of the audience and the factors

which influence their behavior– Helps you to design programs, services, and products

to address those needs.

Examples of Formative Evaluation

• Needs assessments

• Focus group discussions

• In-depth interviews

• Market assessments

• Teacher evaluations

• Pilot programs where content of nutrition education is tested and evaluated, etc.

Process Evaluation

• Process evaluation focuses on how the program operates, the services it delivers, and the function it carries out.

• It addresses whether the program was implemented and is providing services as intended.

Examples of Process Evaluation

• Tracking:– Number of materials distributed – Number of clients viewing a food

demonstration – Number of classes held – Number of participants reached

Outcome/Impact Evaluation

• An outcome or impact evaluation assess the change in knowledge or behaviors as a result of a particular program or intervention.

• The goal of an outcome or impact evaluation is to learn how effective the intervention was in changing the target populations' attitudes, awareness, and behaviors.

Examples of Outcome/Impact Evaluation

• Impact– Pre-/post-test showing an increase in

knowledge

• Outcome– Pre-/post-surveys showing an improvement in

self-reported behaviors

• Point-in-time surveys, tracked over time

Example:

• Objective: – By the end of the class series, 50% of low income 3rd

grade students participating in the FVMM classes will report an increase in fruit and vegetable consumption.

• Action Steps: – 1. Teach FVMM classes to low income 3rd grade

students in 18 schools.– 2. Conduct pre/post-test evaluations.

• Evaluation Plan:– Conduct pre- and post-surveys using FVMM

consumption survey. (Baseline 48%)

Baselines

• Clearly defined starting point from where implementation begins, improvement is measured, or comparison is made

• The first data point for your evaluation– Allows you to create achievable objectives by

providing a starting point.– Allows for tracking trends over time.

Evaluation Plans Should Tie Directly to The Objective

• Process objective=process evaluation– Example:

• Objective: Teach FVMM curriculum to 1,000 students in FY 2010• Evaluation: Track number of students participating via attendance

records

• Impact objective=impact evaluation– Example:

• Objective: 5th graders increasing knowledge of osteoporosis prevention

• Evaluation: Pre/Post-test measuring knowledge

• Outcome objective=outcome evaluation– Example:

• Objective: 3rd graders increasing fruit and vegetable consumption• Evaluation: Pre/Post-test measuring consumption

Your Turn!Write An Evaluation Plan

• Remember:– Evaluation plans tie to the OBJECTIVE– Process objective=process evaluation– Impact objective=impact evaluation– Outcome objective=outcome evaluation

By the end of the class series, 50% of low income 3rd grade students

participating in the FVMM classes will report an increase in fruit and vegetable consumption.

(FVMM survey baseline 48%)

Teach FVMM classes to low income 3rd grade students in 18 schools.

Conduct pre/post-test evaluations.Conduct pre- and post- surveys using FVMM consumption survey

Coordinate with 18 schools to conduct FVMM classes.

Meet with teachers to schedule FVMM classes.

Track number of students using attendance records

Teach FVMM classes to low income 3rd grade students in 18 schools.

By September 30, 2011, teach the importance of eating fruits and vegetables

through the FVMM classes to at least 500 low-income 3rd grade

students.

Things to Note

• You can include evaluation plans for each action step, however at least ONE evaluation plan must directly apply to the objective.

Project Overview

Appendix F

• Send in samples of each evaluation tool used for impact/outcome evaluations

• Stephanie will give you more information in her presentation.

Questions? Comments?

Thank You!Shelley Kuklish602-364-1736

Shelley.Kuklish@azdhs.gov