evaluation: what next? dr fiona fylan director of brainbox research & leader in psychology,...

Evaluation: what next?

Dr Fiona FylanDirector of Brainbox Research

& Leader in Psychology, Leeds Metropolitan University

Beth FylanSocial Researcher

& Director of Brainbox Research

+

Evaluation and beyondFiona Fylan and Beth Fylan

+What we’re going to do

+Evaluation refresher

+Brainbox queries

We need to do an evaluation

Our funders want to know what

we’ve achieved

How can we tell what effect we’re having?

My director wants to know whether we’ve made a

difference

My staff budget is under threat. Help me defend

what my team does

We have to bid for our budget next year and we need to prove

what we do is working

How can we do our own evaluation?

I need to write a questionnaire

+Why evaluate?

intervention

+Evaluate to improve

+Evaluation types

+Audit / monitoringA form of process evaluation

+What evaluation questions should you ask?

+Defining your objectives

+What are the evaluation research questions?

+What different designs are there?

+Evaluation pitfalls

First things first: no clear intervention objectives.

No outcome measures

Process versus outcome – happy sheets

Asking the wrong questions – Did it meet your expectations?

How to interpret the results. Does the drop in perceived skill levels result from increased knowledge of what is required?

+

Your evaluation questions

+Some in reserve

How many people should I include in my evaluation?

Who should I include?

What questions

should I ask?

How should I analyse my

data?

What about E-valu-it?

+What next?

Outputs and stakeholders

+Report types

Written Post-fieldwork summary Full written report

Spoken Informal feedback / verbal debrief

(qualitative)

Full presentation of the research

Immediate Considered

+Basic logic models

+Recommendations

+Revisit your intervention

+Create some headlines

+Mapping and engaging your stakeholders

+

INFORM (low)

CONSULT (mid)

INVOLVE (mid)

PARTNER (high) Joint working group, problem solving team.

Seminars, workshops, project meetings.

Consultation documents, feedback forms.

Presentations, press releases, newsletters, twitter.

Power

High

Low High

Interest

Stakeholder Category Why are they interested/What do we want?

Level Status Actions

1. Fleet managers

Referrer Access to drivers

Partner Manage closely

Short report and action group

2

3

4

5

6

7

+Any questions? Any elephants?