evaluation: what next? dr fiona fylan director of brainbox research & leader in psychology,...
TRANSCRIPT
Evaluation: what next?
Dr Fiona FylanDirector of Brainbox Research
& Leader in Psychology, Leeds Metropolitan University
Beth FylanSocial Researcher
& Director of Brainbox Research
+
Evaluation and beyondFiona Fylan and Beth Fylan
+What we’re going to do
+Evaluation refresher
+Brainbox queries
We need to do an evaluation
Our funders want to know what
we’ve achieved
How can we tell what effect we’re having?
My director wants to know whether we’ve made a
difference
My staff budget is under threat. Help me defend
what my team does
We have to bid for our budget next year and we need to prove
what we do is working
How can we do our own evaluation?
I need to write a questionnaire
+Why evaluate?
intervention
+Evaluate to improve
+Evaluation types
+Audit / monitoringA form of process evaluation
+What evaluation questions should you ask?
+Defining your objectives
+What are the evaluation research questions?
+What different designs are there?
+Evaluation pitfalls
First things first: no clear intervention objectives.
No outcome measures
Process versus outcome – happy sheets
Asking the wrong questions – Did it meet your expectations?
How to interpret the results. Does the drop in perceived skill levels result from increased knowledge of what is required?
+
Your evaluation questions
+Some in reserve
How many people should I include in my evaluation?
Who should I include?
What questions
should I ask?
How should I analyse my
data?
What about E-valu-it?
+What next?
Outputs and stakeholders
+Report types
Written Post-fieldwork summary Full written report
Spoken Informal feedback / verbal debrief
(qualitative)
Full presentation of the research
Immediate Considered
+Basic logic models
+Recommendations
+Revisit your intervention
+Create some headlines
+Mapping and engaging your stakeholders
+
INFORM (low)
CONSULT (mid)
INVOLVE (mid)
PARTNER (high) Joint working group, problem solving team.
Seminars, workshops, project meetings.
Consultation documents, feedback forms.
Presentations, press releases, newsletters, twitter.
Power
High
Low High
Interest
Stakeholder Category Why are they interested/What do we want?
Level Status Actions
1. Fleet managers
Referrer Access to drivers
Partner Manage closely
Short report and action group
2
3
4
5
6
7
+Any questions? Any elephants?