The Needle in the Haystack The role of external evaluations in identifying and promoting good practices

within NGOs

Saul GuerreroSenior Evaluations, Learning & Accountability Advisor

Action Against Hunger (ACF)

A simple story that may be of interest…

First, because organisations may relate to the issues

Second, because our experiences may help others in tackling these issues

Third, because this is the story of what happens when a consultant/evaluator infiltrates an organisation

In 2010

No finalised/approved policy or guidelines for evaluations

The impact of individual evaluations on organisational performance and practice was inconsistent

Evaluations were something that field teams largely saw as someone else’s desire to “audit” a programme

But then an opportunity…

ACF International Evaluation Policy & Guideline

A framework that could help us change ideas about, and contributions of, external evaluations

ACF Evaluation Policy &

Guidelines

Terms of Reference

DAC Criteria

Best Practices

Diversity

Predictability

DAC Criteria

Criteria Rating(1 low, 5 high)

Rationale

1 2 3 4 5

Impact

Sustainability

Coherence

Coverage

Relevance/Appropriateness

Effectiveness

Efficiency

But why the need to quantify?

For field programmes

For Headquarters

Track changes from year to year (or evaluation to evaluation)

Track changes from programme to programme

BUT everyone understood that the numbers meant only so much. What matters is their comparative value, and

the rationale behind them.

Today we can use this to show that programmes that are relevant/appropriate, for example, are those that are good

at recognising and adapting to changing needs

Link it to actual programmes

Promote it as an evidence-based value

Best Practices

Not just a “feel good” exercise

A way of using an often underutilised dimension of working with external evaluators: their experience as an actual filter

to distil the “special stuff” from a programme

“The evaluation is expected to provide one (1) key example of Best Practice from the project/programme. This example should relate to the technical area of intervention, either in

terms of processes or systems, and should be potentially applicable to other contexts where ACF operates”

What is this “Best Practice”?

What makes it good/different/stand out?

How can we do more of it?

Not just an external idea/process

Giving external evaluators a designated role in triggering a much wider learning process

Best Practice

Identified by evaluations

shared with

HQsField

Techies

resulting in

Expanded Best Practice

What is it?How does it work?

How to move forward?

goes into Learning Review

Best Practices from all 46 missions

feeds into

Technical Publications

Technical/PolicyDebate

which influence

Field Practices

which provide

It is not perfect, and there is much to do still…

But after two years, we are starting to see this process come to life

So, what have we learned from it all?

Sort out the haystack firstOrganisations can change perceptions about the

role/usefulness of evaluations – but you need to make it your goal, and you need to work top-down and bottom-up

Enlist help in searchingYou can cater for the learning interests of programmes, and the learning needs of an organisation– but evaluations need

to strike a balance between diversity and predictability of information

Do something with the needle!External evaluations can be given a key role in triggering

wider organisational learning – but its must be seen as part of a larger, more participatory process that involves

different people at different levels