Monitoring, Evaluation and LearningPreparing the foundations -

Theory of Change and Log frames: the DFID approach

Dr Caroline HoyCivil Society Department

Department of International Developmentc-hoy@dfid.gov.uk

1

Four Foundation Stones of MEL

1. Understand your programmea. What is the need for the programme?b. How will the programme respond to the identified need?c. Theory of change

2. Understand your MEL needs and demandsa. Learning and improvementb. Accountability and impactc. Respond to donorsd. Stakeholderse. Beneficiaries

3. Plan your MEL approacha.Structure and monitor your progress (log frame) b.Key evaluation questionsc.Methodsd.Analysis e.Learning and feedback

4. Manage your evaluationa. Roles and responsibilitiesb. Financec. Timeframed. Capacitye. Dissemination and communication

2

DFID Log framePROJECT NAMEIMPACT Impact Indicator 1 Baseline Milestone 1 Milestone 2 Target (March 20??)

Planned

Achieved

OUTCOME Outcome Indicator 1 Baseline Milestone 1 Milestone 2 Target (March 20??) Assumptions

Planned

Achieved

Outcome Indicator 2 Baseline Milestone 1 Milestone 2 Target (March 20??)

Planned

Achieved

Outcome Indicator 3 Baseline Milestone 1 Milestone 2 Target (March 20??)

Planned

Achieved

Outcome Indicator 4 Baseline Milestone 1 Milestone 2 Target (March 20??)

Planned

Achieved

DFID (£) Govt (£) Other (£) Total (£)

DFID (FTEs)

OUTPUT 1 Output Indicator 1.1 Baseline Milestone 1 Milestone 2 Target (March 20??) Assumptions

Planned

Achieved

Output Indicator 1.2 Baseline Milestone 1 Milestone 2 Target (March 20??)

Planned

Achieved

IMPACT WEIGHTING (%)

RISK RATING

DFID (£) Govt (£) Other (£) Total (£)

DFID (FTEs)

INPUTS (£) DFID SHARE (%)

INPUTS (HR)

INPUTS (HR)

Source

Source

Source

INPUTS (£) DFID SHARE (%)

Source

Source

Source

Development Project XXXXX

Source

3

Theory of Change definition

'Theory of change' is a process which applies critical thinking to the design,

implementation and evaluation of initiatives and programmes intended to support

change in their contexts.

The description of a sequence of events that is expected to lead to change

4

Key aspects of a theory of change

•Context, •The current state of the problem,•Long-term change,•Process/sequence of change,•Assumptions,•Evidence or logic base,•Diagram and narrative summary…

… and the voice of the beneficiary.

•But …

5

ContextSocial, political,

environmental …

Current state of the problem

Desired long term change

Sequence of change Assumptions Diagram and narrative summary

Theory of change process 1

6

ContextSocial, political,

environmental …

Current state of the problem

Desired long term change

Sequence of change Assumptions Diagram and narrative summary

Theory of change process 2Baseline Impact/

Outcomes

?Inputs, activities, output … or results chain

Information for log frame

7

Results Chain

• Funds, expertise, time, staff

• Activities, actions … (Education strategy, resourcing plans)

• Specific deliverable of the project and which provide conditions necessary for outcome(s) to be achieved (schools built)

• What will change/who will benefit (children receiving quality education to primary school level yr5)

• Overall goal to which the project will contribute (literacy levels)

Input Process Output Outcome Impact

Context and Assumptions

8

Theory of Change: Case study – Accountability Tanzania (AcT) 1

• www.accountability.or.tz

• Funded by DFID• Aim: to support citizens to make government

more transparent, accountable and responsive to citizens, by working with civil society organisations in Tanzania

• Provides funding and technical support to CSOs

9

Theory of Change: Case study – Accountability Tanzania (AcT) 2

Headline Theory of Change

‘Supporting civil society partners to implement context-specific strategic interventions will

enable them to influence positive change in the attitudes and behaviour of citizens, civil society

and government, making government as a whole more responsive and accountable.’

10

Longer ToC narrative – enabling identification of inputs, process, outputs and outcomes

•‘If civil society grantees are carefully selected and respond to individual support tailored to their programming and internal systems, they will be able to utilise grants to develop targeted strategic interventions which are sensitive to changes over time and in the broader political economy, as well as their geographic location, their sector, institutional mandate and values.

•And if grantees also commit to systematic learning individually and collectively the work they do will be more the effective.

•CSOs implementing programmes will engage in a range of knowledge and information generating and disseminating activities as well as developing the capacity of other stakeholders to articulate their roles and responsibilities.

•Some participatory activities build directly into citizen action and civil society strengthening, whereas others focus on influencing the behaviour of elected and appointed officials and of the judiciary – at local and national levels.

•Influencing activities can be formal or informal, inside track or outside track, and CSOs become more adept at selecting which is going to be most effective under what circumstances.

•The result of the behaviour changes on the part of key stakeholders is the impact level of the programme: ‘Increased responsiveness and accountability of government through a strengthened civil society.’

•The super impact of the programme is the increasing ability of Tanzanian to claim and exercise their rights as citizens’ (Achievement of MDGs 3&8 gender equality/women’s empowerment and partnership in development).

Theory of Change: Case study – Accountability Tanzania (AcT) 3

INPUTS, PROCESS

PROCESS

PROCESS OUTPUTS

OUTPUTS

IMPACT

OUTCOME

MDGs achieved

Increased accountability and responsiveness of Government

Behaviour changeCitizens Government

Influence on behaviour of elected representatives,

govt. offices, judiciary

Citizen action

Civil society

strengthened

Information disseminated Capacity built

Knowledge generated

Targeted strategic interventionsIndividual and shared learning

Selection, tailored support, grantsInputs

Process

Process outputs

Outputs

Outcome

Impact

12

Exercise: Creating a Theory of Change

1. What issue/problem are you trying to address?

2. What are you trying to achieve?

3. What are you doing?

13

Exercise: Creating a Theory of Change3. Break down your theory of change (and create a results chain)

Inputs

Process

Process Outputs

Outputs

Outcome

Impact

14

Assumptions

• Challenges to logic• If not identified can

undermine a programme e.g. nutrition in Bangladesh

• May result in fundamental alterations to a project or programme

15

1

2

Assumptions

3

4

16

Assumptions 1

INPUTS

‘If civil society grantees are carefully selected and respond to individual support tailored to their programming and internal systems, they will be able to utilise

grants

ASSUMPTIONS: INPUTS TO PROCESS

•AcT has a successful selection process that can identify organisations committed to change rather than administering money with a governance spin•AcT has skills and judgement to provide support, manage risk and the portfolio•CSOs have sector and area specific knowledge and understanding•CSO can develop familiarity with, and confidence in, working in changing political economy and develop to work with this

1

17

Assumptions 2PROCESSES

to develop targeted strategic interventions which are sensitive to changes over time and in the broader political economy, as well as their geographic location, their sector, institutional mandate and values. And if grantees also commit to systematic learning individually and collectively the work they do will be more

the effective.

ASSUMPTIONS: PROCESSES TO OUTPUTS

•Systematic learning enables CSOs to grow and move beyond ‘business as usual’; copycat approaches and ‘chasing the money’•CSOs become aware of the positive and negative lessons developed by others•CSOs monitor their own effectiveness and make changes as appropriate•CSOs document and embed learning•CSOs maintain ethics and integrity

2

18

Assumptions 3

PROCESSESCSOs implementing programmes will engage in a range of knowledge and

information generating and disseminating activities as well as developing the capacity of other stakeholders to articulate their roles and responsibilities.

ASSUMPTIONS: PROCESS OUTPUTS TO OUTPUTS

•Citizens are stimulated to respond to knowledge and information•Citizens see the value of taking action on information, knowledge and participation in capacity building•Participatory processes are empowering and stimulate action•Citizens overcome fear and apathy and stimulate others to join in•Decision makers recognise that they will not retain power unless they respond to their citizens•Decision makers are open to citizen and civil society action

3

19

Assumptions 4OUTPUTS

Some participatory activities build directly into citizen action and civil society strengthening, whereas others focus on influencing the behaviour of elected

and appointed officials and of the judiciary – at local and national levels.

Influencing activities can be formal or informal, inside track or outside track, and CSOs become more adept at selecting which is going to be most

effective under what circumstances.

ASSUMPTIONS: OUTCOME TO IMPACT

•Individual elected representatives, appointed officials and member of the judiciary are able to influence the politics and systems that frame their actions•Legislation, state systems and official processes are open to change

4

20

Exercise: Thinking about assumptions

1. What assumptions might you be making about your project or programme?

2. To which level of your results chain/ logic model do they apply?

3. What are the implications for your project or programme? Can you do something about these or simply acknowledge issues through e.g. risk planning?

21

DFID Log Frames 1

• Allows harmonised reporting across DFID• Promotes stakeholder consensus• Summarises and communicates unambiguously• Allows comparison of planned and actual results• Also includes indicators and milestones. Indicators

are performance measures which tell us what we are going to measure, not what we are going to achieve

• Importance of baselines• Importance of disaggregation of information e.g.

by gender

22

DFID Log Frames 2

• Impact – not intended to be achieved by the project – the higher level situation that the project will contribute towards achieving.

• Targets should be Specific, Measureable, Achievable, Relevant and Time-bound

• Source – the list of information you will need to demonstrate what has been accomplished

• Impact weighting – a percentage for the contribution each output is likely to make to the achievement of the overall impact

23

Useful Links

World Bank Impact Evaluation Toolkit: http://web.worldbank.org/

Http://go.worldbank.org/IT69C5OGL0

DFID Theory of Change:

http://www.dfid.gov.uk/r4d/Output/190843/Default.aspx

Theory of Change Community:

https://www.theoryofchange.org/

Kellogg Foundation Handbook:

http://www.wkkf.org/knowledge-center/resources/2010/W-K-Kellogg-Foundation-Evaluation-Handbook.aspx

24