limpopo basin impact pathways workshop 15 – 17 november 2007 birchwood hotel, johannesburg boru...

Limpopo Basin Impact Pathways Workshop

15 – 17 November 2007

Birchwood Hotel, JohannesburgBoru Douthwaite, CPWF Impact Project

CIAT, Cali, Colombia

Impact Pathways Matter

Why make Impact Pathways explicit?

• People plan and implement projects (programs, countries …) on the basis of their change models - their implicit theories about how the world works, i.e., impact pathways

• If you can improve the impact pathways (IPs) you can improve the practice, making impact more likely

• IPs show a project’s rationale and networks– Help communicate what the project is doing

• More fundable

– Help with planning, including MTPs– Provide a basis for evaluation

• Starting point for evaluation is a good model of what you think will happen

• Provide information to support programmatic integration• Provides impact hypotheses for ex-post impact assessment

PIPA makes Impact Pathways explicit

It does so by developing two perspectives ….

1. A problem tree that shows a linear logic linking project outputs to project goal; and

2. Network maps that show the evolving relationships necessary to achieve the goal

Impact pathways – a more complete picture….

<-----the full picture---->

>---------------Actor-orientated perspective---------

>--

----

----

----

Ou

tco

me-

chai

n p

ersp

ecti

ve--

----

----

----

----

>

P

robl

em T

ree

Network maps

Foundations

• Adaptation of concepts from Program Evaluation– Renger and Titcomb (2002) – problem trees– Chen (2005) – program theory– Mayne (2004) - performance stories

• Innovation histories – Douthwaite and Ashby, 2005

• Social network analysis– Cross and Parker, 2004

Workshop Road Map

1. Problem Tree

2. Outputs

3. Vis ion

7. Outcom es logic m odel

4. "Now"network m ap

W hat the pro j ect w ill produce

W here pro j ect is go ing - Goal

Necessaryrelationshipsin placeto producethe OUTPUTS

Helps understand pro ject rationaleand w hat needs to change

5. "Future"network m apNecessaryrelationshipsto achievethe VI S I ON

Ide

nti

fyin

g a

lin

ea

r lo

gic

lin

kin

gp

roje

ct o

utp

uts

to

pro

ject

go

al

Inte

gra

tio

n o

fb

oth

vie

ws

6. ProjectS caling

S trategy

Ide

nti

fyin

g t

he

evo

lvin

gn

etw

ork

of

act

ors

ne

ed

ed

to a

chie

ve t

he

vis

ion

The outcomes the pro j ect w ill help achieve, how , and w ith w hom

Pro

du

cts

pro

du

ced

in

th

e W

ork

sho

pP

rod

uct

s p

rod

uce

da

fte

r th

e w

ork

sho

p

M i lestonesandprogressm arkers

Program - levelnetwork m aps

I m pactlogic m odel

Technologyextrapolationdom ain m aps

I m pactnarrative

Outputs produced after the workshop

Outcom etargets andm ilestones




I m pactnarrative

Use of PIPA OutputsUse of PIPA outputs PIPA outputs used

Ex-ante impact assessment

Essential:Impact logic model, output targets, impact narrativeOptional:Extrapolation domain analysis, scenario analysis

Impact Pathways Analysis Essential:Outcomes logic model, output targets and milestones, visionOptional:Impact logic model

Laying the foundation for ex-post impact assessment

Essential:Outcomes and impact logic modelsOptional:Impact pathways analysis (that updates project impact hypotheses)Extrapolation domain analysis

Expectations

D epressedLivelihoods

LimitedReservoir

Productivity

UnderusedFisheries

ProductionC apacity

Bad HarvestingStrategies

Uncooperativeattitudes of fishers

w ith respect tomanagement of

fisheries resources

Lack of know ledgeof options ofenhancementtechnologies

Processinglimitations

Lack ofaquaculture

activities

S tart Here

D eterminants

Lack of financialresources capacity

to implementscenarios for

improved fisheriesproduction

1st LEVEL

2nd LEVEL3rd LEVEL4th LEVEL W hy is th is problem happening?

W hy?W hy?W hy?

Problem

PN 34 I m proved fisheries productivity Problem Tree

Causal analysis / Problem tree

Refining and presenting your problem tree

• Adapt or develop anew your project problem tree for presentation (PowerPoint or cards)

• If cards, writing one problem per card– Use one color for problems the project will address

– Use another for other problems

• Modify, and add as you see fit – But don’t go into too much detail

• We’ll present the problem trees together with project visions in plenary

Exercise

1

Example of a Problem Tree developed during an IP workshop

Fishers cooperatein fisheries

management

K nowledge o foptions in

enhancementtechno logiestrans ferred

Financial resourcesfo r improved

fisheriesproductivity and

managementobtained

Respons ib leharves t ings trategies

Improvedprocess ingmethods

A quacultureact ivit ies

implemented

Optimized use o ffisheries

productioncapacity

Increasedreservo ir

productivity andmanagement

Enhancedlivelihoods

PN 34 ( I m proved F isheries Production) Ob jectives Tree

Goal1st LEVEL2nd LEVEL

3rd LEVEL4th LEVEL

D epres s edLivelihoods

LimitedReservo ir

Productivity

UnderusedFisheries

ProductionC apacity

Bad HarvestingStrategies

Uncooperativeattitudes of fishers

w ith respect tomanagement of

fisheries resources

Lack of know ledgeof options ofenhancementtechnologies

Processinglimitations

Lack ofaquaculture

activities

S tart Here

D eterminants

Lack of financialresources capacity

to implementscenarios for

improved fisheriesproduction

1st LEVEL

2nd LEVEL3rd LEVEL4th LEVEL W hy is th is problem happening?

W hy?W hy?W hy?

Problem

PN 34 I m proved fisheries productivity Problem Tree

PN 34 Improved Fisheries Management Problem and Outcome Trees

Main problem to Goal

Determinants to Products

Problems to Outcomes

Deriving Products/ Outputs from the Problem

Tree

• The determinants are the problems the project is directly addressing with its outputs

Hint: the use of the output solves the determinant

• Identify, write on cards and add to the problem tree the outputs, showing which determinants they correspond to

Exercise

2

Example from Ground Water Governance (PN42)

Some definitions Activity – what we’re doing inside the project

Hold IP Workshop Outputs – what we produce that other people make use of, that solve the

determinant Improved rice variety; priority setting publication

Determinant – determinants are the problems the project is directly addressing with its outputs

Next users – people and organizations who directly use the outputs End users – the people and organizations that the next users work with. Often the

end users are the ultimate beneficiaries (e.g., resource-poor farmers), but not always.

Politically-important actors– people and organizations whose support is needed for project success

Outcomes – usually the results of the use of outputs by others (often come in chains)

Promotion of rice variety by extension system Adoption of rice variety by farmers Higher rice yields Higher income More children sent to school

Level of influence of Project

High

CONTROL

Low

Research Output Output Outcome Impactactivity target

Level of influence of a project -change

3 years 10 - 30 years

Scaling Out and Scaling Up

• Scaling up - an institutional expansion, from adopters and their grassroots organizations to policy makers, donors, development institutions

• Scaling out - spread of a project outputs (i.e., a new technology, a new strategy, etc.) from farmer to farmer, community to community, within the same stakeholder groups

Develop a vision of project success 2 years after the end of

the project• Take 5 minutes to individually answer the question, then

develop common project vision by filling out Worksheet 1– You wake up 2 years after your project has finished. Your project

has been a success and is well on its way to achieving its goal. Describe what this success looks like to a journalist:

• What was the situation like before the project started (hint – look at the problem tree)

– What were the unmet needs and requirements of next users and end users?

• What are the next users now doing differently?

• How are project outputs disseminating (scaling out)?

• What political support is nurturing this spread (scaling up)?

• What are the end users doing differently?

• What are the benefits they are enjoying as a result of the project?

Kee

p it

rea

list

icExercise

3

Impact pathways – a more complete picture….

<-----the full picture---->

>---------------Actor-orientated perspective---------

>--

----

----

----

Ou

tco

me-

chai

n p

ersp

ecti

ve--

----

----

----

----

>

C

ausa

l Ana

lysi

s

Network maps

Workshop Road Map

1. Problem Tree

2. Outputs

3. Vis ion








Ide

nti

fyin

g a

lin

ea

r lo

gic

lin

kin

gp

roje

ct o

utp

uts

to

pro

ject

go

al

Inte

gra

tio

n o

fb

oth

vie

ws

6. ProjectS caling

S trategy

Ide

nti

fyin

g t

he

evo

lvin

gn

etw

ork

of

act

ors

ne

ed

ed

to a

chie

ve t

he

vis

ion


Pro

du

cts

pro

du

ced

in

th

e W

ork

sho

pP

rod

uct

s p

rod

uce

da

fte

r th

e w

ork

sho

p





I m pactnarrative

What is a network?

Many kinds of entities can be part of a network: people, projects, organisations, documents, events, cities, countries, etc.

Each of these entities can have different levels of influence in the network, and

And there are many kinds of relationships that can link such entities, involving transmission or exchange of information, money, goods, affection, influence, infection, etc.

A network is a collection of people and / or things that

are connected to each other by some kind of relationship

Advantages of network models

Actor-oriented descriptions: observable, understandable, verifiable

Captures real-life complexity: We are subject to multiple influences We influence many others

Network models help us understand innovation processes: Innovation processes happen through different actors,

acting in networks These interactions, relationships and influence are modeled

in network maps

Airline network

Road Network

How change happens

Orlikowski and Hofman, 1997

Improvements in poverty alleviation, food security and the state of natural resources result from dynamic, interactive, non-linear, and generally

uncertain processes of innovation.” EIARD, 2003

A network diagram

(organisations linked by a project)

Influence pathway: actor + relationship + actor + relationship…

CPWF

CREPA

ENTERPRISE

FGs

FRANC

ISSER

IWMI

KNUST

MAs

MOFA

RVAU

SGs

UC

UDS

URBANET

WRC

WRI

A plotted network diagram, multiple relations

The human eye is an analytic tool of remarkable power, and eyeballing pictures of networks is an excellent way to gain an understanding of their structure.

(The structure and function of complex networks, M. E. J. Newman)

A plotted network diagram, one type

of relation

Network tasks…..

1. Identify relevant actors

2. Develop network diagrams of key relationships (research, funding, scaling out and scaling up) for

Your project now Residual network 2 years after project has finished

3. Identify key levels of influence

4. Develop a scaling influence strategy (Worksheet 2)

Identify key actors

• Who are the actors involved in research, funding, scaling out and scaling up in the area your project is working in? Can be positions (eg. DDG-R) or organizationsRemember actors at different scales: community/local, your

own organization, regional, national, international

Some types of actors

Government Organization National Agricultural Research and Extension Organization (NAREO)

CGIAR CentreUniversity

Ultimate BeneficiaryNGO

Research OrganizationPrivate Sector

DonorOther (please specify)

Fill out an actor table (Table 1)

ACRONYM FULL NAME LOCATION TYPE OF ORGANIZATION

ROLE

IRRI Int. Rice Research Institute

Los Baños, Philippines

CG Centre Project implementer

MOFA Ministry of Fisheries and Agric.

Accra, Ghana

GO Politically-important actor

FGs Farmers Groups Northern Ghana

Ult. beneficiary End user

PhilRice Philippine Rice Research Institute

Muños, Philippines

NAREO Next user

DfID Dept. for Int. Development

London, England

Govnt. Org Funding agency

DDG-R Deputy Director General of Research

CIAT, Cali CG Centre Scaling-out actor

Exercise

4a

Develop a network diagram for your project now

Actors: Use cards for nodes Use different colour cards for different types of node

yellow = project implementers, blue = next user, green = end user, red = politically-important actor, red with black dot = donor

Relationships Use arrows to describe direction Use colour to describe relationship type

Green = funding; brown = research / work; red = scaling out; black = scaling up

Don’t use distance/length

Exercise

4b

Identify influence levels and attitudes in the

networks

Actors: Construct influence towers (0-3 chips) for key actors Indicate their attitude towards your project:

positive neutral negative

Exercise

4c

Draw a second network showing how actors need to be linked to achieve the vision Adjust the influence and attitude

Will the attitude of the actor remain the same or change? Will the same actors still be equally influential? Will there be new influential actors in the area?

Develop a future network corresponding to the vision

Exercise

5

Developing a Scaling Strategy (Table 2)

Describe the most important

differences between the two networks

Why is the change important to

achieve the vision?

What are the project’s strategies for achieving the

change?

Exercise

6a

Network Changes Achieved since the beginning of the project

(Table 2)

Exercise

6b

Describe the most important network and influence changes since the beginning of the project

Why was the change important?

What were the project’s strategies (e.g., co-development of knowledge and technology, communication, political lobbying, etc.) for achieving these changes?

Workshop Road Map

1. Problem Tree

2. Outputs

3. Vis ion








Ide

nti

fyin

g a

lin

ea

r lo

gic

lin

kin

gp

roje

ct o

utp

uts

to

pro

ject

go

al

Inte

gra

tio

n o

fb

oth

vie

ws

6. ProjectS caling

S trategy

Ide

nti

fyin

g t

he

evo

lvin

gn

etw

ork

of

act

ors

ne

ed

ed

to a

chie

ve t

he

vis

ion


Pro

du

cts

pro

du

ced

in

th

e W

ork

sho

pP

rod

uct

s p

rod

uce

da

fte

r th

e w

ork

sho

p





I m pactnarrative

Developing the Outcomes Logic Model (a description of the project’s impact

pathways)

• Why (Dart, 2005)?– To evaluate or clarify the logic of the project

intervention– To provide a framework to evaluate the

performance of a project • Before, during and after

• Evaluation can provide information to improve decision making and enhance learning

Outcomes Logic Model (fill one table for all stakeholder groups)

Actor (or group of actors who are expected to change in the same way)

Change in Practice required to achieve the Project’s Vision

Change in KAS required to support this change

Project strategies to bring about these changes in KAS and Practice?

Exercise

7

KAS = Knowledge, Attitudes and Skills

Identify Outcome Targets (prioritize changes from Outcomes Logic Model)

Exercise

8

The key outcomes the project wishes

to monitor

Assumptions SMART outcome

target

Means of verification?By whom?

How?

SMART = Specific, Measurable, Attributable, Realistic, Timebound

Identifying Milestones

SMART Outcome Target

SMART Milestone to be achieved in

the next six months

Means of Verification? By whom? In what form?

Who is responsible for making

the progress

described?

Exercise

9

1

2

3

4

5

Future witho utinte rve ntio n

Vis io n

Im p a ctPa th w a y s

Im pr o ve m e nt

Ti m e

1

Im p a ct Pa th w a y sW o rk sh o p

Where we are now

Impact Pathways Evaluation

1

2

3

4

5


Adjus te dVis io nAd ju s te d

Im p a ct Pa th w a y s

Im pr o ve m e nt

Ti m e

2

Re fle ctio n

The process

1

2

3

4

5


Vis io ns

Im p a ct p a th w a y s

Ac tualim pro ve m e nts

Im pr o ve m e nt

Ti m e

3

Re fle ctio nW o rk sh o p s

Workshop Road Map

1. Problem Tree

2. Outputs

3. Vis ion








Ide

nti

fyin

g a

lin

ea

r lo

gic

lin

kin

gp

roje

ct o

utp

uts

to

pro

ject

go

al

Inte

gra

tio

n o

fb

oth

vie

ws

6. ProjectS caling

S trategyId

en

tify

ing

th

e e

volv

ing

ne

two

rk o

f a

cto

rs n

ee

de

dto

ach

ieve

th

e v

isio

n


Pro

du

cts

pro

du

ced

in

th

e W

ork

sho

pP

rod

uct

s p

rod

uce

da

fte

r th

e w

ork

sho

p

Outcom etargets andm ilestones




I m pactnarrative

Use of PIPA OutputsUse of PIPA outputs PIPA outputs used

Ex-ante impact assessment

Essential:Impact logic model, output targets, impact narrativeOptional:Extrapolation domain analysis, scenario analysis

Impact Pathways Analysis Essential:Outcomes logic model, outcome targets and milestones, visionOptional:Impact logic model

Laying the foundation for ex-post impact assessment

Essential:Outcomes logic model and impact logic modelOptional:Impact pathways analysis (that updates project impact hypotheses)Extrapolation domain analysis

Program network maps Essential:‘Now’ and ‘future’ network maps

Farm ers usingdrought probability

m ap and droughtto lerant varieties

I m proved utility ofcom m unity dugouts

Majority ofcom m unities in

Northern Ghana haveconstructed and are

using dom estic w aterharvesting system s

More tim e forincom e

generatingactivities for

w om en

I m provedincom e for rural

households

I m proved foodsecurity and

rurallivelihoods

I m proved croppingsystem s in Northern

Ghana

Farm ers plant toavoid crop loss dueto draught, m ajority

have intensifiedcropping system s

Farm ers routinelygenerate organic

m atter , e.g.com posting andcover cropping

Farm ers usingappropriate

tillage m ethodsto conserve

soil m oisture

Changes tohousing

structure tom eet w aterharvesting

needs

More w ateravailable for

dom estic needs

High labourproductivity

High land andw ater

productivity

Com m unitydugouts

effi cientlyutilized for fish

production

Effective m anagem entof com m unity w ater

resources

Adequate w atersupply for dry

seasonagriculture

I m proved soilfertility

Project Goals

9

8

11

10

11

Drought probabilitym ap

Drought to lerantvarieties developed

Crop productionguides or m anuals for

MoFA

Best-bet soil and w aterconservation and

m anagem ent optionsm anuals

5

S caling up

Dugoutsenhanced toretain w ater

Com m unitieshave know ledge

of low -costdom estic w ater-

harvestingsystem s

Com m unitiestrained on

effi cient fishproductiontechniques

Manuals on fish culture indugouts and dugout

m aintenance

Manuals onappropriate w ater

harvesting system s

Reduction in w aterrelated diseases

4

Methods developed toinstitutionalize dialogueabout w ater use am ong

m ultiple users

7

Crop Related Outputs

Water Related Outputs

Crop Related Outcomes

Water Related Outcomes

W ater UsersAssociations form ed

and strengthened

Higher cropyields

W ider adoption of project outputs beyondpilot sites

Adoption of pro ject outputs by MoFA forextension after project finishes

National variety releasecom m ittee releases

varieties

6

Scaling Up

3Project Activities

carried out in PilotSites w ith

stakeholders andultimate

beneficiaries

1ScalingOut

ScalingOut

Soil and w aterconservation im proved

in farm lands in N.Ghana

I mprovedknow ledge ofstakeholdersat pilo t sites

Changes instakeholdersattitudes andperceptions

S takeholdersmodify and

innovate

Adoption oftechnologiesand changesin practice

I terations oflearning cycle

2

Impact Logic Model for the Strategic Innovations in Dryland Farming (SIDF) Project

Says little about who does what …….an actor-orientated perspective is missing

……provided by network mapping

….. as part of an impact narrative

Scope of impact

Un

cert

ain

ty

of

imp

act

Local, close-to-project

Basin-scale, global

Secured

Anticipated

Participatory Impact Pathways Analysis

Extrapolation domain analysis

Scenario analysis

Methodologies used for evaluating impact potential

Exploring possibilities

Link between regions through bio-physical similarities

But socio-economic condition also critical

EDA method

Origin

Representative sites selected

Critical success factors (soc)

selected

Homologue WofE

Modelling

P ( soc ) i estimated

P (bio) i estimated

Origin

Target

Representative sites selected

Critical success factors (soc)

selected

Homologue WofE

P ( soc ) i estimated

P (bio) i estimated

Looking for pixels with similar climate (and soil)

Looking for pixels with

similar socio-economic (land use

factors)

Project origin

Extrapolation domain of aerobic rice in Asia

Areas that hold promise for adoption of aerobic rice

EDA: Summary

• Method in development

• Represents what what is believed, on basis of– Current experience– Available global data

• Beware false prophets, BUT…

• …a rapid first step in exploratory dialogue with new stakeholders

Info needed for Extrapolation Domain Analysis

What is the main change that your project is contributing to that is likely to scale-out and scale-up? Who will adopt?

What are the factors likely to influence scaling-out and scaling-up?

Where are your pilot sites in which the change is starting to happen?

Exercise

10

Volta – Research network

Next Steps

• Participants finish worksheets and send to BFP-Impact Project (end of Nov)

• IP writes workshop report (end of Nov)• IP send draft impact narratives to projects

(end of Jan)• Participants respond to queries (end of Feb)• IP write first draft of Limpopo Basin Impact

Pathways Working Paper (end of March)

limpopo basin impact pathways workshop 15 – 17 november 2007 birchwood hotel, johannesburg boru...

Documents

project outputs

project problem tree

impact pathways matter

project goal

goal slide

project visions

cpwf impact project

colombia slide