mixing methods in evaluation of development …

Area: Rural Development and Cooperation Projects

―Selected Proceedings from the 13th International Congress on Project Engineering‖. (Badajoz, July 2009)

MIXING METHODS IN EVALUATION OF DEVELOPMENT PROJECTS:

PCIP CASE STUDY IN LURIGANCHO-CHOSICA (PERU)

José Luis Yagüe

Miguel Salvo

Research Group in Planning and Sustainable Management of the Rural-Local Development (GESPLAN). Technical University of Madrid

Gordon Prain

Nieves Gonzales

International Potato Center. Urban Harvest Program

Abstract

Evaluation of development projects always implies some difficulty to measure socioeconomic impacts. However, It is generally accepted that the sustainability of processes is a key for success, and so processes themselves should be the object for the evaluation. In this sense, evaluation can be considered as a tool to promote or to reinforce those processes.

This article summarizes a case study in Lima outskirts carried out by researchers from the Technical University of Madrid and the International Potato Center. Mixing methods has become an approach to program evaluation, and so different evaluation methods and tools have been combined here: key questions, process mapping and empowerment evaluation.

This methodology has been useful to identify ongoing processes and their links with project achievements. This is the base to involve stakeholders in consolidating those processes.

Keywords: evaluation, empowerment evaluation, process mapping, development projects

1. Introduction

The way we choose for the execution of a development project is the key element of the final quality, since the supported or impelled development process is the main goal. (Ferrero, 2003; Varela, 2007). Consequently, if we try to evaluate the quality and the sustainability, it is very important to determine the flexibility of the project to be adapted to the changes of the environment and the capacity to strengthen the processes.

In this sense, requirements of the financial agencies are focus on the efficacy/efficiency in order to achievement of results. However there is a growing demand to measure sustainability and long-term impacts.

This approach demands to put the attention more in the processes that are started across the Project, and also their learnings than in the direct and short-term results. This implies an adjustment of the systems of monitoring and evaluation.

Evaluation culture has been gaining a growing worldwide recognition (Diaz-Puente et al., 2007), and an increasing influence to foster change processes (Kirkhart, 2000), capacity building and learning (Taut, 2007).

This paper shows how this process-centered approach has been carried out in the ex –post evaluation of a cooperation Project located in the east cone of Lima (Peru). For this purpose, several methods and evaluation tools had been mixed up. Mixing-methods has become a common approach to program evaluation(Green and Caracelli, 1997; Hishigsuren, 2007;

315



Lawrenz and Huffman, 2002), in the belief that ―a combination of methods may bear the

potential to produce a study that is superior to that which can be produced by any single-method approach‖ (Waysman and Savaya, 1997).

2. Methodology

Figure 1 shows the methodology and tools that have been used for the evaluation, organized through key questions. This is the most generalized approach for the evaluation of the programs of the EU because it is very adequate to define evaluation contents (Commission of the EU, 1994). The methodology for answering key questions follows this sequence (Díaz-Puente, 2003):

To structure the information.

To formulate key questions.

To describe the purpose of the question.

To define criteria and indicators.

To analyze quantitative and qualitative information.

To answer the question: conclusions and recommendations.

To describe limitations to the answer.

The task of documenting, previous to the analysis, as in any other evaluation work, includes the review of the different reports of the project, data bases, publications, etc. Another tools are used in a complementary way such as economic analysis of case studies, and mainly participative tools in order to obtain qualitative information: interviews with key informants, focal groups, questionnaires5.

Figure 1. Methodological approach and evaluation tools applied to Project of Integral Productive

Cooperation (PCIP)

5 There are many references about these tools. A traditional one is: UD-NORAD, Evaluación de proyectos de ayuda

al desarrollo. Manual para evaluadores y gestores. IUDC-CEDEAL, Madrid, 1997.

EMPOWERMENT

EVALUATION

WORKSHOPS

TRADITIONAL

EVALUATION TOOLS

PROCESS MAPPING

WORKSHOP

PCIP + URBAN HARVESTORIENTATIONS FOR THE

EVALUATION

GESPLAN-UPM

METHODOLOGICAL APPRACH

PARTICIPATORY EVALUATION

PARTICIPACIÓN

SOCIAL

INTERVIEWS, WORKSHOPS, ,

SURVEYS

ANALYSIS AND RESULTS

ANSWER TO THE EVALUATION QUESTIONS

QUANTITATIVE

INFORMATION

REPORTS, STUDIES, DATA

BASES

PROCCESS MAPPING

EVALUATION QUESTIONS

CONCLUSSIONS AND RECOMENDATIONS Phases with joint learning

KEY

QUESTIONS

METHODOLOGY

316



However, there have been two main methodological complements: process mapping and empowerment evaluation.

Process mapping is oriented to identify processes and sub-processes, conscious or underlying, that have been involved in the Project. On the base of these processes is easier to understand the execution, to structure it and then to define the evaluation questions according to each process.

Empowerment evaluation is oriented to involve the main Project stakeholders into the evaluation: groups that have acted and also had enough entity and structure to continue some aspect of the Project. In this sense Empowerment Evaluation not only directs to participation –as a tool to obtain qualitative information– but also to the perception of the different processes by the most important stakeholders that can contribute to its sustainability.

The mix of these two methodological complements leads to the learning of the involved stakeholders (team Project, beneficiaries, local institutions). This learning is achieved through self-evaluation about how processes have been started, how have being carried out, and also how can continue after the Project is finished.

Key questions approach and the rest of the traditional tools are well known, so only process mapping and empowerment evaluation will be briefly described now.

Process Mapping

This methodological approach is usually linked to disciplines related to business and industrial organization. It has been known as an useful tool for understanding complex processes (Pojasek, 2005) and how an organization works (Matsumoto et al., 2005), but it is also very useful to legitimate and spread strategic changes. Consequently Process Mapping has a reforming and analytical capacity (Fenton, 2007).

Because of these characteristics Process Mapping is useful in processes-centered and learning-oriented evaluation. Learning arises during the production of the map that evaluator conducts.

There are many ways to draw the processes map, but basically it is necessary to take stock of all the activities carried out in the Project and the relations among them –direct and indirect. The basic sequence that has been applied in our case study was as follows:

Review of monitoring reports: the evaluator draws every activity in the draft map, looking for precedence relations among them. He could take the original strategy of the project as framework, but it is not necessarily the better option, rather it suits to rethink if the relations among activities fit better in another scheme that facilitates the comprehension of the project and the coherence.

Interviews: the evaluator supports individual meetings with each of the members of the project team. to penetrate into each of the activities, to discuss the relations among them, and also to identify activities that have been carried out but not reflected adequately in the reports.

Validation workshop: the first joint learning process takes place in it, since it is the first time that all the project team have a global vision of every activity and process that the evaluator has detected. The debate is orientated to change activities, add new ones, identify relationships, sub-processes, etc. A very important task is to agree on the name that it is given to the processes.

317



Drawing the final map: a complete process map is produced that includes every activity, and then a simplified scheme only with processes and sub-processes is drawn. This map is contrasted with the original strategy of the project to see what areas have been covered, what new contributions to the strategy are, and to look for explanations of the complex process that has been developed. Finally both, the evaluator and the project team, select the main areas for being evaluated and the corresponding evaluation questions.

Empowerment Evaluation.

This approach has become a worldwide phenomenon since it was developed by Dr. David Fetterman and was made known in 1993 through the American Evaluation Society (Fetterman, 2001; Díaz-Puente, 2007). Empowerment evaluation and traditional evaluation are not mutually exclusive and can be mutually reinforcing (Fetterman and Wanderman 2007). Empowerment evaluation has a clear orientation and is designed to help the people to help themselves and improve their projects and programs by using self-evaluation. The evaluator has the role of coach or facilitator depending on the internal capacities of the program.

It is the learning base for key actors who are organized under any form of formal or informal association. In this sense it is necessary to work only with organizations that have vocation of permanency and could use the learning to improve. Empowerment evaluation it is not adapted for groups constituted by individuals joined around any singular activity, e.g. credit beneficiaries, attendants to workshops or training courses, members of different kind of platforms without organic entail (round tables, institutional forums, etc).

As described by Miller and Lennie (2005), there are three steps in conducting an empowerment evaluation, through a series of workshops

Developing a mission and vision: This involves developing statements that capture the mission and vision or unifying purpose of the program. The aim is to reach consensus on key phrases in the statements, which are seen as representing the values of stakeholders and the agreed purpose and long-term outcomes of the program. This process is undertaken even when an existing mission and vision statement exists.

Taking stock: This step involves brainstorming, then rating the most important program activities to be evaluated and discussing the ratings in groups. This provides an opportunity for baseline data on the program and its strengths and weaknesses to be assembled.

Planning for the future: Realistic goals for each of the key activities are identified, together with strategies that will help reach these goals and the forms of documentation or evidence that will enable progress towards these goals to be monitored.

The philosophy of this approach seeks to strengthen the organizations through self-learning in an on-going evaluation basis; so it is strongly recommended to carried out empowerment workshops from time to time in order to consolidate achievements and re-plan the following year.

3. Application to the Project of Integral Productive Cooperation (PCIP) in Lurigancho-Chosica.

Lurigancho-Chosica district is located in the East Cone of Lima, and it is one of the widest and less built-up of this province. A wide variety of agricultural, industrial and commercial activities coexist in an area covering half of the useful surface. The Urban Harvest program6 and the 6 The Urban Harvest program is a system-wide initiative of the Consultative Group on International Agricultural Research (CGIAR), that it is coordinated by the International Potato Center (CIP). This program tries to promote

318



Technical University of Madrid proposed a development Project in this place focused on urban agriculture.

The main goal of the project was to reduce urban poverty and to increase food security through empowering local governments and planning for sustainable local development. a Program of Integral Cooperation Productive, was designed and carried out to achieve it. This PCIP was based on the integration of the urban and peri-urban agriculture in the sustainable development of the local and regional governments, and aligned with the national plan of fight against the poverty with perspectives of long-term sustainable development.

The conceptual framework (Prain, 2006) is based on an integral model of urban agricultural production, with a systemic, planned and participative approach, which facilitates the social learning among stakeholders like producers, local governments, research organizations and any other local group. Likewise it promotes social organization, especially among farmers.

The project was executed from 2006 to 2008. and is characterized for presenting a territorial approach, and for promoting the links with local governments to guarantee the sustainability (Salvo, 2006). The activities and results were expected in the following areas:

Training: both urban farmers and municipal managers.

Strengthening institutions: defining a system of co partnership in the urban management among all the entities (local Governments, farmers, irrigation associations, owners' associations, etc).

Social development as a key element of the economic development: improving human development and environment.

Micro investment fund: stimulating new economic activities.

Transfer of results and expansion to other municipalities.

Evaluation tasks

The application of the methodology described above needed 1 workshop of process mapping and 4 workshops of empowerment evaluation, besides a round of interviews to responsible people of the involved institutions, 3 economic analysis of case studies and a survey to beneficiaries of microcredit.

Identification of learning processes and key areas for evaluation purposes

Process mapping was used to get the map that is shown in figure No. 2. This map includes all the realized activities that have been carried out and also the relations among them. It is not the object of this paper to describe all of them in detail, for which a major extension would be necessary, but to show how this methodology allows to offer an image of the complexity of the project, and especially how it allows us to identify processes and sub-processes.

An horizontal reading shows the processes aligned with four big areas of work. Process have been drawn in white color when is considered to be processes and sub-processes that shape the main task. In the other hand, they had been drawn in red if they are considered as derivative processes, normally answering to a vertical reading that shows how they interact in an interesting and sometimes unexpected way.

Figure No.3 outlines the processes -without activities- and shows how they are organized in methodological blocks and how they complement each other mutually. This analysis gives a great more depth to the evaluation that considering the different work areas in a separated way, because it shows how many processes are interdependent. For example, the Microcredit, far from being an independent component, is a crucial element to consolidate the advances

urban and periurban agriculture as a key component to improve sustainability in towns of Africa, Asia and Latin America.

319



obtained in training and marketing, offering the beneficiaries the possibility of putting into practice what they have learned. In other area, the map shows how all the activities and achievements of the institutional strengthening process drive finally to actions of transfer, so both process are not separated but strongly linked.

These processes and their interrelationships shape the real strategy of intervention, which can be compared with the initial strategy of the project.

320



Figure 2. Detailed processes map of the PCIP Lurigancho-Chosica.

YE

AR

20

06

YE

AR

20

08

YE

AR

20

07

SIS

TEM

A D

E

CO

NTR

OL

INTE

RN

O

EC

A C

UY

ES

(1 c

iclo

)

DIS

EÑ

O C

EN

TRO

P

RO

CE

SA

MIE

NTO

CO

NS

OR

CIO

O

PC

EL&

CC

ER

TIFI

CA

CIÓ

N

EC

OLO

ÓG

ICA

DE

LA

S P

AR

CE

LAS

CU

RS

O E

N

MA

NE

JO

INTE

GR

AD

O D

E

CU

LTIV

OS

CU

RS

O

FOR

MA

CIÓ

N

LÍD

ER

ES

CU

RS

O

GE

STI

ÓN

E

MP

RE

SA

RIA

L

FER

IAS

CO

ME

RC

IO

EC

OLÓ

GIC

O E

N

LIM

A

OP

ER

AC

IÓN

FO

ND

O

RO

TATO

RIO

CE

RTI

FIC

AC

IÓN

O

RG

ÁN

ICA

IMP

ULS

O D

E

RE

LAC

ION

ES

C

OM

ER

CIA

LES

Y

PR

OFE

SIO

NA

LES

ES

CU

ELA

S D

E

AG

RIC

ULT

OR

ES

CO

ME

RC

IALI

Z. D

E

PR

OD

UC

TOS

AU

FER

IAS

M

UN

ICIP

ALE

S

TRA

DIC

ION

ALE

S

FER

IAS

M

UN

ICIP

ALE

S E

N

LA E

AU

(A)

VE

NTA

A

DO

MIC

ILIO

EN

LI

MA

(A)

EX

PLO

RA

CIÓ

N

CA

NA

LES

C

OM

ER

CIA

LES

PA

RTI

CIP

AC

IÓN

E

N E

VE

NTO

S D

E

PR

OM

OC

IÓN

(A)

EC

OLÓ

GIC

A-P

ER

Ú

(A)

VE

NTA

DIR

EC

TA

EA

U

(A)

VE

NTA

A

RE

STA

UR

AN

TES

LO

CA

LES

(A)

RE

PLI

CA

CIÓ

N

SE

MIN

AR

IOS

PO

R

LOS

B

EN

EFI

CIA

RIO

S

FOR

MA

LIZA

CIÓ

N

GR

UP

OS

P

RO

DU

CTO

RE

S

CO

NS

TITU

CIÓ

N

EQ

UIP

O D

E

TRA

BA

JO

FOR

MU

LAC

IÓN

D

E E

STR

ATE

GIA

IN

ICIA

L

VA

LID

AC

IÓN

DE

LO

S G

ER

EN

TES

TALL

ER

ES

P

AR

TIC

IPA

TIV

OS

C

ON

LA

P

OB

LAC

IÓN

TALL

ER

DE

A

LCA

LDE

S 2

006

ES

TRA

TEG

IA D

E

AG

RIC

ULT

UR

A

UR

BA

NA

M

UN

ICIP

AL

AU

EN

LA

S

UB

GE

RE

NC

IA D

E

LA D

.AG

RA

RIA

DE

LI

MA

SE

NS

IBIL

IZA

CIÓ

N

EN

ÁM

BIT

OS

S

UP

RA

MU

NIC

IP.

EQ

UIP

O L

OC

AL

DE

AG

RIC

ULT

UR

A

UR

BA

NA

PR

OG

RA

MA

DE

D

ES

AR

RO

LLO

P

OR

CIN

O

ES

TRA

TEG

IA D

E

AG

RIC

ULT

UR

A

UR

BA

NA

OR

DE

NA

NZA

S

MU

NIC

IPA

LES

EN

A

GR

ICU

LTU

RA

U

RB

AN

A

RE

GLA

ME

NTO

DE

B

UE

NA

S

PR

ÁC

TIC

AS

A

GR

ÍCO

LAS

AC

TUA

CIO

NE

S D

E

OR

DE

NA

CIÓ

N D

E

EX

PLO

TAC

IÓN

ES

EN

TRE

GA

DE

M

ÓD

ULO

S A

C

AP

AC

ITA

DO

S

PROGRAMA ―MI

GRANJITA‖

CU

RS

OS

PA

RA

C

RIA

DO

RE

S

CE

RD

OS

(CE

SA

L)

FOR

MA

CIÓ

N D

E

OR

GA

NIZ

AC

ION

ES

D

E P

RO

DU

CTO

RES

(3

org

aniz

acio

nes)

PR

OY

EC

TOS

D

EM

OS

TRA

TIV

OS

D

E C

RIA

DE

RO

S

INS

EM

INA

CIÓ

N

CO

ND

ICIO

NA

DA

A

INV

ER

SIÓ

NC

AP

AC

ITA

CIÓ

N

TALL

ER

DE

A

LCA

LDE

S 2

006

CO

NV

EN

IO M

AR

CO

U

RB

AN

HA

RV

ES

T-M

UN

ICIP

ALI

DA

D

ME

TRO

. LIM

A

RE

PLI

CA

CIÓ

N

ES

CU

ELA

S D

E

AG

RIC

ULT

UR

A

UR

BA

NA

EN

O

TRO

S D

ISTR

ITO

S

TALL

ER

ES

P

RE

PA

RA

TOR

IOS

C

ON

LA

S

GE

RE

NC

IAS

CU

RS

O M

AN

EJO

O

RG

ÁN

ICO

(2 c

iclo

s)

FOR

MA

CIÓ

N

CO

NTÍ

NU

A E

N

LID

ER

AZG

O,

GE

STI

ÓN

, ETC

.

CE

NTR

O

PR

OC

ES

AM

IEN

TO

DE

HO

RTA

LIZA

S

TRA

NS

FER

EN

CIA

FO

ND

O

RO

TATO

RIO

Y

SE

GU

IMIE

NTO

1ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

2ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

3ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

4ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

5ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

6ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

7ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

INFO

RM

E F

INA

L P

CIP

INFO

RM

E

SE

GU

IMIE

NTO

20

06

PR

OG

AM

AC

IÓN

20

06P

RO

GR

AM

AC

IÓN

20

07IN

FOR

ME

S

EG

UIM

IEN

TO

2007

INFO

RM

E

SE

GU

IMIE

NTO

20

08

EV

ALU

AC

IÓN

P

AR

TIC

IPA

TIV

AP

RO

GR

AM

AC

IÓN

20

08

PR

OP

UE

STA

P

RO

YE

CTO

C

HIM

BO

TE

EA

U/S

ALU

D

TRA

NS

FER

EN

CIA

D

E

ME

TOD

OLO

GÍA

S A

P

RO

YE

CTO

S E

N

OTR

AS

RE

GIO

NE

S

DE

PE

RÚ

ES

CU

ELA

S D

E

AG

RIC

ULT

OR

ES

U

RB

AN

OS

(4 n

uevo

s di

strit

os)

CU

RS

O E

N

MA

NE

JO

INTE

GR

AD

O D

E

CU

LTIV

OS

CU

RS

O M

AN

EJO

IN

TEG

RA

DO

DE

C

ULT

IVO

S Y

C

RIA

NZA

TALL

ER

DE

FO

RM

AC

IÓN

DE

TÉ

CN

ICO

S

MU

NIC

IPA

LES

TALL

ER

DE

D

IAG

NÓ

STI

CO

P

AR

TIC

IPA

TIV

O

CU

RS

O D

E

FOR

MA

CIÓ

N D

E

FAC

ILIT

AD

OR

ES

D

E E

CA

S

TALL

ER

E.E

. CO

N

PR

OD

UC

TOR

ES

: P

LAN

TR

AB

AJO

A

NU

AL

RE

CE

RTI

FIC

AC

IÓN

CU

RS

O D

E

CA

PA

CIT

AC

IÓN

EN

M

AN

EJO

PO

RC

INO

TALL

ER

MA

NE

JO

BIO

HU

ER

TO

INTE

GR

AD

O

ES

CU

ELA

S D

E

CA

PA

CIT

AC

IÓN

A

GR

AR

IA

RE

SE

RV

OR

IOS

Y

AC

UIC

ULT

UR

A

NU

TRIC

IÓN

IN

FAN

TIL

AU

INC

LUID

A E

N

LAS

OR

DE

NA

NZA

S

DE

LIC

EN

CIA

DE

A

CTI

VID

AD

ES

FOR

MU

LAC

IÓN

P

CIP

200

5

PR

OY

EC

TO

HO

RTI

CU

LTU

RA

E

N A

ND

ES

C

EN

TRA

LES

TALL

ER

ES

P

AR

TIC

IPA

TIV

OS

D

E R

ED

AC

CIÓ

N

PR

ES

EN

TAC

IÓN

, V

ALI

DA

CIÓ

N Y

C

OR

RE

CC

IÓN

SE

NS

IBIL

IZA

CIÓ

N

A R

EG

IDO

RE

S

TALL

ER

ES

P

AR

TIC

IPA

TIV

OS

D

E R

ED

AC

CIÓ

N

RE

GLA

ME

NTO

DE

C

RIA

NZA

PO

RC

INA

MA

NU

ALE

S D

E

AG

RIC

ULT

UR

A

UR

BA

NA

RE

UN

ION

ES

FO

RM

ULA

CIÓ

N

PR

OY

EC

TO A

U

CO

N L

A M

ML

CO

NS

TITU

CIÓ

N

DE

L FO

ND

O

RO

TATO

RIO

RE

GLA

ME

NTO

FR

AC

MA

NU

AL

DE

C

AM

PO

VA

LID

AD

OO

RG

AN

IZA

CIÓ

N D

E

LA G

ES

TIÓ

NP

RO

MO

CIÓ

N Y

S

OLI

CIT

UD

ES

P

ILO

TO

INTE

RM

ED

IAR

IOS

P

RO

VE

ED

OR

ES

S

UP

ER

MA

RK

ETS

PR

OD

UC

TOS

BE

BÉ

(A)

AS

ES

OR

AM

IEN

TO

CO

NTI

NU

O

RE

ALI

ZAD

O P

OR

E

CO

-LO

GIC

A

VE

NTA

A

EN

TID

AD

ES

DE

C

ON

SU

MID

OR

ES

O

RG

ÁN

ICO

S (A

)

PÁ

GIN

A W

EB

C

OM

ER

CIA

LIZA

C.

SE

GU

IMIE

NTO

S

EM

AN

AL

(pro

ducc

ión,

té

cnic

as, i

ngre

sos,

)

SE

GU

IMIE

NTO

Q

UIN

CE

NA

L (o

rgan

izac

ión,

sis

t. in

tern

o co

ntro

l)

SE

GU

IMIE

NTO

Q

UIN

CE

NA

L (P

lan

de tr

abaj

o)

CU

RS

O

AC

TUA

LIZA

CIÓ

N Y

N

IVE

LAC

IÓN

AS

ES

OR

ÍA P

AR

A

OR

GA

NIZ

AR

C

UR

SO

S D

E

CA

PA

CIT

AC

IÓN

VIS

ITA

S A

LA

S

EA

U

(vec

inos

, ON

G, e

tc)

PA

PA

CO

CTE

L

(CIP

)

INV

ES

TIG

AC

IÓN

P

AR

TIC

IPA

TIV

A

CA

PA

CIT

AC

IÓN

E

N M

AN

EJO

O

RG

ÁN

ICO

(2 c

iclo

s)

TALL

ER

SO

BR

E

CU

ALI

DA

DE

S

NU

TRIC

ION

ALE

S

DE

L P

RO

DU

CTO

TRA

TAM

IEN

TO

AG

UA

S R

IEG

O

FER

OM

ON

AS

TO

MA

TE

PR

EV

IOU

S

AC

TIV

ITIE

S

BUSINESS AND

MARKETING

TRAININGINSTITUTIONAL

STRENGHENTENPR

OC

ES

SE

S M

AP

OF

UR

BA

N H

AR

VE

ST

(P

CIP

)D

r. Jo

sé L

uis

Yagü

e -

Nov

embe

r, 20

08

SE

MIN

AR

IOS

Y

TALL

ER

ES

AV

ES

(o

rgan

iza

subg

eren

cia)

SE

GU

IMIE

NTO

C

ON

TIN

UO

DE

A

CTI

VID

AD

ES

DE

LA

SU

BG

ER

EN

CIA

BIO

HU

ER

TO E

N

ES

PA

CIO

R

ED

UC

IDO

(a

utoc

onsu

mo)

SE

MIN

AR

IOS

C

OR

TOS

AD

HO

C

AS

OC

IAC

IÓN

A

AN

PE

FER

IA

DE

MO

STR

ATI

VA

E

N E

L C

IP

Insti

tuti

onals

en

sit

izatio

n

Rep

licati

on

14

2In

teg

rati

nt

urb

an

farm

ing

in th

em

un

icip

al d

evelo

pm

en

tstr

ate

gy

Tra

inin

g in

Urb

an

Farm

erF

ield

Sch

oo

ls

6T

rain

ing

in b

usin

ess

cap

acit

ies

7A

co

mp

an

yin

g

9O

pen

ing

new

co

mm

erc

ial

ch

an

nels

15

MA

NA

GE

ME

NT

AN

D

MO

NIT

OR

ING

13

Mic

rocre

dit

10

Mark

eti

ng

Reg

ula

tio

ns

develo

pm

en

t3

Org

an

ic

cert

ific

ati

on

11

Co

nso

lid

ati

on o

f

the p

rod

ucers

'

org

an

izati

ons

12

4T

rain

ing

in b

reed

ing

CU

RS

O

CA

PA

CIT

AC

IÓN

C

RIA

NZA

DE

P

OR

CIN

O

MO

DU

LOS

Y

CU

RS

OS

DE

C

RIA

NZA

DE

A

NIM

ALE

S

1

5

CR

EA

CIÓ

N D

E

MA

RC

AS

Y P

LAN

D

E C

OM

ER

CIA

LIZ.

1.2

.0.0

1.2.

1.0

1.2

.2.1

1.2

.2.2

1.2

.3.1

1.2

.3.2

14

.1.0

.0

14.1

.1.1

14.1

.1.2

14

.1.2

.01

4.1

.2.3

14

.1.2

.21

4.1

.2.1

1.3

.0.0

2.2

.0.0

2.2

.1.1

2.2

.2.1

2.2

.2.2

2.2

.2.3

2.3

.0.0

2.2

.1.2

2.3

.1.0

2.3

.2.1

2.3

.2.2

2.3

.2.0

2.3

.3.0

2.3

.4.0

3.0

.0.0

3.1

.1.0

3.3

.1.0

3.2

.1.0

3.3

.2.0

3.3

.3.0

3.3

.4.0

3.2

.2.0

4.4

.0.0

4.1

.0.0

4.2

.1.0

4.2

.2.2

4.4

.1.1

4.4

.2.0

4.4

.3.0

4.3

.1.0

4.4

.1.2

4.4

.1.3

14

.3.0

.01

4.3

.1.0

14

.3.2

.0

15

.1.0

.01

5.1

.1.1

15

.1.1

.21

5.1

.2.1

15

.1.2

.21

5.1

.3.1

15

.1.3

.11

5.2

.0.0

15

.3.0

.0

13

.2.6

.01

3.2

.7.0

13

.3.0

.01

3.2

.4.0

13

.2.3

.01

3.2

.5.0

13

.2.0

.01

3.2

.1.0

13

.2.2

.01

3.1

.0.0

13

.1.3

.01

3.1

.1.0

13

.1.2

.01

3.1

.4.0

8.2

.0.0

8.2

.1.0

8.2

.2.0

8.2

.3.0

8.2

.4.1

8.1

.1.0

8.1

.2.0

5.2.

1.2

5.2.

1.1

5.2.

2.0

5.1

.0.0

5.2.

0.0

5.2.

1.3

5.2

.3.0

5.2

.1.4

6.1

.0.0

6.2

.1.0

6.2

.2.0

6.3

.0.0

6.4

.1.0

6.4

.2.0

6.5

.0.0

7.3

.0.0

7.4

.0.0

7.1

.0.0

7.2

.0.0

5.4

.1.0

5.4.

2.0

12

.1.0

.0

12

.2.0

.0

12

.3.0

.0

12

.4.0

.0

12

.5.1

.0

12

.5.2

.0

11

.1.1

.01

1.1

.2.0

11

.1.3

.01

1.1

.0.0

11

.1.4

.0

9.1

.0.0

11

.1.5

.0

9.1

.1.1

10

.1.1

.0

9.1

.1.2

9.1

.2.1

9.1

.2.2

9.1

.3.1

9.1

.4.1

9.1

.4.0

9.1

.4.2

9.1

.4.3

9.2

.0.0

9.0

.0.0

9.2

.0.0

10

.2.0

.01

0.1

.2.0

10

.3.0

.01

0.1

.3.0

10

.4.0

.0

11

.1.6

.11

1.1

.6.2

0.1

.0.0

0.2

.0.0

2.4.

0.0

4.4

.4.0

5.3

.1.0

5.3

.2.0

CO

NV

EN

IO C

ON

H

UA

CH

YP

A P

AR

A

RE

ALI

ZAR

AC

TIV

. C

ON

LA

OR

GA

NIZ

.

14

.1.3

.114

.2.0

.0

14

.1.3

.2

TALL

ER

C

AP

AC

ITA

CIO

N E

N

ME

TOD

OLO

GÍA

E

AU

TRA

NS

FER

EN

CIA

M

ETO

DO

LOG

ÍA

EA

U

14

.1.3

.3

8.2

.4.2

CU

BIE

RTA

S

CU

LTIV

O:

PLÁ

STI

CO

CU

BIE

RTA

S

CU

LTIV

O: M

ALL

A

AN

TIÁ

FID

OS

EV

ALU

AC

IÓN

DE

M

ICR

OO

RG

AN

ISM

. E

FEC

TIV

OS

(CIP

)

Part

icip

ato

ry

resarc

h8

8.2

.5.0

CO

NV

EN

IOS

CO

N

4 N

UE

VO

S

MU

NIC

IPIO

S

AS

ES

OR

ÍA P

AR

A

CR

EA

CIÓ

N

SU

BG

ER

EN

CIA

P

AC

HA

CA

MA

C

8ª

CO

NV

OC

ATO

RIA

FO

ND

O

RO

TATO

RIO

13

.2.8

.0

SE

MIN

AR

IOS

Y

TALL

ER

ES

P

OR

CIN

O

(o

rgan

iza

subg

eren

cia)

4.2

.2.1

CR

EA

CIÓ

N D

E L

A

SU

BG

ER

EN

CIA

DE

A

GR

ICU

LTU

RA

U

RB

AN

A

RE

UN

ION

ES

IN

TER

INS

TITU

C.

2003

Y 2

004

1.1.

0.0

2.1

.0.0

321



Figure 3. Schematic processes map of the PCIP Lurigancho-Chosica.

OP

ER

AC

IÓN

FO

ND

O

RO

TATO

RIO

CE

RTI

FIC

AC

IÓN

O

RG

ÁN

ICA

ES

CU

ELA

S D

E

AG

RIC

ULT

OR

ES

CO

ME

RC

IALI

Z. D

E

PR

OD

UC

TOS

AU

ES

TRA

TEG

IA D

E

AG

RIC

ULT

UR

A

UR

BA

NA

M

UN

ICIP

AL

SE

NS

IBIL

IZA

CIÓ

N

EN

ÁM

BIT

OS

S

UP

RA

MU

NIC

IP.

PR

OG

RA

MA

DE

D

ES

AR

RO

LLO

P

OR

CIN

O

OR

DE

NA

NZA

S

MU

NIC

IPA

LES

EN

A

GR

ICU

LTU

RA

U

RB

AN

A

CA

PA

CIT

AC

IÓN

RE

PLI

CA

CIÓ

N

ES

CU

ELA

S D

E

AG

RIC

ULT

UR

A

UR

BA

NA

EN

O

TRO

S D

ISTR

ITO

S

TRA

NS

FER

EN

CIA

D

E

ME

TOD

OLO

GÍA

S A

P

RO

YE

CTO

S E

N

OTR

AS

RE

GIO

NE

S

DE

PE

RÚ

ES

CU

ELA

S D

E

AG

RIC

ULT

OR

ES

U

RB

AN

OS

(4 n

uevo

s di

strit

os)

CO

NS

TITU

CIÓ

N

DE

L FO

ND

O

RO

TATO

RIO

INV

ES

TIG

AC

IÓN

P

AR

TIC

IPA

TIV

A

14

2

6

7

9

15

13

10

3

11

12

4

1

5

12

.3.0.0

8

MA

IN

ME

TH

OD

OL

OG

ICA

L

BL

OC

K

CO

MP

LE

ME

NTA

RY

ME

TH

OD

OL

OG

ICA

L

BL

OC

K

SU

PP

OR

T E

LE

ME

NT

LO

BB

Y &

AC

CO

MP

AN

YIN

G

YE

AR

20

06

YE

AR

20

08

YE

AR

20

07

BUSINESS AND

MARKETING

TRAININGINSTITUTIONAL

STRENGHENTENPR

OC

ES

SE

S M

AP

OF

UR

BA

N H

AR

VE

ST

(P

CIP

)D

r. Jo

sé L

uis Y

agüe

-N

ovem

ber,

2008

Insti

tuti

onals

en

sit

izatio

n

Rep

licati

on

Inte

gra

tin

tu

rban

farm

ing

in th

em

un

icip

al d

evelo

pm

en

tstr

ate

gy

Tra

inin

g in

Urb

an

Farm

erF

ield

Sch

oo

ls

Tra

inin

g in

bu

sin

ess

cap

acit

ies

Aco

mp

an

yin

g

Op

en

ing

new

co

mm

erc

ial

ch

an

nels

MA

NA

GE

ME

NT

& M

ON

ITO

RIN

G

Mic

rocre

dit

Mark

eti

ng

Reg

ula

tio

ns

develo

pm

en

t

Org

an

ic

cert

ific

ati

on

Co

nso

lid

ati

on o

f

the p

rod

ucers

'

org

an

izati

ons

Tra

inin

g in

bre

ed

ing

Part

icip

ato

ry

resarc

h

322



From these blocks of processes, key questions of evaluation can be established with more concretion, sense and direction that those who would correspond to the areas of the initial strategy:

In what measure has the Urban Farmers Field School been an instrument adapted for the training in opposite to other conventional means? Basic indicators of global participation, level of permanency in the courses, intensive participants, level of satisfaction, links to other activities of the project, etc. are used to compare between these Schools and conventional training courses.

In what measure has PCIP helped to reinforce social organization and producers organization by means of marketing? Several indicators are used to answer this question, related to the constitution and consolidation of new companies and organizations, case studies of profitability of some activities, and also qualitative indicators about perception of the level of participation in the community.

In what measure has the Urban Farmers' Rotating Fund –microcredit– worked as an effective instrument for the promotion of sustainable productive activities? Basic indicators are used, mainly related to execution (returns, number of delinquent debtors, number of financed projects, etc), but also to the level of satisfaction of the beneficiaries, and the analysis of the procedure of management and monitoring.

In what measure has PCIP helped to include Urban Agriculture in the municipal agenda as a strategic element for the development? Answering from the analysis of institutional changes that has involved (municipal flowchart, schedule), direct effects in policies (regulations, programming), and new activities carried out in other municipalities.

Finally, two transverse questions are added as in any other project:

In what measure has PCIP been managed in an effective way for the fulfillment of the goals? The answer focus on levels of execution of budget, procedures of management, control of the activities, profile of the project team, transfer of results and interaction with other institutions and departments, etc.

In what measure has PCIP taken account of questions about gender and environment? Qualitative indicators of perception of the actors have been used.

The project team works jointly with the evaluator both in the construction of the map and in the identification of areas and questions of evaluation, therefore a first learning is reached, especially about the changes that are necessary to improve the methodologies, and also in relation to the possibilities of applying these methodologies to another places.

Empowering stakeholders to enhance sustainable processes

Empowerment evaluation workshops were directed to formalized groups that were able to continue their activity, and consequently were able to introduce changes thanks to this instrument. In this sense we emphasize how evaluation can be considered as one of the last activities in the project, thanks to which the beneficiaries take consciousness about their own organization and be able to think about the future.

In the case of the Urban Agriculture Office, the workshop gave a greater visibility and support to this office inside the municipality, by offering an overview of the achievements of the PCIP to the rest of people responsible of municipal management, and also offering the possibility to discuss among them about the implications in the municipal policy.

323



One of the best achievements of the project has been to organize, to create and to consolidate two producers' organizations with access to new markets, with demonstrative effect in organic management, and with support and municipal recognition. Both of them constitute a permanent shop window of the effects that PCIP have achieved, and so workshops were directed to each one of these organizations. Another workshop was carried out with an organization that had not been shaped across the project –though many members had been benefited of the activities–, as a control group.

To give an idea of the usefulness that this approach provided, figure no. 3 shows how the sense of organization, and therefore its continuity, was identified in the Mission of the new organizations, which in addition was connected with many of the carried out processes. On the contrary, the Mission of the control group showed a more individual point of view and scarcely incorporated links to a couple of processes.

Figure 4. Comparison of mission statements developed by producers' organizations through Empowerment Evaluation workshops and their relation with the identified processes

The orientation of these workshops was not mainly directed to obtain information for the evaluation, but to encourage participants to thing about the future of their organization after the project is finished. There is a double learning effect: learning obtained from all the activities that

PRODUCE

HEALTHY FOOD

GOOD PRACTICES

TO IMPROVE

ENVIRONMENT

IMPROVE

TECHNOLOGIES

OF PRODUCTION

ACCESS TO A

GROWING

MARKET

INCREASE

ECONOMIC

BENEFITS

MISION

GENERATE

ENVIRONMENTAL

CONSCIENCE

QUALIFY OTHER

PRODUCERS

IMPROVE SELF-

CONSUMPTION

AND NUTRITION

TRAINING IN

BUSINESS

MANAGEMENT

CONSOLIDATE

ORGANIZATION

TRAINING IN

ORGANIC

PRODUCTION

Cosecha Sana CaparapongoSAC (COSANACA)

6

9

10

5

12

11

8

14

13

IMPROVE PRODUCER´S QUALITY OF LIFE

OFFER AN ALTERNATIVE

OF JOB

PROFESSIONALIZATION

OF THE PRODUCER

ACCESS TO NEW

MARKETS

INVESTMENTS IN

GENETICS AND

INFRAESTRUCTURE

WASTE MANAGEMENT

AND PROPER

TECHNIQUES

ACCESS TO

CONTINUOUS

TRAINING

BETTER PRICES TO

IMPROVE REVENUES

TO PROTECT THE

ENVIRONMENT OF

WORK

MISION

Asociación de Productores de Saracoto

4 13

324



have been carried out, and learning obtained from the reflection towards the future. This is the base of the philosophy of empowerment, and for that reason this approach allows participants to chose which elements in the strategy are relevant –and so considered in the analysis– and which are not.

Process mapping had helped the project team to reinforce the overall view of strategy as an well integrated package. For that reason the last empowerment workshop was directed to this group. It has been very important in order to help them to define the last activities of the project, which also affect to several of the final recommendations in the evaluation report: the systematizing and publication of the developed experience, the systematizing and publication of the applied methodologies, the possibility to apply them in new project or places, etc. Definitively a reinforcement of the methodologies applied in the project has been achieved through this evaluation approach.

5. Conclusions

The evaluation presented as case study appoints to a methodological approach that is based on direct and joint participation of the actors involved in the project. The aim is to provide a judgment of value from the perspective of local learning generated across the activated processes, and not from the traditional perspective of accountability

Mixing methods and tools of evaluation, and especially process mapping and empowerment evaluation in a standard methodology of key questions, has proved to be specially useful.

Process mapping offers a great potential in the previous stage of the evaluation. Firstly it helps the project team to self-learning, offering an overview of all the activities out of the frame of the foreseen strategy, and also how processes take place as a logical succession of activities. Secondly it supports the following steps of the evaluation, because it helps to formulate the evaluation questions and to centre the criteria and indicators that are needed. But in addition process mapping is crucial in an learning-centered evaluation, since it allows to see the routes and processes that new projects can follow in the future.

Development processes always imply to manage communities, and consequently unpredictable reactions and also changes due to the local context. The final map shows the result of adapting the original strategy to that changing reality. Consequently it must be considered to be an input for learning that reflects how sustainable results can be obtained. This map is the base to elaborate new proposals, serves as contrast for other evaluation tools, and gives useful information about how to copy those processes into other projects.

Empowerment evaluation approach complements to other participatory tools as describes how processes are perceived by the stakeholders. But in addition this approach helps to promote the sustainability of these processes by working with organizations able to continue them, re-adapt them or apply them in any other places. In this respect empowerment evaluation contributes to the sustainability of the project.

It will be necessary to carry out new case studies in which this methodology is carried out, and also to evaluate the adjustment to other types of development projects, and to evaluate the adequacy to the requirements of the financial agencies. Nevertheless this experience shows how this mix of evaluation tools can contribute to enhance impacts of development projects.

References

325



Commission of the EU, The MEANS collection: Evaluating socio-economic programmes.

Luxembourg: Office for Official Publications of the European Communties, 1994.

Díaz-Puente, J.M., Diseño y aplicación de un modelo para el seguimiento y evaluación del

desarrollo rural en la UE. PhD diss., Universidad Politécnica de Madrid, España, 2003.

Díaz-Puente, J.M., Cazorla, A., Dorrego, A., ―Crossing national, continental and linguistic

boundaries: Toward a worldwide evaluation research community in journals of evaluation‖,

American Journal of Evaluation 28(4), 2007. pp. 299-415.

Díaz-Puente, J.M., ―La evaluación en planes y programa de desarrollo: Estado actual y

principales tendencias a nivel internacional‖, in Empowerment Evaluation: Metodología para el

seguimiento de programas y proyectos, Fetterman, D. y Cazorla, A., Madrid, 2007. pp. 21-82.

Díaz Puente, J.M.; Yagüe, J.L.; Afonso, A., ―Building Evaluation Capacity in Spain: A Case

Study of Rural Development and Empowerment in the European Union‖, Evaluation Review Vol 32 (5), 2008. pp. 478-506.

Fenton, E., ―Visualising Strategic Change: The role and impact of process maps as boundary

objects in reorganization‖, European Management Journal, Vol. 25(2), 2007. Pp. 104-117.

Ferrero, G., De los proyectos de cooperación a los procesos de desarrollo. Hacia una gestión

orientada al proceso. PhD diss., Universidad Politécnica de Valencia, España, 2003.

Fetterman, D., Foundations of Empowerment Evaluation, Thousand Oaks, CA: Sage, 2001.

Fetterman, D. and Wandersman, A., ―Empowerment evaluation: Yesterday, today, and

tomorrow‖, American Journal of Evaluation 28(2), 2007. Pp. 179-198.

Green, J. and Caracelli, V., ―Defining and describing the paradigm issue in mixed-method

evaluation‖, in Advances in mixed-method evaluation: The challenges and benefits of

integrating diverse paradigms, New Directions for Evaluation, vol. 74, San Franciso: Jossey-Bass, 1997. Pp 5-17.

Hishigsuren, G., ―Evaluation misión drift in microfinance: Lessons for programs with social

mission‖, Evaluation Review 31(3), 2007. Pp. 203-260.

Kirkhart, K., ―Reconceptualizing evaluation use: An integrated theory of influence‖, in The

expanding scope of evaluation use, New Directions for Evaluation, Vol. 88, Caracelli, V and Prskill, H. San Francisco, 2000. Pp. 5-24.

Lawrenz, F. and Huffman, D., ―The archipelago approach to mixed method evaluation‖,

American Journal of Evaluation 23(3), 2002. Pp. 331-338.

Matsumoto, I. et al., ―Use of process maps to develop a management briefing sheet for a

design consultancy‖, Engineering Construction & Arquitectural Management, Vol.12(5), 2005. Pp. 458-469.

Millers, W. and Lennie, J., ―Empowerment evaluation. A practical method for evaluating a

national school breakfast program‖, Evaluation Journal of Australasia, Vol. 5(2), 2005. Pp. 18-26.

Pojasek, R.B., ―Understanding processes with hierarchical process mapping‖, Environmental

Queality Management, Vol. 15(2), 2005. Pp. 79-86.

Prain, G., ―Urban Harvest: a CGIAR global program on urban and periurban agriculture‖, Food

& Fertilizer Technology Center. Extension Bulletin nº575, 2006, pp.14-36.

326



Salvo M., Arce B. y De los Ríos I., ―Integración de las entidades locales en la promoción del

desarrollo rural local consideraciones para Latinoamérica desde la experiencia del Cono Este

de Lima (Perú)‖, Proceedings of the X International Congress on Project Ingeneering, Valencia, 2006.

Taut, S., ―Studying sefl-evaluation capacity building in a large international developmet

organization‖, American Journal of Evaluation, Vol. 28(1), 2007. Pp. 45-59.

Tuggle, F. and Golfinger, W., ―A methodology for mining embedded knowledge from process

maps‖, Human Systems Management, Vol.23, 2004. Pp. 1-13.

Varela, F. et al., La Calidad de las Intervenciones de Desarrollo. CIDEAL-EPTISA (con la colaboración de la Agencia Española de Cooperación Internacional ), Madrid, 2007.

Waysman, M. and Savaya, R., ―Mixed method evaluation: A case study‖, Evaluation practice, 18(1), 1997. Pp. 227-.237.

Acknowledge

The authors gratefully acknowledge the collaboration of the staff of Urban Harvest program and people responsible of the Lurigancho-Chosica Urban Agriculture Office (Subgerencia de Agricultura Urbana) and the Association of Irrigation Users (Junta de Usuarios de Riego). In a special manner they acknowledge every beneficiaries of the PCIP that have unselfishly participated in evaluation tasks that constitute the basis of this paper.

Contact:

José Luis Yagüe Blanco Research Group in Planning and Sustainable Management of the Rural-Local Development (GESPLAN) Technological University of Madrid Dpto. Proyectos y Planificación Rural. E.T.S.I.Agrónomos Avda. Complutense, s/n 28040 Madrid Phone: +34 91 336 38 37 E-mail : [email protected] URL : www.grupogesplan.es

327

mailto:[email protected]

http://www.grupogesplan.es/

mixing methods in evaluation of development …

Documents