logframe for scientific program planning and evaluation 1988

8/8/2019 Logframe for Scientific Program Planning and Evaluation 1988

1/15

1

THE LOGICAL FRAMEWORKIN

RESEARCH PLANNING AND E V A L U A T ~ O N

sosrInternational Service for National Agricultural Reseatch


2/15

IThe Ir.ternational S e , ~ l c e tor ~ G ~ l o n G l Airicul tural Research (ISNAR)began operating at itn headquarters In The F.ague, N e t h e r l ~ n d 8 onSeptUlber I , 1980. It ",ao eatabHohed by th e Conoultative Group onInternat ional Agricultural Research (CGIAR)j on th e ba.is ofrecoll' lt:lendattono from an internationrl t aoK, force, for the purpose ofGooist:ing govenunento of d e ' r e l c ~ l ~ c o u n t r , ~ e B to strengthen thei ragricultural reaeareh. Tt ia a non-profit autonomous asency,Int.ernatlonal In character, and is llon.,HI"dti'::lll in lIanag!'ment, s taff ing,and operatlona.

Of th e 13 ccntera in th e CGIAR network, ISRAR Is the only one thatfocuseo primarily on natiJual agricul tural reaearch JAsues. I t providesad"dcc to governments, upon lequest, on research pol!..:y, ... rgan1zation,and mandgement IssueG, thus c o ~ p l e m e n t i n g the act iv i t ies of ocher~ s B i 8 t a n c e agencies.

ISRAR haD active adviaory aervice, research, and training p r ~ g r a m a .ISNAR is supported by a number of th e members of CGIAR, an informalgroup of approximately 43 donora, including countries , ~ e v e l o p m e n t banks,internat ional organizations, and foundations.


3/15

. .

WORKING PAPER NO. 12

THE LOGICAl. FRAMEWORKIN

R5SEARCH PLANNING AND EVALUATION

D.McLEAN

June 1988

ISnsrIntematlonal service for N8tlon.1 Agrlcultur81 A....rch


4/15

IinAR WORgINq PAPERS

The ISNAR working papers eeries is a flexible in st ruoent f or sha ri canalysis and information nbout r e l ~ v a n t organization and managementproblems of th e agricultural research systems in developing ~ o u n t r i ~ s .

In t he cour se of i ts activit iel - direct asoiat4nce to nationalagricultural research s y ~ t e m s , trainiug, and resp-arch - ISNAR generates abroad r a n g ~ of information and materials which eventually become th eformal products of i ts pUblication program. The work!ng papers seriesenhances this program in s ev era l important ways:

1. These papers ar e intended to be a rapid means of presenting th eresults of work and e x p e r i ~ n c e s that ar e s t i l l in progress, but arealready producing results that could be of use to others.

2. They are intended to be an effective vehicle for widenina th ed iscussion of continuing work, thereby inc reas ing the qua li ty o fth e f inal products. Critical comment is welcomed.

3. '[he series provides an outle t fo r diffllcing caterials andi n f o r m a t i ~ n Which, because of their l imited coverage, do no t meetth e requirements o f "gene ra l audience" publication.

The se ries is intended mainly for diffusion of materi al s produced byISNAR staff , bu t i t 1s also available fo r the pUblication of documentsproduced by other insti tutions, should they wish to take advantage of th eO?portunity.


5/15

.

THE LOGICAL FR.Ai'!EWOU IN RESEARCH PLANNING AND EVALUATION

Re.earch -.n&lement concerna many people: polic, .aker. , national relearcbl e 3 d . r l ~ d W Y e l o ~ n t or,aaizationa, prOlr.. chiefl tation manAserl. andcearcber.. In order to ..ke tbe 80It of tb . r ource. available torea.arcb. aaD4Jer. auat aware ~ tbe re.earch prlori t ie . defined bypollc , . . terl and ~ ~ l o n a l leadera. th e a , r i c u l t u r a ~ conatrainta andtechnical opportuniti.1 fo r rea.arcb. and the capabllltiea ofre..a r c b e r ~ . l rcb "nac . r . m u ~ t f o ~ l a t . prolrams which bave thebu t Ukellbood of fulf l l l in, national reaearcb objective takinl intoaccount th e perceived need. of f a ~ r l and t he t echn ical and r.aourceconstraint. whlch exl . t . Tbi. i . a coaplex teak. wbich require. th econaideration of ..ny thinl includinl:1. th e relationahip of prolrams to national r earch objectivea;2. the d e t e ~ Q Q t i o n of pro ,rama. vbether ba.ed on commoditie rellonfactor or dilciplioe.;3. the allocation of reaourcea a.)! l& prolr.... ba.ed upon opportunlti.aCor oucc and potential l8pact;4. the d e t . ~ i n a t i o n of projecta witbin prolrama from . -onl the manyalternative. po ible. bearine in . ind the importance of . ta f fcapability. in . t i tute relource cOllplementarity witb o t h ~ r project

and the l ikelihood of re.ulta which jUitify the inveatment.In this ,'Orkinl paper, we dcrib. a froaework for conceptuallz in.relearch projecta and prolr.... called lhe Lo,ic.l Framework (Fieure 1).The Lolical Fr...work i . limply a tool which provid.1 _ . tructure fo rlpeeifyinc the ca-ponents of an activity and the lOlieal l inke,e. betweena le t of eean. and a . t of enda. I t place. th e project in i t . lar lerfr.-avork of objective within the program and within the nationalre.earch .yatem. I t aerve uaeful tool for defininl inputa. tiRetable. , au.ption. fo r .uceeal, outputs, and indicator. fo r 8Onitor!naand evaluatinl p e r f o n u ~ l e e . l .e amin l to uae the Lolical Frameworkrequirea GO _ eor.centrated effort. and i t is of ten offered in " u l e _ n ttrainin. eour.e. . I t ia no t an eential teehllique bu t l t i . a hl,hlyeffective plannin, tool. Whether or not thi . technique i . Uled. theb lc i n f o ~ t i o n i t provide. i ential to adequate p ! ~ n n i n l . and . an int roduction to th e Fr.-ework i . uaeful.Thi. ISNAR workinl paper is one of .everal which de.cribe useful p r o j ~ c tplanoln, and manalement technique.. Sub.equent workinl paper. will coverth s development of project proposala and ~ o r k p l a n a , project managementtechnique. u.eful in performance monitoring. auch as c h e e k l ~ 8 t bar~ h a r t . , and the more complex cri t ical path network, and monitoring andevaluation. The author haa attempted to be aa s u ~ c i n c t a. possible inthi . paper. r ea li zing tha t researchers and re.eareh managera ar e b ~ p e o p l ~ who need practical management techniques. These t e c h n i q u e ~ areaimed to improve your ~ e s e a r c h . not to make rese&rch management an end initnlf


6/15

Figure 1

2

A LOGICAL FRAMEWORK MATRIX

~

Narrative Objectively Means of ImportantSummary Verifiable Verification AssumptionsIndicators (OVI) (MOV)Inputs Nature and levelof resources InitialSources of assumptions aboutNecessary cost information the project

Planned startingdate

Outputs Magnitudes of S o u r c ~ o f Assumptionsoutputs information affecting the Inputs-

Outputs linkagePlanned completion Methods useddate

Pu,pose Sources of Assumptionsnd of projectstatus information affecting the

Output-PurposeMethods used 1 1 1 ~ k a g e

- 'Goal Measures of go.1 Sources of Assumptiomachievement i n f o r m ~ t i o n affecting the

P u r p o ~ . .GoalMethods used link.ge


7/15

I

3Before b ~ g i n n i n g a d i s c u ~ s i o n of the Logical Framework, i t is necessaryto define th e terms "program" and "project", since they have differentmeanings in different research organizations. Programs are coordinatedresearch act iv i t ies whose combined scient i f ic outputs address nationalresearch objectives. Programs are long-term and someWhat continuous, andare composed. in some caees. of sub-programs. and of projects . Projectsaddress specific research problems. and have explicitly defined timeframes, resources. and t a r g e t ~ . Each p ro je ct i n turn comprises a numberof specific operations or experiments.The Logical Framework, or "logframe", can be used a t any level ofplanning and decision making, from th e development of programs toexperiments. I t is most aptly u s ~ d by small groups. as a framework forbrainstorming and discussion. The program logframe ~ best completed bycompiling m o ~ e specific, detailed logframes for each individual projectunder the program; these in turn are based on step by step work plans foreach operation. These prograr.mling documents help to define th e keyindicators used in monitoring and evaluation, and provide th e frameworkfo r progress reporting.The information required both to design and evaluate ~ act ivi ty can besumcarized on i ts four by four matrix: the rows represent differentlevels of project objectives, including the means r equir ed to ach ievethem (the vertical logic); th e columns indicate how the achievement ofthese objectives can be verified and th e assumptions that were made (thehorizontal logic). Table I indicates what type of information would beincluded in th e analysis ot a research program with various r.omponentprojects. Using this a& a guideline s h o ~ l d make supplying specificinformation easier.THE VF.RTICAL LOGICFrom t he bot tom to th e top in th e left column is a "narrative sUlllllary" ofthe four levels of objectives of a program, including the inputs.outputs, purposes and goals. I t should provide a clear, concisestatement of project object ives . and indicate th e plausibility of theassumed linkages b e t w e l ~ n levels.Inputs compr is e the per sonnel . physi ca l r esources and financial elementsneeded to achieve the stated outputs. TItese ty?ically include manpower,infrastructure. equipment, supplies, support services, and funds. Thespecific ~ e q u i r e m e n t s are define& from the development of an operationworkplan. In research activit ies i t is also valid to include leadershipand a defined Get of r p . ~ e a r c h objectives as i r ~ u t s .Outputs include those achievements derived directly from the managemento f i np ut s. For example, a maize breeding pcojp.ct within the maizeprogram with sufficient manpower, faci l i t i es and support (inputs) wouldbe expected to identify or develop within an estimated time frame newgp.rmplasm with certain desired characteristics (outputs).The Purpose is what th e project is expected to achieve once completed.The purpose is generally defined as the research objec t ives of a givenproject. In the example of a breeding program, i t is assumed that if av a r i e t ~ is identified with th e desired characteris t ics (output). thenproducers will adopt i t and production will increase (purpose achieved).


8/15

Table 1: toaieal Framevork: Research Programs

NarrativeS ~ r y Verifiableindicators Mean.of verification Important.ssUlllption.

If INPUT - by proJecta eccording to operation vorkplans

If OUTPUT: Program objectives - compiled for a ll projectsThen

RWII&Jl re.ourcesFundin .FacilitiesEquipment andsup[llie.Trainiu,Scientificleadership

Preliminaryresearch resultsComplettodresearch resultsResearch capacitystrengthened

Staff andfacil i t ies inplace by endyear I

Courses completedSenior:juniorstaff 1:5

Data from surveys/experimentsR e c ~ ~ n d a t i o n sby program comm.Improved II taff&. facil i t ies

Quarterly andannual reportsAccounting andadministrativerecordaTrainins recordaPersonnel data

Research reportsProgram recordsAnnual reftOrtsAdmin. :recordaPeer review

Funds" s taf fapproved willbe dbbursedand availcbleCoursea availableTime" me.ns fo rstaff sU(lerviaion

Scientific:a tandard' l upheldProcedure.: exis tfor rele.ae

If PURPOSE: Research system objectives - compiled for all programsThenNew knowledgeexiats ofinterest toresearch,extension. nndpolic:ymakers

Releaaedtechnologies orrecolJIIIendationsProgram recordsCertificationRes/extenllionCOllllllUnicationson policy

Inputll a\"ail4blePrices favorableExtension s.ervice.Seed mult capacity

GOAL: National development objectivesNev technologycontributes tonationaldevelopmentobjectives

Production dataChanges in croppatterns/inputsReduced erosionIncreased incomes

Farm surveysInput statis t icsSurvey methodsVillage surveys

Posi tive econ"micenvironmentStabil i tyAdequate road!.markets. etc.


9/15

5The Coal i . the u lt imate objec ti ve for under taking the r ~ s e a r c h project .In the broad conte2t of national development i t i_ uaually a desiredecoDOaic achievement fo r which th e a tt ai nmen t o f r es ea rch project orproaraa objectives are necessary bu t no t alway_ 9ufficient. Bere. USinlthe ..ia e program example. the o l l p ~ c t a t i o n i8 that if better maizetecbnolo8Y i8 available (output) more maize will be produced (purpoae).th.reby contribut ina to a nationsl goal of fuod self-sufficiency. I tshould be evident th6t increaaed maize production alotle iB not sufficientto ensure national food Belf-sufficiency.There ia a direct cause and effect relationship preaumed between input.output. and purpoae. Thi. cause-and-effect linkage can be e x ~ r e a a e d interma of an IF-----THEN relationship.

IF inputs are provided THEN outputs will be produced.IF outputs are produced THEN the purpose will be achieved.

The relationahip between purpose and goal ia less direct and causal.aince many exogenous factora may influence goal attainment. In thiacase. ach ieving the project purpose ia conside red neces sa ry but notlufficient fo r achieving th e go al .

IF the purp08e is achieved THEN the goal may be achieved i f othercauBal factors ar e also met.At the input-output-purpos8 levela the research manager has muchinfluence over the a tt ainmen t o f objective8. At all levels. theaaaumptions l i l ted should indicate the neceauary conditions fo r achievingthe planned objectives. Evaluators should be able to articulate clearlythe cause-effect relationship which was presumed when a given objectivewas al.igned to research. Figure 2 is an example of a Logical Frameworkused to describe a specific research activity.JHE HORIZONTAL LOGIC1be second column, Verif iab le Indicators . specifies the type of evidenceneeded to verify the achievement of objectives at each level. and thethird column, Means of Verification, indicates how that evidence can befound and measured. Both have consequences fo r monitoring andevaluation:

they define the data collection and reporting r equi rement s dur ing theimplementation of the activity (monitoring), andthey define from th e outset of an activity the standard against whichactual results will be measured (evaluation).

Indicators and their means of verification must be careful ly selected.Becauae there are costs associated with collecting and a n a l y ~ i n g data,indicators Ahould be kept to a minimum. They should:cLearly indicate the cri teria fo r attaining objectives;specify the nature, quanti ty , qua li ty , and time required fo r theobjective to be achieved;


10/15

6Fis.u rc Z LOGICAL fRAMEWORKNARRATIVESUMMARY

VERIFIABLEINDICATORS

MEANS OFVERIFICA TION

-IMPORTANT IASSUI\t1PTJONS

INPUTMaize breederMaize agronomistWeed sc ient is tSupport StaffCermplasmFacil i t iesFunds

Staff and faci l i t iesin place by end ofyear IQuarterly and annualreportAccounts and a d ~ i n i -strative reports

- annual fundir.g,;>vailable Oi l tililea:1d in accJrdancewith budget- varieties availablefrolll IlTA

- close l iaisonextensioll servicefo r on-farm testing- Experts r e c o r d ~

- Research reports- On-farm test

OUTPUT Ye.:lr 3:I , _ 2 hybrid, 2 composite- Maize varieties id - & 4 open varietiesentified resistant identifiedto Striga 3 Striga antagonist- Striga-specificherbicides identi- crops identifiedfied cultural practicesto suppress Striga- Cultural techniques identifiedidentified to sup- Striga herbicides Il-...;p;;,;r,;,;e;.;;s;.;;s;.;;;.;;;S;,;t.;;r.;.i.._a +_1_,d_e_n_t_i_f_i_e_d_a_n_d_r_a_t_e_d... .f- '"rr infestation

IPURPOSETo increase production of maize

By Year 6:- production of maizeincre3lled 40% inStriga areas- associated beanproduction increased20%

- market & productionsta t is t ics

- farm surveys- extension reports

- fert i l izers , herbicides, farm creditavailable- demand fo r maizecontinues to giveincentive to producers

GOALTo increase cashincome of ruralproducers

Changes in patternsof expenditure Farms & village :l,urveyl Policy cont inues tosupport ~ a i z e marketing

I-. .L - ....& - -J _


11/15

7be of an appropriate scale. and focus on key processes;be r.ufficient in n ~ b e r and detail to adequately measure achievementof o b . ~ t l c t i v e s ;be ludependent of the biases of evaluators;be objectively vedfiable .and unambiquoWi.

Indicators fo r th e inputs to a program are easy to determine. since theycan be e x ? r ~ s s e d in tern of ~ e s o u r c e s or act ivi t ies . such as personneltime. ~ u p p l i e s used. couraes attended. or fundo e x p ~ n d e d . The inputG att hi a s tage are \Ulually specified. and can be measur:e':l or assess'!!ci.verifying that implementation is proceeding as planned requires t.1:ackingactua l input s aeainst proposed inputs. in a given t i frwwc. forinstance by keepifig logs of stlliff time and act ivi t ies undertake.; Monitoring progr&" leadership llnd sound progr.:mning procedures is IIlOredifficult and must be dealt with in more qualitative ways. such asthrough peer. reviev and standard reporting.When selecting indicators of the outputs level. i t is h elp ful to thin k ofthe expected output and purpose of the activity in term of targets,answering the questions of what. how many. with which character is t ics .when. If one of a program's expected outputs is a new variety of maizewhich permits double cropping and higher. yields. then an appr?priateindicator might be the certification of a variety by year 7, which haa a90-day cycle. and which yields more than 2 tons/ha under farmconditions. The means of verif ication in this case would be recorda fromexperimental t r ials, results of on-farm testing and verif ication, andrecords from lh e v a r i e ~ a l certification boards.At the input-output-purpose levels of inquiry, document& of programplanning meetings. quarlerly and annual research reports, researchproposals, survey results , and scientific publications can be U 8 ~ d toevaluate research p r o g r 2 ~ execution. In an ideal system these reportswould b a v ~ been routinely gathered ~ n monitored by researchers andmanagement to identify implementat ion problems. Table 2 summarizes someof the indicatora auitable for research program MIE and their means ofv e r i f i c ~ t i o n . This table is by no meana exhauative; i t is sUlgeated as al iat which may guide research managera in defining an appropriate l ia tfo r their syatema.The laat column, Important Asaumptions. l i s t s thoae factora which are notcontrolled by the project bu t which influence i ts implementation andchances for succesa . For example. fixed national commodity price. couldinf luence the purpoae to goal relationship by making maize productionunattractive. eVftn i f better technology w e r ~ available. Assumptiona atthis level a re o fte n difficul t to influence, but they should be definedin advance and monitored.The as.umptions column is meant to keep deciaion makers realiat ic intheir expectationa; if a situation looks particularly hopeleas, theseleaders should reorient their research programs to take th ia in toaccount. Sometimes. where national pol ic ie s a re concerned, researchmanagers can be succesafully involved in pol ic y d ial ogue to ensure that


12/15

8Table 2: Example. of Rearch Program Indicatora~ o achievelllllnt Pouibleindicator Heana ofverification Re.ponaibi li ty fordata coLlectionINPUTS - Detena ined by projecto. ba.ed on operation workplana:

1

- I ~ r a o n n e l- fundin.- faciUtiell- equ1 s-ent" aupplle.- le&derabip- trainina

adenti t ic and.upport tbleexpendituresconatruction oracquiaitionacquiaitlonactual uaeproject meetingapro.rlllll _et ine .c o u r a e ~ co.pleted

tiM aheetaaccoWlting dataon-/Jite reportprocurement datap r o c u r ~ m e n t dat .lab/ataUon logameeting reporta!lIeetin, report.trainin, record.

individual reportaaccoWltin, officein.t i tute en,ineeraccoWltin. officea c c o u ~ t i n . officelab/atation ..na.erproject headpro;ru headtraining officer

OUTPUTS - Conaidered both by projecta and by progr... :- prelblinary reaearch data reaearcb report. sc i .n t i l treaearch fro. experiments publication. project beadreaulta and .urvey.- ca.pleted pro,ram committee . , ro, r_ recorda program headre.earch recollllllendationa "'nnual reporta HARS directorreaulta- reaearch trained ~ r a o n n e l training recorda training officercapacity " improved auiniatration adminiatratorillproved fadUt ie . recordaPURPOSE - C o n t r i b u t i o ~ of knowledge from reaearch program. to reaearch.d e v e l o ~ n t and policy making bodie.:- IUIW knowled,eof intere.tto reaearcb,extenaion "pol iCYllllkera

relea.edtechnOlogy orrecolllllleodationaprogram rec("rd.certif iC:1tionrea/extenaioncOlllllWlicationaon policy

pro,ram headnational bodyexten.ion aerviceHARS director

GOAL - Reaearch relationship to national development objectivea:- increaaed cropproduction- intenlifiedland uae- conaervation" ldnd use- increaaedincome- improvednutrition

productIon datachangea in croppatterns " inputareduced eroaionreaource planningpe r capita changeincreaaed spendingdecreased diaeaae" IIIOrtali ty

farm surveysinput ata th t iesaurver methodaplanning docwmentnat iona l datavillage surveysnutrition aurveya

statiatica dept.devel . minia tryland use bodyplanning bodystat is t ics dept.de.. . '-. minbtrynationa.\ healthservice


13/15

9an asaumption comes to pass. Assumptions are particularily important fo rresearch managers a t th e input and output levels. where the l i s t ofassumFtions seeves as a red flag to management that they must activelymonitor and assure that th e conditions l is ted are achieved.While r e a e a r ~ h m a n ~ g e r s are primarily interested in i n p ~ t . output andpurpoae level information. ex ante and impact e v a l u a t i o n ~ are c o n ~ e r n e dwith the relationship of research program6 to larger developmentobjectives, and so the entire framework is useful. The primary purposefo r conducting any a n a l y ~ i s at this level is to squarely understand th eexpectations placed upon th e national r e s e a r c ~ s y s t ~ m , th e validity ofthese expectations. and whether the research program6 planned andoperating in country are logical responses to ~ ~ e s e expectations.Figure 3 indicates how the Logical Framework matrix can be usedspecifically as a monitoring and evaluation tool. The targets asainatwhich performance i s measured is found in th e Veri fiable Indicatorscolumn. The actual data monitored are in th e Means of Verificationcolumn fo r each level of the management hierarchy. The assumptions ar eusually better defined and more "manageable" ,at th e inputs and outputslevels than at th e purpose and goal levels . and can. therefore. be moreeasily monitored and evaluated.Project evalustors are primarily interested in th e targets set and th eassumptions made at th e input and output levels . Project p e r f o ~ n c e(efficiency) is th e primary focus though th e quality and r el evance of th eresearch may also be reviewed. Comprehensive program evaluations areconcerned with program strategies and th e achievement of programobjectives, and a re th er efo re more interested in purpose levelachievements. Project complementarity within th e program is alsoconsidered.Impact evaluations. or the effect research has on national developmentobjectives. ar e most concerned with thos e indicators monitored at thegoal level. These are usually socioeconomic in nature. more expensive tocollec t , and analyzed 10-15 years after the technology fromreaearch hasbeen released.In summary. the Log ical Framework is an effective tool fo r researchmanagers fo r both th e planning and evaluation of research. By followinga logframe approach, managers are apt to more thoroughly consider th eresources needed, the t ime frame of the research. t he expec ted targets.and the conditions assumed necessary fo r th e research to be successful.In addition, th e Logical Framework places the research project in alarger framework of program and national research objectives. thusincreasing th e likelihood that research projects will be complementarywithin programs. and that the}' ....i l l address important. defined nationalresearch objectives.


14/15

ifgure 310

LOGICAL FRAMEWORKNarrative l Verifiable I Means of ImportantSUIIlIIlary I Indicators Verification AssumptionsINPUT

Specified time- Data to be Assumptions toframe &esources collected be monitored!

managed

OUTPUT~ e c t Evaluatio?Project efficiency Targets Data to be Assumptions to&effectiveness (annual & final) collected monitored!

managed

PURPOSE~ r e h e n s i v eProgram EvaluationProgram strategy Targets Data to be Assumptions to&achievement of (3-10 years) collected be monitored!objectives managed

Consider projectcomplementari ty

GOAL~ a c t EvaluationContribution to Targets Data to be Non-researchDevelopment collected factors affect-Goals ing impact

I


15/15

~ i ( --REFERENCES

Agency fo r International Development. "Design and Evaluation ofAID-Assisted Projects", Training and Development Division, AID,Was hington , D.C. 1980.Mut'phy, Josette. "Monitoring and Evaluation of Agricu ltural Research,Concepts, Organization, Methods." draft. International Service fo rNational Agricultural Research, 1985.

logframe for scientific program planning and evaluation 1988

Documents