A Heuristic Tool for Building a Balanced D-MERL System in a Post response Recovery

AboutthistoolGiventheemergentnatureofprogramminginapostresponserecoverytransitionperiod,D-MERLplaysavitalroleinevolvingprogramstrategyandenablingcollaboration,learningandadapting.Insuchcontexts,D-MERLsystemsshouldbeflexibletoaccommodatetheprogram’sneedsforurgentactionandallowforiterationsofmajorD-MERLproductstotakeplaceovertimeasconditionschange.Forexample,aspectsoftheD-MERLsystemmayneedtochangeorbeiterated,afterreviewofinitialprogramresultsoroncenewpartnersorprogramcomponentsareaddedon.However,D-MERL“stakesinthesand”needtomarkeachstage, so that implementation and other D-MERL activities can proceed without delays in planned andcoordinatedwaysandnotad-hoc.

ThepurposeofthisdocumentistohelpUSAIDstaffplanforandimplementeffectiveandefficientprogramsandMERLsystemsinapostresponserecovery.ThisheuristictoolisquickreferencedocumentdevelopedtoassistprogrammanagersandMERLpractitionersnavigatetheprocessofbuildingthebalancedD-MERLsysteminthisevolvingcontext.

SixBuildingblocksofabalancedD-MERLsystemandtheircomponents

ThefollowingarethesixbuildingblocksofaD-MERLsystemandcorrespondingframingquestionfacedbystaff:Buildingblock1:Partnersandcollaboration

• HowcansuccessfulpartneringandcollaboratingintegrateMERLandadaptivemanagement?

Buildingblock2.Programstrategy-thebigpictureplanning

• IstheprogramTheoryofChangerealisticallydefinedanddoesitreflecttheinputsandbuy-inofallkeystakeholders?

BuildingBlock3.ResultsframeworksandMERLplans

• Are IP results frameworks andMERLplans appropriately coordinatedandharmonized to informaprogram-levelresultsframework?

BuildingBlock4.Reportingsystem

• Doestheprogramreportingsystemmeetinformationandknowledgeneedsinthemostefficientandeffectivemannerpossible?

BuildingBlock5.Data-basedtargetsetting

• Whatmattersandhowmuchisneededtomeasuretimelyprogresstowardresults?

BuildingBlock6.Performancemonitoring,evaluationandlearning

• Whatopportunitiesarethereforlearningandadaptivemanagement?Each of these blocks requires implementation ofmultiple components that are necessary for successfulcompletion.Thefollowingfigurepresentsspecificsubcomponentsofeachbuildingblockagainstthegeneralimplementationtimeline.

ApathwaythroughimplementationofbuildingblocksInthesequencingprogressionofD-MERLbuildingblocks,somesubcomponentscantakeplaceconcurrently.Inordertodososuccessfully,programscanplaceD-MERL“stakesinthesand,”sothatD-MERLactivitiesproceedinplannedandcoordinatedways,ratherthanadhoc.AspectsoftheD-MERLsystemmayneedtochangeoriterateafterreviewofprogramresults.However,thisisnotareasontodelayimplementingtheD-MERLsuccessstrategies.

Thefollowinggraphhighlightsatimelineofahypotheticaltwoyear-longprogrampostresponserecoveryprogram. It illustrates general heuristics to show which building blocks start when and demonstratespossibleoverlaps.

Foreachbuildingblock,thesummarytablesbelowpresentadditionaldetailsthatinclude:

• Timelineforwheneachbuildingblocktakesplace• Interdependenciesthatillustratetheconnectionsbetweenthesebuildingblocksandhowsomeare

prerequisitetoothers• Trade-offswhichrepresenttheconsequencesthatUSAIDstaffmightanticipateifthisbuildingblock

isnotaddressed,and• Informationneededtobuildtheblocksuccessfully

IconLegend:

-Timeline; -Interdependencies; -Trade-offs; -Informationneeds

BuildingBlock1.PartneringandCollaborationThesuccessofpostresponserecoveryinterventionsdependsonD-MERLsystems-basedsolutionsthataddressandintegratebothshortandlonger-termdevelopmentobjectives.

Addresscomponentsofthisbuildingblockwithinthefirst3monthsofimplementation

Thisbuildingblockisfoundationaltoallotherssotherearenodependencies.

• Learningcannotbeconsolidatedacrosscomponentsoforthewholeprogramunlessitisproactivelyplannedandexecuted.Opportunitiesforneededchangestotheprogrammaybemissed.

• WithoutalignmentofD-MERLexpectations,understandingofcapacities,alongwitharticulationofD-MERLrolesandresponsibilities,CLAmaybeadhocanddisorganized.Thiswouldreduceitsefficacyandreach.

• D-MERLcompetenceandknowledgemanagementcapacity.• ExpectedkeyrolesanddedicatedMERLstaffingfortheprogram.• Learningtopicsandagendafortheprogram.

BuildingBlock2.Programstrategy-thebigpictureplanningPartnersneedtoelevateplanningtothebigpicturelevel.Inordertodothis,allstakeholdersmustdefinetheTheoryofChange(ToC)withrealistic inputsandachievebuy-in.Effectivepostresponserequiresclearlydefined,yetflexible,programstrategies.

Completeallelementsoftheinitialprogramstrategywithinthefirst90-120daysofimplementation.Thisincludesconsolidatedprogramapproach,assumptionsandriskassessmentsandmitigationstrategies.ProgramstrategycanrunconcurrentlywithpartneringCLAactivities.Tofurtherexpeditetheplanningandimplementation,partnersalreadyoperatingonthegroundshouldbeincludedinCLAefforts,toincorporatetheirlearningsandhelpthemadaptasnewinformationbecomesavailable.OnceanagreementisreachedontheToCamongpartnersandkeystakeholders,buildingresultsframeworksandMERLsystemcanbeginandrunconcurrentlywiththeremainingactivitieswithinthisblock.

ProgramstrategyisfoundationaltoallotherdesignandMERLactivities.Efficiencyoftheprocessanddevelopmentofsynergiesacrossprojectswithintheprogramdependsinpartontherelationaldimensionsofthepartnershipandlevelsofcollaborationestablishedbythispoint.

• WithoutaconsolidatedprogramToC,theoutcomeindicatorstowhichallimplementingpartnerscontributearecompromised.Thismayresultinmissedopportunityforbaselinemeasurement,useofinvalidindicators,andsubsequentreporting,learningandperformanceevaluationchallenges.

• Withoutassumptionandriskanalysis,importantcluesintotheprogramdriversbehindefficiencyandrelevancemaybemissedandnotsufficientlymonitored,resultinginsubsequentchallengestoadaptivemanagementandperformanceevaluation.

• Relevantorsimilarprograms,bothpastandcurrent,implementedinasimilarcontext.• Needsandvulnerabilitiesofdirectbeneficiaries,withconsiderationtogenderand

socialdynamics.

• Driversaffectinglevelsofresilience.• Institutional,communalandhumanresourcesavailable,includingsocial,financial,and

naturalcapitalthattheprogramcanleverage.• Relevantsecondaryandtrenddata.

BuildingBlock3.ResultsframeworksandMERLplansThefoundationforperformancemonitoringsystemsisaresultsframework.Planningshouldanswerthreequestions:Arewedoingtherightthings?Arewedoingthingsright?Howdoweknow?

Oncetheprogramoutcomesandoutputsaredefined,workcancommenceondefiningtheKPIsattheprogramlevel.IdeallymostKPIswillbedefinedandreferenced(PIRS)priortooperationalizingthereportingsystem.

Thisblockdependsonprogramstrategy.WhileitispossibletoidentifymanyKPIsbasedonthe basic ToC for the program, completion of assumptions and risk analysis is key toarticulating a comprehensive results framework and set of indicators, including contextmonitoring.

• IfcommonKPIshavenotbeenestablishedataprogramlevel,thenearlyandvaluableinsightsintothenearandmedium-termoutcomescanbelost,ataminimum.

• Differentstakeholdersmayrequiredifferentreporting.Withoutawell-articulatedresultsframework,requirementsforreportingtosomestakeholdersmaynotbeabletobemet.Thisisparticularlyimportantifpotentiallydisenfranchisedstakeholdersarekeyprogramdecision-makers.

• LackofdefinedKPIsnegativelyimpactstheconfigurationandimplementationoftheprogram’smanagementinformationsystem

• Implementingpartnerstrategiesandprojectplans.• Partnerandcontractrequirementsforreportingtofunders.• Analysisofcurrentsystemsusedforreporting,aswellascontextualtechnologyassessmenttoinformdecisionsonsystemsinvestment.

• Datacollectionresourcerequirementsandotherconsiderations,human,financialandother.

BuildingBlock4.Reportingsystem

ThegoalofperformancemanagementsystemsistoproducetrustworthyinformationandknowledgethatpartnerscanemployforCollaboratingLearningandAdapting(CLA)andreporting.Thereportingsystemshouldincludeaccurate,detailedandcompliantreportingrequirements.

ThisworkcancommenceonceKPIsaredefinedandcompletedbeforethefirstreportingperiodandmajorreviewoftheprogram.

Reliable,accurateandtimelyreportingonprogramoutcomesandcontributionsdependsonaconsolidatedprogramstrategyandaresultsframeworkwithKPIs,PIRS,understandingofrolesandresponsibilitiesandoverallD-MERLcompetence.

ReportingcanbeopentointerpretationwithoutdetailedPIRSforeachindicatorandstandardizedguidanceacrosstheprogram.Unlessclearreportingguidanceisgivenfromprogramlevelmanagers,implementingpartnerswillstartreportingusingtheirown

templatesandformats.Lateronthisbecomeshardertoaggregateandintegrateforthepurposeofprogramreportingandthereforecanrequireconsiderableefforttoremediate.

• DatafromKPImonitoring.• Clarityonstakeholdersandtheirpreferreduseofsystemsandreports.• Systemscurrentlyinuseandunderstandingoftheimplicationsofchangingprocesses.• UserrequirementsandavailablebudgetforaPMISsystem.

BuildingBlock5.Data-basedtargetsettingEstablishingcleartargetskeepsimplementingpartnersfocusedonwhatmatters.Definingandtrackingtargetsforeachindicatorfocusesattentiononresultsratherthanday-to-daymanagementandlogistics.

AsKPIsaredefined,planningforacquisitionofbaselinedataandestablishingtargetscancommenceandwithgoalofcompletionbeforethefirstmajorprogramlevelreview.

• Baselineandtargetingrequireagreeduponandwell-definedresultsframeworksandKPIs.

• Likewise,establishingbaselines,targetsandexpectedmilestonesbringsfocustoplannedprogrammonitoring,reviews,ongoingriskmanagementandfutureprogramevaluations.

• Someformofabaselineassessmentisnecessarytounderstandthestatusquoofacontextatcommencementofanactivityandsubsequentachievements,butmaynotalwaysrequireprimarydatacollection.Ifnotdoneattheonsetoftheprogram,reconstructingbaselineslateronwillrequireadditionalresourcesandmayintroducebiasesandlimitations.

• Targetsandmilestonesgroundtheperformanceplansaroundrealisticexpectationsofachievementandsetthestageforlearningandinquiringaboutnegativeandpositivedeviancefromexpectedresults.Inapostresponserecovery,itislikelythatplanningandimplementationconstraintswillimpactachievements.Realisticandwell-informedtargetspreventthelossofimportantreferencepoints.

• Understandingofcontextandexistingprimaryandsecondarydata.• Thelearningagendafortheprogram.Inotherwords,whatquestionsdoestheprogram

wantanswered?• Methodologiesfordatacollectionandanalysisplans.

BuildingBlock6.Performancemonitoring,evaluationandlearningRegularperformancemonitoringcontinuesthroughouttheentireperiodofprogramperformance.Thesemechanismshelpprogrammanagementevaluatetheopportunitiesforlearningandadaptivemanagement.Ongoingevaluationsdeterminewhenmitigationisontrackorwhenachangetotheprogram’sdirectionisrequired.

PerformancemonitoringcannotcommencebeforeKPIsaredefinedandreferencedthroughpreviousblocks.Ideallycompletedbaselineswillalsoexist,althoughthisisnotabsolutelynecessary.Thisisanongoingactivitythroughoutthelifeoftheprogramanditscorrespondingprojects.Evaluationscanoccuratvariouspoints,mostcommonlyhalfwaythroughand/ortowardtheend,dependingonthedurationoftheperiodofperformance.

Comprehensiveperformancemonitoringandevaluationisdependentuponalltheotherbuildingblocks.Ifdonewellandcomprehensively,itprovidesthedataandinformationrequiredto

managetheactivityadaptivelyandthereforecanlinkbacktoprogramstrategy,resultsframeworksandindicatordefinitions,andalsoinformtheefficiencyofthereportingsystem.

• Itisassumedthatpostresponserecoveryprograms,particularlythoseinprotractedcrises,areemergentandthereforeneedadaptivemanagement.Thisrequiresefficientperformancemanagementsystems,anevaluableprogramandcompletedevaluations.Withoutthese,learningwouldatbestberetrospectiveandunsustainable,andwouldmostlikelybeintuitive,whichisoneoftheleasteffectivemethods.

• Whoprimaryintendedusersare,theirinformationneedsandpreferredinformationdeliverymechanisms.

• Purposeofeachevaluationandhowtheywillbeusedsothatexercisescanberationalizedtomaximizereturnonresources.

• Expectationsformonitoringandevaluationprocessesandquestionsthattheyneedtoanswer.

• Timeframeandbudgetavailabletosupporteachtaskascostofdatacollectioninsomeprograms(suchashealth)arehigherthaninothers.

• Whoarekeyinformantsandhowwhatarethebestengagementmechanisms.• Limitationsofstudiesanddatacollection(thingsoutsideofprogram’scontrol).

Illustrativeworkplan

ThisillustratedtimelinesuggestsaworkplanforbuildingabalancedD—MERLsysteminahypotheticalpostresponserecoveryprogramoftwoyearsormore.Shorterprogramsshouldadjustthetimelineaccordingly.