mid-term review of ideas to impactideastoimpact.net/sites/default/files/doc_research... · 1. ideas...
TRANSCRIPT
Mid-TermReviewofIdeastoImpact
ConductedforDepartmentforInternationalDevelopmentResearchandEvidenceDivision24February2017
TheLaw&DevelopmentPartnershipLtd
1-3StapleInn,LondonWC1V7QJ
T:+44(0)2076366856F:+44(0)2079006886
Mid-TermReviewofIdeastoImpact
Page2of68
Contents
Executivesummary.......................................................................................5IntroductiontoIdeastoImpact.........................................................................................................5
MTRpurposeandapproach...............................................................................................................5
Cross-prizefindings............................................................................................................................5
Programmemanagementfindings....................................................................................................7
Strategicrecommendations...............................................................................................................9
Chapter1:Background................................................................................11TheIdeastoImpactprogramme......................................................................................................11
PurposeoftheMTR.........................................................................................................................17
Approachandmethodology............................................................................................................17
Chapter2:Programmelevelorcross-prizefindings–fivekeythemes.........19Prizes as a catalyst for innovation and problem solving: I2I is effectively testing its core
hypothesis.............................................................................................................................19
Outreach and participation: I2I has drawn in solvers ‘beyond the usual suspects’ but shouldcontinuetomitigatesolverrisk............................................................................................20
Prizesecosystems:I2Ishouldcapitaliseonecosystemstrengthsandaddressdeficits(particularlysolvercapacitygaps).............................................................................................................21
Innovative partnerships: I2I demonstrates that prizes can incentivise coalition building around‘thorny’developmentproblems...........................................................................................22
Balancing research and application: I2I should clarify relationships between prizedesign/implementation,evaluationandwiderknowledgesharing.....................................23
Chapter3:ClimateInformationPrize...........................................................25Background......................................................................................................................................25
Headlines.........................................................................................................................................25
Applicationandaward.....................................................................................................................26
Outreachandparticipation..............................................................................................................27
Prizemanagement...........................................................................................................................28
Partnershipsandcoordination........................................................................................................29
Valueformoney...............................................................................................................................29
Impactandsustainability.................................................................................................................30
Strategicrecommendations.............................................................................................................30
Mid-TermReviewofIdeastoImpact
Page3of68
Chapter4:AdaptationatScale....................................................................32Background......................................................................................................................................32
Headlines.........................................................................................................................................32
Design,judgingandaward...............................................................................................................33
Outreachandparticipation..............................................................................................................33
Prizemanagement...........................................................................................................................35
Monitoringandlearning..................................................................................................................35
Partnershipsandcoordination........................................................................................................36
Valueformoney...............................................................................................................................36
Impactandsustainability.................................................................................................................37
Strategicrecommendations.............................................................................................................37
Chapter5:UrbanSanitationChallenge........................................................39Background......................................................................................................................................39
Headlines.........................................................................................................................................39
Application,judgingandaward.......................................................................................................40
Outreachandparticipation..............................................................................................................41
Prizemanagement...........................................................................................................................42
Partnershipsandcoordination........................................................................................................42
Impact,sustainabilityandvalueformoney.....................................................................................43
Strategicrecommendations.............................................................................................................44
Chapter6:DreampipeII..............................................................................45Background......................................................................................................................................45
Headlines.........................................................................................................................................45
DreampipeIandredesign................................................................................................................46
ImplementingDreampipeII.............................................................................................................46
Prizemanagement...........................................................................................................................47
Strategicrecommendations.............................................................................................................48
Chapter7:GlobalLEAPRefrigerationPrize..................................................49Background......................................................................................................................................49
Headlines.........................................................................................................................................49
Partnershipsandcoordination........................................................................................................50
Outreachandparticipation..............................................................................................................50
Mid-TermReviewofIdeastoImpact
Page4of68
Prizemanagement...........................................................................................................................51
Monitoring,evaluationandlearning...............................................................................................51
Strategicrecommendations.............................................................................................................52
Chapter8:Programmemanagement...........................................................53Governanceandmanagementarrangements.................................................................................53
Valueformoneyandfinancialperformance...................................................................................54
Chapter9:Strategicrecommendations........................................................56Programmeduration.......................................................................................................................56
Targetingprogrammeinvestments.................................................................................................56
Opportunitiesforscalingandreplication........................................................................................57
AnnexA: StatusofI2Iprizes.....................................................................59
AnnexB: Documentsconsulted................................................................62
AnnexC: Interviewees..............................................................................64
AnnexD: Exampleinterviewprotocol.......................................................66
Page5of68
Executivesummary
IntroductiontoIdeastoImpact
1. Ideasto Impact(I2I) isafiveyear,£10.9mprogrammewhich istestingtheapplicationof innovationprizesasafundingmechanismforcatalysinginnovationonintractableproblemsintheenergy,waterandclimateadaptionsectors.
2. IMCWorldwideleadstheprogramme,managingaconsortiumwithwide-rangingexpertise.I2Iisstructuredaround five currently operational prize programmes, most of which are individually managed by animplementing agent, with cross-cutting support provided by prize designers and Itad on evaluation andlearning.
MTRpurposeandapproach
3. ThisMTRtakestheformofanoperationalreviewtoguidetheI2Iprogramme’sfuturestrategicdirectionandresourceallocation,aswellastoimprovevalueformoneywherepossible.Specifically,itseekstoaddresssixmainquestions:
§ Isthelevelofmanagementappropriateacrosstheprogramme?
§ Couldgovernanceandriskmanagementsystemsbeimproved?
§ Istheinvestmentinevaluationandresearchappropriate?
§ Howcanvalueformoneyacrosstheprogrammebeimproved?
§ HowcanDFIDandtheprogrammeenhanceimpactandensuresustainability?
§ Whereshouldfutureinvestmentsintheprogrammebetargetedand/orrestricted?
4. TheMTRhasusedqualitativemethodstoreviewI2I’sactivitiesacrossitsprizeportfolio.AnextensivereviewofprogrammedocumentsandI2Iproducts,aswellasprizeliterature,hasbeenundertakenandaseriesofsemi-structuredinterviewsconducted,includingfieldworkinGhana,KenyaandNepal.Datacollectionwasconductedaccordingtoacommonframeworktofacilitatecross-prizecomparisonsbutadaptedaccordingtothespecificobjectivesandcharacteristicsofeachprize.
Cross-prizefindings
5. Attheprogrammeorcross-prizelevel,theMTR’sfindingscentreonfivekeythemesormessages,asfollow:
Prizesasacatalystforinnovationandproblemsolving–I2Iiseffectivelytestingitscorehypothesis
§ I2Ihassetahighlevelofambitionbydeliberatelyselectingintractableproblemsaroundpathwaysto use (i.e. market/systems change). Valuable lessons have been learnt around rendering these‘softer’ problems ‘prizeable’ – that is, defining the parameters of problems to induce thedevelopmentofinnovativesolutionswhichcanberewardedviaaprizemodality.
Page6of68
§ Prizes are effectively incentivising the development and deployment of innovativesolutions,andareona trajectory todeliveringa rangeof ‘prizeeffects’.Theexactnatureof thisincentive varies by solver, but relates asmuch to recognition associatedwith the prize as prizewinningsthemselves.However,theprogrammehassofarhadrelativelylittleopportunitytocollectrobustevidencearoundthevalueaddedbytheprizemodality.
§ Although not a comparative exercise, MTR findings suggest that prizes may offer added valuerelativetootherfundingmodalities–e,g,grants,loans,equityinvestments–vis-visreachingandincentivisingnewtypesofsolverthroughcompetitionandrecognition,andactingascatalystswhichbuild on and accelerate the momentum of existing enthusiasm and/or initiatives. However,comparedtomoreconventionaltypesoffunding,workingthroughprizesentailsadditionalcostsassociatedwiththe innovativenatureofmodality itself–not leastthe levelofeffort involvedinbuildingunderstandingofthemodalityamongstpotentialparticipants.
§ InfutureDFIDmightconsiderhybridfundingmodalitieswhentackling‘softer’problemsorworkingwithsolverswithmorelimitedcapacity–drawingonknowledgegeneratedbyI2Iastowhenandwherethecatalyticeffectoftheprizemodalityismosteffective.
Outreachandparticipation–I2Ihasdrawninsolvers‘beyondtheusualsuspects’butshouldcontinuetomitigatesolverrisk
§ Inducement prizes are argued to offer a means of reaching a wider group of participants thantraditionalfundingmechanisms.I2Iistestingthishypothesiseffectivelybytargetingawiderangeofdifferent solver types (government, CBO/NGO, SME). The programme team have, however,concludedapurefinancialincentivetobeinsufficienttoinducechangebylargerprivatesectorfirms,whoarealreadyprofit-driven.
§ Whether prize design has accurately anticipated the resources needed of participants and set areasonable risk/reward curve will be demonstrated in prize Stage 2s, where the (financial andopportunity)costsofentryaremuchhigher.I2Ishouldcontinuetomitigaterisksforapplicantsbyminimisingcostsandincreasingpotentialreward(e.g.byfacilitatingengagementwithothersourcesoffinance).
Prizeecosystems–I2Ishouldcapitaliseonecosystemstrengthsandaddressdeficits
§ I2Ihasattempted tonestleprizeswithin contextswithhigh levelsofexistinggovernment/donoractivityandsupportinfrastructure,withpositivebutvaryinglevelsofsuccess.Themaindeficitsinprizeecosystems1centreonsolvercapacitygaps.ItwillbenecessaryforI2Itoengageinadditionalsupportive activities for solvers to mitigate delivery risks, ensure impact and sustainability, andmaximisevalueformoney.
§ Prizesshouldensurethattheyarebuildingonthestrengthsoftheprizeecosystemsinwhichtheyareoperating–forexample,by leveragingtheresourcesofexistingprogrammesand institutionsworkinginthesectororengagingwithnationalpolicydevelopmentprocesses.
1Prizeecosystemisdefinedasresourcesavailabletosolvers(andhencepotentialbarrierstoaccess).
Page7of68
Innovativepartnerships– I2Idemonstrates thatprizescan incentivisecoalitionbuildingaround‘thorny’developmentproblems
§ I2Idemonstratesthepotentialvalueofusingprizestoincentivisediverseactorstobuildcoalitionsaround development challenges, and is testing a range of donor, public-private andsupplier/financierpartnerships.
§ Whereprizeshavenotyet facilitated suchpartnerships, this representsa risk todelivery–mostnotablytothesustainabilityofsolutionswheresolversarenotwell-networkedtoleverageadditionalinvestment.
Balancingresearchandapplication–I2Ishouldclarifyrelationshipsbetweenprizedesign/implementation,evaluationandwiderknowledgesharing
§ I2Imay have been incentivised to place too great an emphasis onmaximising direct beneficiarynumbers.A‘litmustest’forprizescouldbethathighbeneficiarynumbersarenotpursuedattheexpenseofopportunitiestoinducethedevelopmentofgenuinelyinnovativesolutionswithpotentialforlargescale,albeitindirect,impact.
§ Staggered interventions have enabled valuable ad hoc sharing of lessons amongst prize teams.However,endofstageevaluationsof individualprizescouldhavebeendeliveredmorerapidlytofeedintoiterativeprizedesign.
§ MoreattentionandresourceshouldbedevotedtoI2I’swiderknowledgesharingandinfluencingagenda.Learningproducts shouldcapture learning relevant to theprogramme’scentral researchquestionsbutalsooperationallessonsaroundmanagingprizesinthedevelopmentsector.
Programmemanagementfindings
Governanceandmanagementarrangements
6. Leadership andmanagement of theprogramme is very strong. Relationships amongst the PMU, in-houseIMCprizemanagers,consortiumpartnersandimplementingagentsareopen,collaborativeandeffective.
7. I2Ioperatesadaptively,withDFIDprovidingeffectivesupporttothisapproachbyofferingadegreeofflexibility on management arrangements, whilst retaining effective oversight of implementation.Successful examples of adaptation in response to lessons learned and contextual changes includedecisionstoclosedowntheGhanaCleanCookingprizeandredesignDreampipe.
8. However, both working iteratively and through local implementing agents has necessitated highermanagementcoststhanoriginallyenvisaged.
9. Theprogrammecoulddomoretocaptureandsharewithanexternalaudiencelessonsaboutworkingadaptivelyintheinnovationsectorandonprizesspecifically.
Page8of68
Riskmanagement
10. Managersatalllevelsdemonstrateasoundawarenessofmajordeliverrisks,whichtendtocentreon:operating environment (e.g. changes to political environments, security and corruption concerns,naturaldisasters);insufficientsolverinterest(lowregistrationnumbers,poorqualitysubmissions);andparticipantattrition(asaresultofe.g.misjudgedrisk/rewardcurve,inadequatesolvercapacity).
11. Risk ismanaged effectively at prize level,with teams giving careful consideration to the scope andappropriateactions formitigation.Riskmanagementprocessescouldbe formalisedandcentralised,with risk currently owned by prize managers and not reviewed as part of regular programmemanagementprocesses.
Valueformoneyandfinancialperformance
12. Economy: although management costs have been higher than originally envisaged, the team haveworked hard to maintain the proportion of budget (30/70) allocated to programme managementrelativetotheprizepurseandotherin-countryactivities.
13. Efficiency:Theinterfacebetweenprizemanagersandimplementingagents(levelofoversightneeded/ delegation appropriate) is the primary driver of prize efficiency. The global prizes offer themostefficient management arrangements, whilst valuable lessons have been learnt around the relativeefficiencyofrecruitingin-countrystaffandcontractinglocalimplementingagents.However,efficiencymaydecreaseastheprogrammeengagesinincreasedsolvercapacitybuildingactivities.
14. Effectiveness:Certainmeasuresofprogrammeeffectivenesssuggestedbyprogrammedocuments–forexample,relatingtovariancebetweenforecastedandactualbudgets–maycreateunhelpfulincentivesfor the programme given its adaptive mode of working. Effectiveness should be measured morestraightforwardly,withreferencetologframeoutcomes.
15. O1)thedevelopmentanddeploymentofinnovationsolutionsandachievingofpositiveoutcomesforbeneficiaries, will be best demonstrated through evaluations’ measurement of prize effects, withcompletedStage1evaluationspromising.Moreattentionshouldbedevotedtoknowledgesharingandinfluencing to ensure that the programme to ensure that the programme delivers against O3)influencingotherstoimplementprizes‘better’andO4)developingarobustevidencebasecentredonprogrammelearningstreams
16. Equity:Women’srepresentationamongstsolversvariesconsiderablyacrossprizes.InKenya’sClimateInformationPrizewomensubmittedalowernumberof,butonaveragestronger,applicationsthanmen,withaconversionrateof32%(winnersasapercentageofcompletedapplications)comparedto7%formen.Bycontrast,inthecaseofNepal’sAdaptationatScalenoneofthe15prizewinnerswerewomen’sorganisations,withthemainbarrierscapacitylimitationsandpersistentsocio-culturalbarriers.
17. Thereisemergingevidencethatsolutionsinducedacrossdifferentprizesaregearedtowardsbenefittingpoorandvulnerablegroups.BothCIPandA@Ssolutionsareworkingthroughwomen’sgroupstotacklegenderspecificvulnerabilities,aswellastostrengthenwomen’svoiceanddecision-makingpower.
Page9of68
Strategicrecommendations
Programmeduration
1.1 ItisrecommendedthattheoveralltimeframeoftheI2Iprogrammebeextendedbyninemonths(threemonths beyond the current contract end date). Specifically, Adaptation at Scale, Dreampipe II andEnergyAccesswouldbenefitfromsixmonthextensionstoenablesufficienttimeforthedevelopmentanddeploymentofsolutions,anddemonstrationofpositiveoutcomesforbeneficiaries.ThequestionofanextensionshouldalsoberevisitedfortheUrbanSanitationChallengeshoulddelaysarisingfromthepost-electiontransitioninGhanaprovesignificantoverthecomingc.3months.
1.2 ThiswouldbringthedateforthefinalclosureofprizestoJune2019(ratherthanDecember2018),afterwhichtheevaluationteamwillneedninemonthstoconductplannedevaluationandresearch.Giventhat three months are already provided for in existing programme timelines, this would bring theprogammeenddatetoMarch2020,sixmonthsafterthecurrentcontractenddate2–i.e.atotal(no-cost)extensionofninemonths.
Targetingprogrammeinvestments
18. Itisrecommendedthat,atprizelevel,theprogrammeinvestsavingsfromthereducedprizepurseinthoseareaswhichmitigatemajorriskstoStage2delivery–toensurethatprizesachievesustainableimpactandmaximisevalueformoney.Prizelevelfindingssetoutspecificsforeachprize,butsolvercapacitybuildingisaconsistenttheme–notleastsupportonengagingwithpotentialfunderstoleveradditionalinvestment.
19. Attheprogrammelevel,I2Ishouldprioritiseinvestmentin:
1) Knowledgesharingand influencing– includingthegenerationanddisseminationofbothprize(e.g.successfulinnovations)andprogramme(e.g.operatinginnovationprizesfordevelopment)levellearning
2) Scoping for scale up / replication of specific prizes where concrete opportunities presentthemselves,withtheUrbanSanitationChallengethestrongestcontender.
20. TheMTRsupportsexistingevaluationplanstomeasurespecificprize‘effects’aswellastheadditionalityoftheprizemodalityasallowedbytheexistingbudgetforevaluation.Increasestothisbudgetcouldbeconsideredforthoseprizeswherethereisstillscopeforflexible,responsiveevaluationsupporttofeedintoStage2design(DreampipeIIandtheGlobalLEAPrefrigerationsisterprize).Acrosstheboard,itmay be necessary tomakemodest budget increases for evaluating the effectiveness of, previouslyunplanned,capacitybuildingactivities.
21. Whereverpossible,I2Ishouldseektomaximiseimpactbybuildingpartnershipswith–andleveragingtheresourcesandnetworksof–existing,well-establishedinstitutionsandprogrammesoperatinginthesamesectors.
2TheprogrammeheadcontractextendssixmonthsbeyondthecurrentenddatefortheI2Iprogramme,totakeintoaccountdeverytimelinesfortheFrontierTechaddon,whichfallsoutsideofthescopeofthisMTR.
Page10of68
Opportunitiesforscalingandreplication
22. TheexperienceofdesigningandestablishinginnovationprizesfordevelopmentthroughI2Ihasbeenasteeplearningcurve–butoffersvaluablelessonsandlikelyeconomiesofscalewhenscaling/replicating.
23. At present, the strongest candidates for scaling or replication are theUrban SanitationChallenge –owingtotheleveloftractiongainedwithgovernmentandearlyindicationsthateffectivepublic-privatepartnerships are being facilitated – and the Climate Information Prize – which is working with amotivatedandrelativelywellcapacitatedsolvergroup,andhencefaceslowerbarrierstosustainability.Thoseprizeswherebarrierstoscalingsustainablyarehighest–includingwheresolvercapacityismostlimited–maynotbethemostobviouscandidatesforreplication.
24. Inpractice,specificopportunitiesforreplicationhavesofaremergedinadhocwayatlevelofindividualprizes.I2Ishouldbepragmaticandpursueconcreteopportunitieswheretheyarise,providingthattheprizeinquestionremainsonatrajectorytowardsdeliveringsustainableimpact.
25. This does not, however,mean that other prizes should be de-prioritised.With I2I fundamentally aresearch programme, opportunities to capture learning about, for example, working with differentsolversandindifferentsectorsshouldbemaximised.
Page11of68
Chapter2: Background
TheIdeastoImpactprogramme
2.1 IdeastoImpact(I2I)isafiveyear,£10.9mprogrammewhichsupportsresearchanddevelopment(R&D)in energy, water and climate adaptation solutions for developing countries. The programme waslaunchedinApril2014totesttheapplicationofinnovationprizesasafundingmechanismtocatalyseinnovationonintractableenvironmentalanddevelopmentalproblems.I2Ihasdesignedandlauncheda variety of ‘innovation prizes’ to stimulate and incentivise the development and deployment ofsolutionsfor lowincomeconsumers,focusingonimprovingpoorpeople’saccesstoaffordablecleanenergy,safedrinkingwaterandimprovedsanitation,andresiliencetoclimatechange.
Table1:OverviewoftheIdeastoImpactprogramme
Dates 31March2014to30March2019
Budget £10.9m
Description
I2I is testing theapplicationof innovationprizes as a fundingmechanism tocatalyse innovation on intractable environmental and developmentalproblems. I2Ihasdesignedand launchedavarietyofprizestostimulateandincentivise the development and deployment of solutions for low incomeconsumers, focusing on improving poor people’s access to affordable cleanenergy,safedrinkingwaterandimprovedsanitation,andresiliencetoclimatechange.
OutcomesbyMarch2019
§ Newideas,concepts,technologies,systemsorprocessesdevelopedthatarejudgedtohaveawardvalue(73cumulative)
§ Prize driven activities deliveringmultiple innovations that benefit lowincomehouseholds(52cumulative)
§ Prizes and challenges stimulate new investment in technologies andbusinessmodels(£35m)
§ Examples of discussions among key stakeholders around innovationprizesindevelopmentorforsocialoutcomesrefertotheprogramme(24cumulative)
Impactby2025
12 million people will have strengthened adaptive capacity and/or new orimproved access to services (energy and water and sanitation) throughaffordabletechnologies,improvedserviceprovisionandnewbusinessmodelsforlowincomeconsumers.
2.2 IMCWorldwideleadstheprogramme,managingaconsortiumwithwide-rangingspecialistexpertise.I2I is structuredaround five currentlyoperationalprizeprogrammes,mostofwhichare individuallymanagedbyanimplementingagent,withcross-cuttingsupportprovidedbyprizedesignersandItadonevaluationandlearning.Detailsofeachofthefiveprizeswhicharecurrentlyoperationalaresetouton
Page12of68
thefollowingpage(table2)andanoverviewoftheprogressofeachagainstkeymilestonesprovidedinAnnexA.
2.3 The programme defines an innovation prize “as a financial incentive that induces change throughcompetition”.Asummaryofthepurposeandcharacteristicsofsuchprizesisprovidedinbox1.
Page13of68
Table2:TheI2Iprizes
Implementer Sector Country Solvers Description
ClimateInformationPrize
CardnoKenya CC
adaptation–
climate
information
Kenya Smalland
medium
enterprises
TheClimateInformationPrize(CIP)aimstostrengthentheadaptivecapacityofpoorand
vulnerablegroupsby:drivingthedevelopmentofnewwaystoengagecommunitiesinaccessing
andusingclimateinformation;andraisingawarenessofclimateinformationuptakeanduse
CIPisformedof2stages:
• Stage1(“WazoPrize”)offeredawardsforinnovativesolutionsonmakingclimate
informationmoreusabletocommunities.
• Stage2(“TekelezaPrize”)willrewardapplicantsthatcandemonstratethegreatest
impactinimplementingtheirideawithintheprizeprogrammeperiod.
• Inaddition,recognitionprizes(“Tambuaprize”)arebeingrunannuallyduringthe
period2016-2018withafocusonrecognisingandshowcasingnovelideasandactorsin
theareaofclimateinformation.
AdaptationatScale(A@S)
FormerlyIMC;
implementing
agent
currently
being
recruited
CC
adaptation
Nepal NGOs/CBOs
/businesses
A@Saimstoidentify,rewardandincentivisethescalingofexistingsuccessfulclimatechange
adaptationinitiativesthatworkwithlocalcommunities.
A@Sisformedof2stages:
• Stage1(“ProtsahanPrize”)recognised15successfulinitiativesthathavesupported
adaptationtoclimatechangeamongpoorandvulnerablegroupandrewarded
innovativeplanstoscaletheseup
• Stage2(“KaryanwayeinPrize”)willbeawardedtoapplicantswhosuccessfully
implementtheirplantoscale-up,demonstratingpositiveoutcomesforpoorand
vulnerablecommunities
• Inaddition,arecognitionprize(“PahichanPuraskar”)willraiseawarenessofadaptation
andtheprize,andmaintainmomentum.
UrbanSanitationChallengeforGhana(USC)
IRCGhana WASH–
urban
sanitation
Ghana Metropolita
nand
Municipal
District
Assemblies
(MMDAs)
USCisincentivisingMMDAswithapopulationofmorethan15,000toprioritisethedeliveryof
improvedurbansanitation,withtheaimofincreasingaccesstosustainablesanitationservices
forallhouseholds,withaparticularfocusonthepoor.
Thechallengeisformedof2stages:
§ Stage1(“Duapa”)offereda3monetaryprizesforthebestliquidwastemanagement
strategies,aswellas21honoraryawards.
Page14of68
§ Stage2(“theDignifiedCityAward”),inwhich17MMDAsareparticipating,willtrack
theirperformanceinimplementingtheseplansandprovidemonetaryrewardtothose
whohavehadthegreatestimpact.
§ Abaselineassessmentisalsobeingconductedtoenablerobustmeasurementofthe
impactofimplementation.
USChasrecentlybeenawardedfundingbytheBillandMelindaGatesFoundationtoimplement
asisterprize–asanitationawardschemetofundspecificthematicawardsundertheumbrellaof
theUSC–inpartnershipwiththeCommunityWaterandSanitationAgency.
DreampipeII
IMC WASH–
non-revenue
water
Global Water
utilities,
financiers
and
transactions
advisers
DreampipeisaglobalWASHprize(focusedonDFIDprioritycountries)thatseekstoencourage
thedevelopmentofworkableandreplicableideasforhowtoexpandthefinancingavailablefor
non-revenue-water(NRW)reductionactivitiesindevelopingcountries
DreampipeIIisformedof3phases:
§ Phase1–awardswillbemadeforthebestbusinessplanssettingouthowStage2
requirementswillbecarriedout.
§ Phase2–applicantswillbeassessedoncarryingout(andfullydocument)a
demonstrationprojecttoreduceNRWinaselectedwaterutility,aswellasanupdated
businessplanfocusingonfinancingandcontractingforamajorexpansionprojectin
thesameutility.
§ Phase3–applicantswillsubmittermsheetsforallmajorprojectandfinancing
agreementsneededfortheexpansionprojectintheselectedutility,includinga
performance-basedcontract.
Nb.DreampipeIwaslaunchedinFebruary2016andawardedinJuly.Dreampipehassincebeen
redesignedtocreateamorefocused,tournamentapproachtominimiserisk.
GlobalLEAPawards
Energy4
Impact
Energy
Access3–off
grid
refrigeration
Global Manufacture
rsand
distributors
ofoff-grid
appliances
I2IissupportingaDFID/USAIDcollaborationontheGlobalLEAP(LightingandEnergyAccess
Partnership)awards–anexistingprogrammeoperatedbyNGOCLASP–byprovidingfundingto
theprizepurseforitsoff-gridrefrigerationprizeandrunningfieldtestingofshortlisted
appliances.
I2Iisalsodevelopingasisterprizeforlargerscalerefrigeration–theOff-GridColdChain
Challenge–forwhichtheconceptnoteshasrecentlybeenapprovedbyDFID.Theprizewillbe
formedoftwostages,thefirstfocusingonideationandtheseconddemonstrationofresults.
3Nb.theoriginaldesignofI2I’sworkintheenergyaccesssectorwasformedofatrioofprizestosupportashiftinpolicytosupportLPGusage.Ofthese,oneprize(‘theCylinderPrize’)toidentifyappropriate
methodstorecycleandreuseoldLPGcylindersthatdidnotmeetnewsafetystandardswasimplemented,butdiscontinuedowingtochangesinthepolicyenvironment.
Page15of68
Box1:Innovationprizes–asnapshot
Inducementprizeshavebeenusedsinceantiquitytostimulatesolutionstoproblems–largeand small. Initially a tool of governments,4 those seeking to kick start new businesses,industryorevenfundsciencealsoseizeduponprizes5,6.Prizeshaveevolved–morerecentlyduetothesuccessoftheAnsariXPRIZEin2004forsuborbitalspaceflight7–intoatoolusedbynon-profits,governments8andcorporations9.Innovationprizesseektousethepromiseofprestige(agoldmedal,sealofapproval,PRetc.),amonetaryreward(prizepurse)andthevisionofafuturemarket/customertostimulatetheinterestofproblemsolversandattempttobring innewsolvers toaddress thechallenge.Theyaimtostimulateinnovationratherthanrewardgoodperformanceex-post(asamorestandardrecognitionprizewoulddo).10Theprincipaldifferencebetweeninducementprizesand competitive grants is that prizesmay havemultiple solvers, none ofwhom are pre-selectedbythefunderandanyofwhommaywintheprizeuponcompletionofthetask.11Innovationinthiscontextcanbeunderstoodinabroadsense,as“renewing,advancingorchangingthewaythingsaredone”.12Innovationdoesnothavetobetechnicalbutcaninvolveachangeinbehaviourorpractice,orthedesignofnewbusinessmodelsthatcansuccessfullyscaleuptechnologies.13Prizescan:14
§ Produceamultiplierintheorganiser’sreturnoninvestment.
§ Shortenthetimetomarket/implementationforsolutions.
§ Resultinentirenewindustriesorbusinesssectors.
Prizesworkbestwhen:15
41765AwardoftheBritishLongitudePrizefordetermininglongitudeatsea.1773AcadémiedeBesançon’sPrizeforsubstitutefoodswonbythepotato.1795Napoleonawardsprizetoavoidspoilageoffoodledtothedevelopmentofcanningtechnology.5Hanson(1998)Patternsofpatronage:whygrantswonoverprizesinscience.California:UC,Berkeley.(http://mason.gmu.edu/~rhanson/whygrant.pdf)61927OrteigPrizefortransatlanticflightprovidedbyahotelierinterestedinpromotingtourismandwonbyCharlesLindbergh.7http://ansari.xprize.org8EUHorizonPrizes(https://ec.europa.eu/research/horizonprize/index.cfm?pg=about)USAGovernmentPrizes(https://www.challenge.gov/list/)9OnlineplatformencompassesprizesfromNGO’s,individuals,corporations,non-profits(https://www.innocentive.com/ar/challenge/browse)10I2I(2016)AnnualReport2016.11I2I(2016)AnnualReport2016.12Everett,B.(2011)Evidencereview–environmentalinnovationprizesfordevelopment.London:DEWPoint.(https://www.gov.uk/dfid-research-outputs/evidence-review-environmental-innovation-prizes-for-development-dew-point-enquiry-no-a0405)13I2I(2016)AnnualReport2016.14Hendrix,M.(2014)ThePowerofPrizes.WashingtonDC:USChamberofCommerceFoundation.(https://www.uschamberfoundation.org/power-prizes-incentivizing-radical-innovation-0)15NestaCentreforChallengePrizes(2014)Challengeprizes:apracticeguide.London:Nesta.(http://www.nesta.org.uk/sites/default/files/challenge-prizes-design-practice-guide.pdf)XPRIZE(n.d.)WhatisanXPRIZE?CulverCity:XPRIZE.(http://www.xprize.org/about/what-is-an-xprize)
Page16of68
a) Thecurrentstateoftechnologysuggeststhatproblemisactuallysolvableina<10yeartimeframeandyouunderstandwhatthecurrentbarriersaretosolvingit.
b) Theproblemtobesolvedcanhave itsoutcomespecified inaclearmanner.This isessentialtobeabletojudgeandcommunicatecleararoundthewinner(s)
c) The risk to be takenonby the solver is reasonablewhen compared to the reward(monetaryprize,futuremarket/customers).
d) Thematurityoftheecosystemofknowledge,productsandservicesandregulationsthatpertaintosolvingtheprizeiswellunderstood.
e) Solvingtheproblemwillhaveanimpactthatprovidesthelevelofrecognition/prestigesufficienttointerestandreachabroadcommunityofproblemsolvers.16
Designing a prize that has the best chance of success requires investigating and devisingstrategiestoanswereachofthesepoints.Thesepointscanbeaddressedbyusinganumberoftechniques,17including:
§ Milestone or staged sub-prizes that relate to developing the capacity and/orcompetencyneededtomakeasignificantimpactontheprimaryproblem.
§ Prizeecosysteminvestments–regulatorychange,provisionofknowledge,securingofproductsandservices(includingtestfacilities)forproblemsolvers.
§ Market/customerstimulationandeconomicdevelopmentincentives.
§ Investmentinoutreach(publicandpolicymaker),educationandawarenessaroundtheproblemandbenefitsofsolvingit.
Not everything is ‘prizeable’. Prize design and execution involves a lot of projection andassumption.Rigorousresearchattheoutset isrequiredtoensurerobustdefinitionoftheproblemandthebarrierstosuccess.Thisleadstoamoreeffectiveidentificationofwhereaprizecanbeappliedtogeneratethegreatestleverage.Itcanoftenbenecessarytotweakoradapt theprize,however substantive changeswouldusually indicate flaws in theoriginalresearch.Awelldesignedandexecutedinducementprize,however,canbringabouttransformativechange.
16Bruntetal.(2012).Inducementprizesandinnovation,JournalofIndustrialEconomics,XL(4),pp.657-696.(http://people.hbs.edu/tnicholas/IPI.pdf)17Innocentivecasestudies(https://www.innocentive.com/resources-overview/case-studies/),NestaCentreforChallengePrizes(2013)Thegivingchallengeprizes.London:Nesta.(http://www.nesta.org.uk/sites/default/files/the_giving_challenge_prizes.pdf)
Page17of68
PurposeoftheMTR
2.4 ThisMTRtakestheformofanoperationalreviewtoguidetheI2Iprogramme’sfuturestrategicdirectionandresourceallocation,aswellastoimprovevalueformoneywherepossible.
2.5 Specifically,itseekstoaddresssixmainquestions:
§ Isthelevelofmanagementappropriateacrosstheprogramme?
§ Couldgovernanceandriskmanagementsystemsbeimproved?
§ Istheinvestmentinevaluationandresearchappropriate?
§ Howcanvalueformoneyacrosstheprogrammebeimproved?
§ HowcanDFIDandtheprogrammeenhanceimpactandensuresustainability?
§ Whereshouldfutureinvestmentsintheprogrammebetargetedand/orrestricted?
Approachandmethodology
2.6 TheMTRhasfollowedasetofguidingprinciples–accordinglyithasbeen:
§ Strategic,reviewinghoweffectivelyI2Iistestingitscorehypothesis–thatinnovationprizesareausefulmethodtocatalyseinnovationonandsolvesignificantenvironmentalproblems–toinformfuturedirectionandresourceallocation.
§ Conducted at an operational level, to inform practical recommendations onmaximisingimpactandvalueformoney.
§ Carriedoutinbothasystematicandparticipatorymanner,toensurerobustnessoffindingsandbuildownershipoffindingsamongstimplementingpartners.
2.7 The MTR has used qualitative methods to review I2I’s activities across each of its prizeprogrammes in a structured and systematic manner. An extensive review of programmedocumentsandI2Iproducts,aswellasprizeliterature,wasundertaken(seeAnnexA)andasetofcoreareasforexplorationdeveloped.Theseincluded:
§ Prizedesign–e.g.howproblemswereidentifiedandselected,andprizeparametersdecidedupon;anydeficitsidentifiedintheprizeecosystem
§ Outreachandparticipation–e.g.typesofsolvers;sufficiencyof levelofandchannelsforpromotion;sufficiencyandappropriatenessofincentivesforapplicants;qualityofproposals;riskstosolversandmitigation
§ Judging and award – e.g. appropriateness of incentives created by judging criteria;effectiveness,efficiencyandtransparencyofjudgingprocesses
§ Partnershipsandcoordination–e.g.anyevidenceofinnovativepartnerships;effectivenessofcoordinationwithotherprogrammesororganisationsoperatinginthesamesectors
§ Programme and prize management – e.g. effectiveness and appropriateness ofmanagementarrangements;majorriskstothesuccessoftheprizeandwhethertheseare
Page18of68
managed effectively; appropriateness of and lessons learned on budgeting; quality andeffectivenessofrelationshipsamongstprogrammeimplementers
§ Monitoring, evaluation and learning – e.g. data collection for prize level M&E; relativeinvestment in and relationships between prize design/implementation, prize levelmonitoringandevaluation,andwiderlearning
§ Valueformoney–e.g.driversofVFM;opportunitiestoimproveVFM;effectivenessofVFMmeasures
§ Impactandsustainability–e.g.evidenceofandopportunitiestoensuresustainableimpact;evidenceofsocialinclusionatsolverandbeneficiarylevel;solvercapacitytoleveradditionalinvestment;scopetoscaleand/orreplicatetheprize
2.8 AlistofintervieweecategorieswasdevelopedfollowingdiscussionswithDFID,IMCWorldwide,ItadandIDS,andaseriesofprotocolsdesignedtoguidesemi-structuredinterviewsacrosstheI2Iprizeportfolio,includingfieldworkinKenya,GhanaandNepal(seeannexesBandC).Theseprotocols provided a common framework for data collection to facilitate cross-prizecomparisonsbutwereadaptedaccordingtotheparticularobjectivesandcharacteristicsofeachprize.
2.9 Reportfindingsaresetoutinfourmainparts:
§ Section2takesacomparativeviewacrosstheI2Iportfoliotoidentifyfivekeycross-prizeorprogrammelevelfindings
§ Sections3to7exploreeachprizeinturnwithreferencetotheareasidentifiedabove.Eachconcludesbysettingoutprize level recommendationsontimeframe,scopeforscalingorreplication,andprioritisationofprogrammeinvestment(includingtherelativeimportanceofandapproximatelevelofresourcerequiredforrecommendedinvestments18).
§ Section 8 sets out programme management findings, with a focus on governance, riskmanagementandvalueformoney
§ Section9concludesbyputtingforwardaseriesofstrategicrecommendations.Thesedonotrevise prize specific recommendations made in earlier sections but address issues oftimeframe, programme investment and opportunities for scaling / replication at theprogrammelevel.
18Recommendedlevelofinvestmentisapproximate,andindicatedwithreferencetothefollowingcategories:L=<£5,000;M=>£5,000-15,000;H=>£15-25,000;VeryHigh=£>25,000.
Page19of68
Chapter3: Programme level or cross-prize findings – five key
themes
3.1 At the programme or cross-prize level, the MTR’s findings centrE on five key themes ormessages,asfollow.
Prizesasacatalystforinnovationandproblemsolving:I2Iiseffectivelytesting
itscorehypothesis
Problemselectionandprizedesign
3.2 As noted above, I2I’s core objective is to test the application of innovation prizes as analternate/additional donor funding mechanism for catalysing innovation on intractableenvironmentalanddevelopmentalproblems.ItiscommendablethatI2Ihassetahighlevelofambitionfortestingthishypothesisbydeliberatelyselectingintractableproblemsratherthanthose which are most easily ‘prizeable’. An early shift in focus from ‘hard’ technology topathwaystouse(i.e.market/systemschange)hasbeenvalidatedbythesubsequentworkofotherprominentprizefunders.
3.3 Nevertheless,theinclusionofEnergyAccess–withtheGlobalLEAPawards’focusonappliancesforoff-gridrefrigeration–offersbalancetotheI2Iprizeportfolio.(Althoughbeyondthescopeofthisreviewit isworthnotingthat learningarounduseof innovationprizesfortechnologyalsobeingpursuedviathenewFrontierTechaddon.19)
3.4 Valuable lessons have been learnt around rendering ‘softer’ problems ‘prizeable’ – that is,articulatinganddefiningtheparametersofproblemsinsuchawayastofacilitatedevelopmentofinnovativesolutionswhichcanberewardedviaaprizemodality.Forexample:
§ Thesixthdesignoption(prize)proposedintheI2Idesignreportwasintendedtodealwithdistribution but not implemented as it proved too challenging to define and specify theproblemsufficientlytoformthebasisforaninducementprize.
§ Dreampipehasbeenredesignedtonarrowitsfocusastheoriginalcallproducedtoowideanarrayofproposals/productstodevelopacoherentportfolio.
Learningaroundtheprizemodality
3.5 It is clear that the prizes are effectively incentivising the development and deployment ofinnovativesolutions,andareonatrajectorytodeliveringarangeofprizeeffects(seeprizelevelfindings).Theexactnatureofthis incentivevariesbysolver,but it isclearthattheincentiverelatesasmuchtotherecognitionorprestigeassociatewiththeprizeastotheprizewinningsthemselves.
19http://www.ideastoimpact.net/frontier-technology-livestreaming
Page20of68
3.6 However, although these early indications are promising, the programme has so far hadrelatively little opportunity to collect robust evidence around the value added by the prizemodality. I2I could test this by selecting a specific prize for comparison with programmesworkinginthesamecontextwhichhavesimilaraimsbutusedifferentmodalitiessuchasgrantsorchallengefunding.
3.7 Althoughnotacomparativeexercise,MTRfindingssuggestthatprizesmayofferaddedvaluerelativetootherfundingmodalities–e,g,grants,loans,equityinvestments–vis-visreachingandincentivisingnewtypesofsolverthroughcompetitionandrecognition(seeparagraphs3.10–3.15,4.13),andactingascatalystswhichbuildonandacceleratethemomentumofexistingenthusiasm and/or initiatives (see paragraphs 4.33 and 5.8). However, compared to moreconventionaltypesoffunding,workingthroughprizesentailsadditionalcostsassociatedwiththe innovative nature of modality itself – not least the level of effort involved in buildingunderstandingamongstpotentialparticipantsaboutthepurposeandstructureofprizes(seeparagraphs4.5,5.13and6.5).
3.8 I2Iexperiencesuggeststhatsome(‘softer’)problemsmaybe lessconducivetobeingsolvedthroughpureprizesthanothers.Likewise,itcanbechallengingtodrawincertaintypesofnewsolverswithmore limitedcapacity (see ‘building solver capacity’below). In futureDFIDandothersmightconsiderhybridfundingmodalitiesinthesecases.KnowledgegeneratedbyI2Iastowhenandwherethecatalyticeffectofprizemodalityismostandleasteffectivecouldbeusedtostructure/sequencedifferentmodalities.
Outreach and participation: I2I has drawn in solvers ‘beyond the usual
suspects’butshouldcontinuetomitigatesolverrisk
Drawinginnewentrants
3.9 Inducement prizes offer ameans of reaching awider group of participants than traditionalfundingmechanisms.Theyseektodrawawiderarrayofparticipants intoproblem-solving–thatis,to“getnewpeopletotackleoldproblemsinnewways”.20
3.10 I2I istestingtheprogrammehypothesiseffectivelybytargetingarangeofdifferenttypesofsolversacrossgovernment,smallandmediumenterprises,NGOsandCBOs.Prizelevelfindingsreflectonwhoisbeingrewarded,thesufficiencyofoutreachactivities,andappropriatenessoftheprizeincentiveforthetypeofsolverinquestion.
3.11 ItisworthnotingthatI2IhasnotworkedwithprivatesectoractorsasextensivelyasDFIDmighthaveoriginally intended.During thedesign phase the team concluded that a pure financialincentivetoinducechangebyprivatesectorfirms,whoarealreadyprofit-driven,waslikelytobe insufficient. Nevertheless, there are early indications that both the Urban SanitationChallenge and Dreampipe II have effectively incentivized innovative partnerships involvingprivatesectoractors.
20PMU
Page21of68
Mitigatingsolverrisk
3.12 TheI2Iteamhavebeenmindfuloftheethicalchallengesassociatedwithoutsourcingrisktosolvers–asdemonstratedbyuseofstagedprizestructures(withStage1prizesfocusingonideation).Whetherprizedesignhasaccuratelyanticipatedtheresourcesneededofparticipantsandseta reasonable risk/rewardcurvewillbedemonstrated inStage2,where thecostsofentryaregenerallymuchhigher.
3.13 The main risks reported by solvers relate to financial and opportunity (time) costs, withrespondentsmostoftenpointingtothepotentialburdenofStage2monitoringrequirements.TherewillbeanongoingneedtomitigatetheserisksduringStage2wherepossible,bothbyminimisingcostsforapplicantsandincreasingpotentialreward,byfacilitatingengagementwithothersourcesoffinance.
3.14 Developinganoptimalrisk/rewardcurveforsolvers isan integralpartofstresstestingprizemodelsasameansofinducingandacceleratingchangearoundaparticularproblem.Workingwith a range of solver types, I2I is well-placed to capture lessons on this, as well as theappropriatenessoftheprizemodalityfroma‘donoharm’perspective.Light-touchevaluationsofthecostsandbenefitsofparticipationforbothwinnersandnon-winnersforparticularprizescouldcontributetothis,aswellas informanassessmentoftheeffectivenessofprogrammeeffortstomitigatesolverrisk.ThesecouldbeconductedwithafocusonStage1non-winnersduringStage2toavoidknockoneffectsontheoverallprogrammetimeline.
Prizesecosystems:I2Ishouldcapitaliseonecosystemstrengthsandaddress
deficits(particularlysolvercapacitygaps)
3.15 Asa ‘purist’prizeprogramme, I2Ihasattempted tonestleprizeswithincontexts (countries,sectors)withhighlevelsofexistinggovernment/donoractivityandsupportinfrastructure,withpositivebutvaryinglevelsofsuccess.TheClimateInformationPrizeinKenya,hasforexample,benefitted from an enabling environment conducive to entrepreneurship, whilst, in Nepal,AdaptationatScalehascapitalisedongovernmentcommitmentstoprioritiseadaptation.
Needtosupportsolvercapacitybuilding
3.16 Although varying by prize, the main deficits in prize ecosystems21 identified across theprogrammecentreonsolvercapacitygaps–primarily:
§ solvercapacitytoprepareapplicationsandimplementandmonitorsolutions
§ solvernetworksnecessarytoengagepotentialinvestorsandleverageadditionalfinance
§ infrastructureforcollaboration/networkingwithothersolverstoenablecollectiveproblemsolving
3.17 IthasbeenandwillcontinuetobenecessaryforI2Itoengageinadditionalsupportiveactivities–notleastsolvercapacitybuilding–tomitigatedeliveryrisks,ensureimpactandsustainability,
21Prizeecosystemisdefinedasresourcesavailabletosolvers(andhencepotentialbarrierstoaccess).
Page22of68
andmaximisevalueformoney.Thisisakeylessononthespecificchallengesofoperatingprizesindevelopingcontexts.Futureprizesmightbeembeddedwithinlargerprogrammestomitigatetheserisks.
Capitalisingonthestrengthsofprizeecosystems
3.18 Prizes shouldalsoensure that theyarebuildingon the strengthsof theprizeecosystems inwhichtheyareoperating.Forsomecertainprizes,suchasCIP,thiswouldmeanidentifyingandleveraging the resources of existing programmes and institutions working in the sector, toensuretheadditionalityoftheprizeandenhancevalueformoney.Forothers,suchasA@S,this could involve engaging with ongoing national policy development to ensure thesustainabilityofinnovationsbeingscaledundertheprize.
Innovativepartnerships:I2Idemonstratesthatprizescanincentivisecoalition
buildingaround‘thorny’developmentproblems
Testingdifferenttypesofpartnerships
3.19 Partofwhatismostinnovativeabouttheprogrammeisitsdemonstrationthatdifferenttypesofpartnershipsaroundprizesfordevelopmentcanwork(i.e. incentivisingdifferentactorsto“coalesce around a common good”.22 Examples of those being effectively tested by theprogrammeincludethefollowing:
§ The Urban Sanitation Challenge, which has gained traction with local government andprovided a platform for public-private partnerships around urban sanitation challenges.These are starting to be put into practice asMMDAs begin to implement their plans incooperationwithprivatesectorpartners.
§ The DFID/USAID partnership on Global LEAP refrigeration prize, which indicates thepotentialvalueofdonorcollaborationoninnovationprizes.I2Iis,forexample,strengtheningtheincentiveofferedbytheexistingprizebyrunningfieldtestingofshortlistedappliances,andofferingmanufacturersanddistributorswithvaluableconsumerfeedback.
§ Dreampipe II’s facilitationofnon-IFI investment (banks,pension funds,privateequity) insolutionsforreducingnon-revenuewatermanagement.Valuablelessonsarebeinglearntaboutdrawinginnewsolversthroughpartnerships–ithasprovenmosteffectivetodrawinex-cityfinancierthroughpartnershipswithsectorexperts(waterutilities),withthe‘usualsuspects’leadingbutdrawinginnewsolverstosupportthem.
Mitigatingdeliveryriskthroughpartnership
3.20 Whereprizeshavenotyetfacilitatedsuchpartnerships,thisrepresentsarisktodelivery–forexample,tothesustainabilitytosolutionswhereCIPsolversarenotwell-networkedtoleverageprivatesectorinvestmentorA@Ssolutionsdonotsecuregovernmentbacking.
22PMU
Page23of68
Balancing researchandapplication: I2I should clarify relationshipsbetween
prizedesign/implementation,evaluationandwiderknowledgesharing
Resultsmanagement
3.21 TheProgrammeManagementUnitmayhavebeenincentivisedtoplacetoogreatanemphasisonmaximisingbeneficiarynumbers.ThishasbeenprioritisedtoalignwithDFID’spreferencefor maximising implementation and impact for beneficiaries (in the context of condensedtimeframesforprizeevaluations).Butitisworthreconsideringthebalancebetweendirectandindirect outcomes – that is, between beneficiaries impacted directly by the solutions beingdeveloped and implemented under the prizes, and inducing the development of reallyinnovative solutions with potential for largescale, albeit indirect, impact (see Dreampipe IIfindings,forexample).
3.22 I2Icoulduseproxiestomeasureindirectbenefitsifaparticularprizeproducesasolutionwitharelativelylownumberofdirectbeneficiariesbutdemonstrablepotentialforsuchlargescaleimpact.However,wedonotrecommendthattheteamrevisitbeneficiarytargetsatthispoint.Ratherprizeteamsshouldbearthistensioninmind–forexample,byensuringthateligibilitycriteria for applicants relating to beneficiary numbers do not rule out support to genuinelyinnovativeandpotentiallyimpactfulsolutions.
3.23 TheMTRteamalsosupportsI2I’sdecisiontodemonstrateresultsandcapturelearningbeyondtheprogrammeResultsFrameworkwithreferenceto(alimitednumberof)anticipatedprizeeffectsforeachprize.Thiswillenabletheprogrammetocaptureinterestinglessonsaboutthepotentialofprizesfordevelopment,aswellastodemonstratevalueformoney.
3.24 Monitoringinformationsystemscouldalsobeimprovedbystreamliningquarterlyreportingtoalign with the programme’s adaptive mode of operating. This could include use of moreinnovativemediumssuchasinfographicsorvideocasestudies.
Flexibleandresponsivelearningandevaluation
3.25 Staggeredinterventionshaveenabledvaluableadhocsharingoflessonsamongstprizeteamsonpracticalissues,withthoseearlieronintheprizepipeline(A@S,DreampipeII)abletoadoptthosetoolswhichhaveprovenmostsuccessful,suchaseffectivecommunicationsmethodsandcontractingmechanisms.
3.26 Prize level evaluationswere originally designed in a traditional, linear fashion,whereas theprogrammehascometoworkinaflexibleanditerativemanner.Itisworthreconsideringthetrade-off between academic rigour and the pace necessary to inform changes to prizeimplementation.Ideally,evaluationscompletedafterawardingofStage1scouldinformdesignchangesaheadofthelaunchofStage2s.ThisapproachhasnowbeenemployedinthecaseofAdaptationatScaleandcouldbeusedforthoseprizes (Dreampipe IIandtheEnergyAccesssister prize) which have not yet awarded their Stage 1 prizes. However, the MTR teamrecognisesthatevaluationsneedtostrikeabalancebetweenspeedandresponsiveness,onthe
Page24of68
onehand,andtheneedtocollectrobustevidencetofeedintotheI2Iresearchstreamslateronintheprogramme.
3.27 The nature of the relationship between prizemanagers and evaluators has developed overtime, offering useful lessons around the need for evaluators to bemore closely embeddedwithinratherthanentirelyindependentfromprizeteamswhereprogrammeobjectivesrequireahighlevelofadaptation.Likewise,theprogrammebenefitsfromhavinganexternal‘criticalfriend’intheresearchrole,butreviewsneedtobeproportionateandtimelytoenabletheteamtoworkquicklyandadaptively,andfacilitateimprovementstoprizes.
Knowledgesharingandinfluencing
3.28 ThereisaneedtodevotemoreattentionandresourcetoI2I’swiderknowledgesharingandinfluencingagenda.Recentadditionstothelogframeincludeanoutcomeindictoronthisbroadinfluencingagenda,aswellasanindicatorontheinfluencingactivitiesof individualprizesatoutputlevel,butprogrammeactivitiesdonotyetfullyreflectthis.
3.29 The PMU has already made efforts to bring together prize managers more regularly andfacilitatecommunicationsusinganonline tool i.e. isdeveloping the infrastructure toenableincreased cross-prize reflections. However, I2I should prioritise the production anddisseminationofglobalpublicgoods,andengagementwithother(potential)prizefundersandimplementers to improve the way in which other prize programmes are designed andimplemented. Learning products should capture prize level learning aswell as operationallessonsaroundrunningprizesinthedevelopmentsector
3.30 Thetimingforincreasingthelevelofeffortdedicatedtoknowledgesharingisoptimal.Nowthatthe prizes are established, there is scope to draw cross-prize reflections, whilst prizemanagement is expected to become less time intensive owing to the longer timeframesbetweenawardstages.
Page25of68
Chapter4: ClimateInformationPrize
Background
4.1 The Climate Information Prize (CIP) aims to strengthen the adaptive capacity of poor andvulnerablegroupsby:drivingthedevelopmentofnewwaystoengagecommunitiesinaccessingandusingclimateinformation;andraisingawarenessofclimateinformationuptakeanduse.
4.2 CIPisformedof2stages:
§ Stage 1 (“Wazo Prize”) offered awards for innovative solutions on making climateinformationmoreusabletocommunities.
§ Stage2(“TekelezaPrize”)willrewardapplicantsthatcandemonstratethegreatestimpactinimplementingtheirideawithintheprizeprogrammeperiod.
4.3 Inaddition,recognitionprizes(“Tambuaprize”)arebeingrunannuallyduringtheperiod2016-2018withafocusonrecognisingandshowcasingnovelideasandactorsintheareaofclimateinformation.
Headlines
4.4 Keyachievementsinclude:
§ Successful awarding of prizes to promising organisations – including “people that DFIDdoesn’tusuallyreach”23–withmanywinnersactivelypursuingtheirproposedinnovations.The prize has helped to “generate creative solutions”24 with innovations including theapplication of climate information approaches not previously used in Kenya and to newarease.g.livestocktrading.
§ Successfulawarenessraisingaroundthevalueofclimate informationand its relationshipwith improved livelihoods and poverty reduction. There is consensus that “demystifyingtheseissues”25representsrealaddedvalue.
4.5 Challengesinclude:
§ EngagementwiththeKenyaMeteorologicalDepartment(KMD)owingtotheirexpectationof payment, contrary to DFID funding rules. Innovators are also using alternative datasourceswhichoffermoreusableformats(e.g.moreregularlyupdateddata)morewidelythanexpected.
23Implementingagent,applicants.24Implementingagent25Implementingagent
Page26of68
§ ResourceintensivenessofdesignandStage1implementation,especiallymanagementtime,given the innovative nature of prize programming. The team have had to work hard tomaintaintheproportionallocatedtomanagementversustheprizepurse.
§ Resource intensiveness of outreach activities, including the need to educate applicantsaround the difference between prizes andmore traditional fundingmodalities. There isscopetoexpandthescopeofmarketingactivitiesandstakeholderengagement if itwerepossibletoaddressbudgetconstraints.
§ Securingthetimecommitmentofjudges.
4.6 Opportunitiestostrengthentheprizeincludescopeto:
§ Enhanceoutreach–givenresourceconstraints,itisnotrecommendedthattheteamexpandmarketingactivities.However,theteamcouldconsidertargetingpartofexistingresourcestoreachcommunitiesofparticularinterest–forexample,vulnerableandharder-to-reachpotentialapplicantsincludingfromthenortherncounties.
§ Improve stakeholder engagement – given the crowded nature of the adaptation andinnovationsectorsinKenya,theprizecoulddomoretocoordinatewithandleveragebetterestablishedprogrammes,institutions,andnetworks.
§ Enhancesustainability–primarilybysupportingwinnerstoengagewithothersourcesoffinance.Weakinnovator/fundernetworksrepresentthemaindeficitintheprizeecosystemandacorerisktotheprize’ssuccess.
Applicationandaward
Accessibilityofapplicationplatformandqualityofapplications
4.7 Theprize’suseofanonlineapplicationplatformhasgenerallyprovenuserfriendly,althoughinterviewees reported someplatform issues. The socialmedia aspect of shortlisting for theinducementprizesmaybechallengingwhereapplicantsarenotalreadywell-networked.
4.8 Judgesreportedsignificantvariationinthequalityofapplications.Wazowasabletoawardto13applicantscomparedtoa targetof15, indicatingthatprocessqualityrequirementsweremet.However, thatonly13of 89eligible applicantswereprizeworthy indicates thatmanyapplicantsfounditchallengingtounderstandandarticulatehowtheirsolutionwouldtackletheproblemtheprizeisseekingtoaddress.
Effectivenessandefficiencyofjudgingprocess
4.9 Securingthetimecommitmentofjudgeshasprovenchallenging–withtheprizeteamrequiredtoexpendsignificanttimechasing,originallyunpaid,judgestocompletetheirassessmentsofWazoapplications.Theissueofpaymenthassincebeenreflectedonandaddressed.
4.10 The in-country prize team are mindful of the need for the judging process to be fair andtransparent,whichwasgenerallysupportedbytheviewsofapplicants.However,theuseofa
Page27of68
publicvotingsystemtoshortlist–althoughausefulmarketingtoolfortheprize–mayskewjudgingoutcomesawayfromthemostinnovativeanddeservingapplications.
4.11 Thejudgingprocesscouldbemademorerobustby:clarifyinghowtheprizedefinesinnovation;categorising applications when judging for ease of comparison; adding a second round ofjudgingi.e.shortlistingbyjudges;andincreasingthetimeavailableforapplicantpresentationstoenableamorethoroughassessmentofproposals.Theteamhasalreadytakenthestepofpayingjudgesfortheirtime.
Appropriatenessofmonitoringandverificationsystems
4.12 Verificationofsubmissionsforbothprizescouldhavebeenmorerobusttoensurethecredibilityof applicants. Identifying Tekeleza indicators that are sufficiently robust for verification andmonitoring purposes but do not present a barrier to entry has been challenging. It will beimportant to keep a close eye on the impact of these requirements – on the number ofsubmissions aswell as the prize’s ability to assess performance – during Stage 2 andmakeadjustmentsifnecessary.
Outreachandparticipation
Additionalityoftheprizemodality
4.13 There are promising signs that the prizes have incentivised the development of innovativesolutions with significant potential to benefit poor people (see ‘impact and sustainability’below).Alltypesofintervieweeattributedstronguptakeandtheinnovativenessofproposedsolutionstothecompetitiveaspectoftheprizewhichdrawsinandmotivatesdifferentkindsofapplicants:“Itgivesthemtheconfidencethatwhattheyaredoingissomethingthathassocialandeconomicvalue”/“Beingchosenmademefeelthatthisisavalidproject…thatcanbenefitotherpeople”.26
Sufficiencyofoutreach/marketingactivities
4.14 Theteamhavemadeconsiderableeffortstomarkettheprizewithlimitedresources.Applicantswhoattendedprizeeventsreportedhavingheardabouttheprizefromnewspaperarticlesandwordofmouth.Hostingstallsatlargereventshasreachedalargeraudience(andprovenmorecosteffective)thanholdingindependentevents.
4.15 However,socialmediausemaynothavebeenaseffectiveasanticipated.Astrongfocusonsocialmediamayalsohavelimitedtheprize’sabilitytoreachmarginalisedcommunities,whichrequiresin-personoutreach.
26Implementingagent,judges,applicants
Page28of68
Typesofsolvers/socialinclusion
4.16 Participantsareamixtureofstart-ups,small/mediumsocialenterprisesandNGOs.Thewiderprivatesectorhasnotparticipated,likelyowingtothevalueoftheprizesonoffer(amaximumof$15,000forWazo,$7,500forTambuaand$200,000forTekeleza).
4.17 Theprizehasbeensuccessfulinattractingapplicants(andwinners)beyondtheusualsuspectsi.e.notonlythosewhomighthavesecuredgrantfundingotherwise.TheStage1evaluationfoundthatonly3of10Wazowinnershadpreviouslyappliedforaninnovationprize.Thereisalso a reasonably good geographical spread among applicants, although could securemoreapplicationsfromthenorth(participantsfrom30of47counties).
4.18 Intervieweesestimate thatapplicantsusually fallwithin theage20-45bracket. ForWazo,7winnersweremenand6women. Interestingly,womenappear tosubmit relativelystrongerapplications,withaconversionrateof32%(winnersasapercentageofcompletedapplications)comparedto7%formen.InStage2theprizecouldconsidertargetingandsupportingwomensolverstocompletetheirapplications.
Supportiveactivitiesandmitigatingrisksforparticipants
4.19 Whethertheprizedesignhasaccuratelyanticipatedtheresourcesneededofparticipantsandsetareasonablerisk/rewardcurvewillbedemonstratedbyTekeleza.Itisworthbearinginmindthat the cost of entry for this third prize ismuch higher thanwas the casewithWazo andTambua (particularly monitoring requirements). The Stage 1 evaluation has found thatapplicationbarrierscentreoninvestmentoftimeandfinances.
4.20 IthasbeennecessarytodevelopdetailedapplicationguidelinesandprovideadditionalsupporttoweakerTekelezaapplications.Despite theprogramme’s ‘purist’prizeapproach, theprizeteamareexploringthepossibilityofholdingaworkshopongrowingasuccessfulstart-upbasedonfeedbackfromwinners.Thereisscopeforvaluableassistanceonimplementationofideas,businessdevelopment,mobilisingfinanceetc.Inparticular,riskofattritioncouldbeaddressedbydoingmoretosupportwinnerstoengagewithothersourcesoffinance.Indeed,themaindeficitidentifiedintheprizeecosystemrelatestoinnovator/fundernetworks.
Prizemanagement
Governanceandrisk
4.21 Managementarrangementsaregenerallyeffectiveandappropriate.TheimplementingagentispositiveaboutitsinteractionswithIMC,withoneteammemberdescribingthisas“averyopenandcollaborativeworkingrelationshipratherthanatraditionalsubcontractorrelationship”.
4.22 The implementing agent appears to have given careful thought to the scope formitigatingdeliveryrisks.Themainriskstothesuccessoftheprizearei)thelevelofuptakeandqualityofTekeleza applications and ii) that winners drop out over time, including owing to financialconstraints. The team is working hard tomitigate these by providing additional support to
Page29of68
applicants who submit sub-standard applications, and requesting quarterly reports fromwinnerstomonitorandsupportimplementation.
Monitoringandevaluation
4.23 AstheprizemovesintoStage2plansformonitoringandverifyingsolvers’activitiesshouldbedecidedupontoensurelegitimacyandthatsolversaredevelopingimpactfulsolutions.
4.24 Teammembers assess lesson learning around theWazo prize to have been effective – butreflectedthatitwouldhavebeenusefultofeedevaluationfindingsfromStage1intoStage2design.
Partnershipsandcoordination
4.25 ItmightbeworthreconsideringthelevelofeffortbeinginvestedinimprovingtherelationshipwithKMD,giventhechallengesencountered(andassociatedlevelofeffort),aswellasthatithasbecomeapparentthatKMDdataisnotnecessarilybestsuitedtoI2Isolversandthesystemstheyaredeveloping.Nevertheless,itisunderstoodthatKMDcouldcontinuetoplayaroleinqualitycontrol.
4.26 However,CIPisoperatinginarelativelycrowdedsectorandcouldmaximisetheprize’simpactby building partnerships with longer-established actors e.g. the Kenya Climate InnovationCentre. A higher level of stakeholder engagement with other programmes and institutionsworkinginthesectorwouldhelptoraiseCIP’sprofileandenabletheprizetoleveragenetworksandresourcestoaddresstherisksidentifiedabove.
4.27 Tapping into the private sector (e.g. banks, Safaricom) to build such partnerships has beenparticularlychallenging.ThereisscopetocapitaliseonCSRinterestsbutbuildingrelationshipsdirectlywithprivatefundersrequiressignificanttimeinvestment.Whenidentifyingsourcesofadditionalfinancewithwhichtoconnectsolvers,theprizeshouldbestrategic–forexample,byworkingwithexistinginvestornetworks.
4.28 Thereisalsoscopetobuildacommunityofpracticeamongstwinnersandothersworkinginthesamespace.
Valueformoney
4.29 Management effort has been higher than anticipated owing to innovative nature of prizeprogramming. The team have worked hard to maintain the proportion allocated tomanagement versus the prize purse, and achieved a great dealwith limited resources (see‘ProgrammeManagement’fordetails).
4.30 Thereispotentialforasingleinnovationtohaveareal(replicable)impact;if/whenthiscomestofruition,theprizemayhaveplayedanimportantcatalyticrolewhichoffersexcellentVFM.The prize is already assisting winners to simplify climate information for better use anddisseminationtoawideraudience.PrizeVFMis,ofcourse,inextricablylinkedtothequestionsofimpactandsustainability(seebelow).
Page30of68
Impactandsustainability
Potentialimpactandsocialinclusion
4.31 Winningsolversproposedgenuinelyinnovativesolutionse.g.useofSMSandvoicetechnologiestoproviderreal-timeweatherandagronomicdatatosmallholderfarmers.However,therewasatendencyamongstintervieweestoassumetechnologytobea‘silverbullet’andhavegreaterreach thanmayactuallybe thecase. It isworthmaintainingabalance in theprizeportfoliobetweentechnologicalandnon-technologicalinnovations.
4.32 Nevertheless,solutionsdoappeartobegearedtowardsbenefittingthepoore.g.improvingthesupplyofinformationtothepoorwhoaredisproportionatelyaffectedbyknowledgepoverty.Some are targeting and empowering marginalised groups e.g. use of women’s groups todisseminateclimateinformation.TheimpactofsolutionsanddegreeofsocialinclusionremainstobeverifiedbyStage2monitoring.
4.33 Theprize’scatalyticroleiscriticaltoitspotentialimpact.Prizelevelevaluationcouldseektoassess the potential effectiveness and efficiency of awarding – enhancing the work andbroadeningthehorizons–ofpeoplealreadyworkingonCIissueswithcommunitiesi.e.locallydriven.
Sustainability
4.34 Some concerns were expressed around the sustainability of solutions, given the resourceconstraintsfacedbysolvers.Thiscouldbeaddressedbyworkingwithprizewinnerstosupportimplementationandminimiseattrition–forexample,buildingbusinessandmonitoringskills.
4.35 Winnersgainexperienceofpitchingaswellasprestige,bothofwhichareusefulwhenseekingfurther investment. The judging criteria also foreground commercial viability. However,innovatorsarenotnecessarilywell-networked,meaningthatsupporttoengagewithpotentialsourcesoffinancewillbecriticaltosustainability.
Strategicrecommendations
Scopetoscaleupand/orreplicate
4.36 Tekeleza will demonstrate whether the prize has pitched the risk/reward curve for solversaccurately,whilstmonitoringandverificationoftheactivitiesofWazoandTambuawinnerswilltesttheimpactofsolutions.Takentogether,theseprovidethedataneededtoinformadecisionrescaleupand/orreplication.Theredoes,however,significantpotentialbasedonthesolutionsproposedandwinners’motivationandcapacity(inspiteofthedeficitsidentified,thecapacityofCIP solvers is relativelywell-developed), providing thatbottlenecksmentionedaboveareaddressed.
4.37 Optionscouldincludelaunchinginspecificcountiesincludingthemostaffected(withtheaddedbenefit of aligning with DFID Kenya’s interest in working in the northern counties) and/orreplicatingsuccessfulsolutionsinotherregions.
Page31of68
Prioritisingprizeinvestment
4.38 Tomitigaterisks,andenhanceimpact,sustainabilityandvalueformoney,itisrecommended
thatthefollowinginvestmentsintheprizebeprioritised(indecreasingorderofpriority):27
§ Stakeholderand,particularly,investorengagement,topromotetheprize,leverageexistingnetworksandinstitutions,andsupportwinnerstosecureothersourcesoffunding(M).
§ Solver capacity building to ensure quality implementation – particularly in the areas ofbusinessdevelopmentandmonitoring(H).
27Recommendedlevelofinvestmentisapproximate,andindicatedwithreferencetothefollowingcategories:L=<£5,000;M=>£5,000-15,000;H=>£15-25,000;VeryHigh=£>25,000.
Page32of68
Chapter5: AdaptationatScale
Background
5.1 Adaptation at Scale (A@S) aims to identify, reward and incentivise the scaling of existingsuccessfulclimatechangeadaptationinitiativesthatworkwithlocalcommunities.
5.2 A@Sisformedof2stages:
§ Stage1(“ProtsahanPrize”)recognisedsuccessfulinitiativesthathavesupportedadaptationtoclimatechangeamongpoorandvulnerablegroupandrewardedinnovativeplanstoscaletheseup
§ Stage2(“KaryanwayeinPrize”)willbeawardedtoapplicantswhosuccessfullyimplementtheir plan to scale-up, demonstrating positive outcomes for poor and vulnerablecommunities
5.3 Inaddition,atleastonerecognitionprize(“PahichanPuraskar”)willraiseawarenessoftheprizeandeffectiveadaptations,andmaintainmomentum.
Headlines
5.4 Keyachievementsinclude:
§ Completionoftheprocessofmappingwhichorganisations/groupsaredoingwhatintheadaptationspaceandrecognisingtheir innovations– in thecontextofa largenumberofexisting,butisolated,adaptationactivities.
§ Confidencethatawardshavebeenmadetoorganisationsworkingonpro-poorandinclusivesolutions – primarily focused on natural resource management and building economicenterprisesontheseecosystems.
5.5 Thereisconsensusacrossintervieweesthatthemainchallengeencounteredbytheprizerelates
tooutreachtovulnerableandmarginalisedgroups.Noneofthe15prizewinnerswereCBOsorwomen’sorganisations.
5.6 Opportunitiestostrengthentheprizeincludescopeto:
§ Level theplaying field for smaller organisations to ensure strongStage2 applicationsbytargetgroupsamongstStage1winners.
§ Enhancesustainabilitybysupportingwinnerstoengagewithothersourcesoffinance.
§ Engagewithongoingnationalpolicymakingprocesses,withtheaimofintegratingthemostsuccessfulsolutionsintogovernmentstrategy.
Page33of68
Design,judgingandaward
Sufficiencyofprizeincentive/additionalityofprizemodality
5.7 SomeintervieweesexpressedconcernthatthevalueofboththeStage1andStage2awardsmaybe lowgiventhe levelofeffortrequiredofapplicants.However,theteam’sdecisiontopitchthelevelofawardsoastomitigateriskforsmallerorganisationswithoutthecapacitytomanagesudden influxesof funding issound.Theprizecould,therefore,consideralternativemeansofstrengtheningtheincentiveforapplicants–forexample,bynetworkingwinnerswithpotentialsourcesofadditionalfinance.
5.8 However, there is agreement amongst interviewees, including winners, on the potentialadditionality offered by the prizemodality. Prizes are seen as a ‘catalyst that can trigger aforward-looking approach’, with competition appearing to increase motivation amongstwinners:
“Aprizeisarecognitionorareward…sopeoplearemotivated.Agrantisonlytherefor
some planned things, but this is a reward for success so in society there is a kind of
prestige.”
Bycontrast,grantfundingisperceivedtobeaccompaniedby:
“Institutional&structurallimitationsthatmeanbeneficiariesdon’thavethefreedomto
decidewheretoinvestscarceresources…[prizesgive]recipients,asrepresentativesofthe
poorandvulnerable,thefreedomtoputfinancialandlivelihoodsdecisionsinthehandsof
thoseaffected…toinvestwheretheythinkwilloffermaximumvalueformoney.”
5.9 Itisalsoexpectedthattheprizewillhelpestablishwinningorganisationsascredibleproblemsolvers–andsotheireligibilityforotherfunding.
Effectivenessandefficiencyofjudgingprocess
5.10 A@S benefitted from a particularly successful judging process. Although there was somedisagreementamongstjudgesastowhichapplicationsweremostprizeworthy,theygenerallyagreedthattheprocessusedrobustscoringcriteriavis-à-visbothtechnicalandsocialinclusionaspects,drewonarangeofcomplementaryexpertiseandwasorganisedinsuchawayastoencourageneutrality.Judgesalsodemonstratedveryhighlevelsofcommitmenttotheirtask.
Outreachandparticipation
Targetgroup/typesofsolvers
5.11 Astrongemphasisonsocialinclusionintheprize’sdesignwasnotnecessarilyborneoutinStage1,withtargetingofCBOsandwomen’sorganisationstocaptureindigenousinnovationsnotaseffectiveashoped.
5.12 Stage 1winners consist primarily of local, national and internationalNGOs of varying sizes.There is a good spread across different, including remote, geographic locations andtopographies,aswellasrepresentationofdifferentcastes,includingDalit.However,mostof
Page34of68
theseorganisationsalreadyreceiveDFIDorotherdonorfunding,raisingquestionsaroundtheextenttowhichtheprizeisdrawinginnewsolvers.
Sufficiencyofoutreach/challengesassociatedwiththeapplicationprocess
5.13 Theprizeteamputconsiderableeffortintooutreachactivities,withface-to-faceworkshopsandcommunicationsviafederatedmemberorganisations(i.e.reachingouttomarginalisedgroupsthroughlocalinstitutionswhichrepresentthem)provingthemostsuccessfulmeansofengagingpotential applicants. By contrast, return on investment in media engagement wascomparativelypoor.Challengesofoutreachincludedthedifficultyofexplainingtheconceptofprizemodalityinacontextwheregrantfundingisthenorm.
5.14 Barriersforat-riskapplicantsinclude:
§ That some smaller organisations assumed themselves not to be eligible as they are notregisteredatdistrictlevel.
§ Lack of capacity (and data) to document and demonstrate the impact of a successfulinnovation,asacorerequirementoftheprize.
§ That although applicants could apply in Nepali, using paper applications, most of thesupportingdocumentsneededtobeprovidedinEnglish.
§ Lack of analytical and writing capacity needed to put together a strong application i.e.describingwhathasbeenachieved,articulatinghowlessonslearnt,andputtingtogetheraconvincingplanforscaleup.
§ Socio-culturalbarriers–ofthe15organisationsinvitedtopresentafterbeingselectedforaward,notonebroughtasinglewomanpresenter.
Stage2targeting
5.15 TheCBOsandwomen’sorganisationswhohavebeeninvitedtoparticipateinStage2mustnowcompetewith‘established,powerfulandwell-resourced’NGOs(i.e.‘notalevelplayingfield’),placingadisproportionateburdenonthemandreducingtheirchancesofsucceedingatStage2.
“Ifwe have a situationwhere the national NGOs andmost of the same, familiar crowd of
organisationsapply in thenext stage, then the realobjectiveof theprizemoneywillnotbe
realized…weneedto…makeveryconcertedeffortsaroundthistarget.”
5.16 Potential solutions for overcoming the barriers to submitting prize worthy applicationsidentifiedaboveincludethefollowing:
§ Increasing face-to-face engagement through, for example, workshops in which set outapplication process and requirements or community mobilisers who could maintainmomentumamongstparticipantsandassistwithsubmissions.
§ Encouraginglarger(I)NGOstosupportpartnerorganisationstosubmitapplicationsdirectlyratherthanapplyingontheirbehalf.
Page35of68
§ ExploratoryworktounderstandwhatisbeingdonebyorganisationsofinterestwhowerenotsuccessfulatStage1,andidentificationofcapacitygaps.
Prizemanagement
Governance
5.17 Team performance has been most impressive as regards the level of effort dedicated tomarketingandstakeholdermanagement.However,theteamhashadinsufficientcapacitytomeetthelevelofdemandforinformationandsupportfromapplicants.Thiswillbeaddressedbyplannedchangestomanagementarrangementsi.e.theongoingtenderprocesstorecruitanimplementingagent.
Risk
5.18 Keyriskstothesuccessoftheprizeareasfollow:
§ That winners from target group will either be unable or insufficiently incentivised toimplementscalingplans,resultinginattritionamongstparticipants,includingtargetgroups.
§ Insufficientwinnercapacitytoidentifyandleverageappropriatesourcesofexternalfunding.
“Peoplehavegiventheirplan,butweareyettoseeit.Muchwilldependonwhethertheyare
successfulinmobilisingfunding…thatisthemainrisk.Forthesecondstageoftheprizewinners
mustimplementeverythingattheirownrisk.Thatisalsoachallenge.”
5.19 Mitigatingactionscouldinclude:
§ CommencingandmaintainingcommunicationswithwinnersaroundtheireligibilityfortheStage2awardi.e.reinforcingthattheawardwillbebasedonsuccessfulimplementationofscalingplans.
§ Supporting solvers to develop a methodology for implementation, and identifying andaddressing capacity gaps -- particularly around monitoring and ‘research into use’ i.e.documentingpilots.
§ Supporting winners to leverage funding e.g. support with identifying suitable sources,facilitatingpartnershipsbetweenwinnersandgovernment/donors.
§ Inclusionofinfrastructure(institutionalcapacity,networks)toaddresstheseasacriterionforimplementingagentselection.
Monitoringandlearning
5.20 Thereisconsensusamongstintervieweesthatvirtualverificationconductedsofarshouldbesupportedbyonthegroundmonitoringandverification.TheMTRteamunderstandsthatlighttouchmonitoringfallswithinthescopeofexistingplansandwouldjudgethis,combinedwithplannedsolverreporting,tobesufficient.
Page36of68
5.21 Thereisalsoaneedtoupdatecommunications,withprizewinnersnotyetannouncedonthewebsiteandwinnersunsurewhomtocontactaboutpaymentofprizewinnings/plansforStage2whilstI2Iisrecruitingtheimplementingagent.
Partnershipsandcoordination
5.22 Astheprizeearliestoninthepipeline,A@Shasparticularlybenefitedfrompracticalcross-prizelearninge.g.onefficientcontractingmechanismsforjudges.
5.23 Winners and those involved in theprize as e.g. judges andadvisors, agreed that the awardceremonywasavaluableopportunitytohearfromwinnersabouttheirproposedactivities–andgenerated realenthusiasmamongst interviewees.There isdemand fora communityofpractice to open up opportunities for peer-to-peer learning and collective problem solving.(‘Theonusshouldbeontheorganisertofacilitatetheseinteractionsand,indoingso,empower
participants,enablelessonlearningandpreventattrition.’)
5.24 Despiteexamplesofeffectivestakeholderengagement,thereisaneedtosecureaminimumlevelofengagementwithe.g.DFIDCOanditsprogrammes.ICIMODhas,forexample,expressedinterestinplayingacoordinationroleacrossrelevantDFIDprogramming.Aligningtheprizewithgovernmentand/ordonorprioritieswillbecriticaltoitssustainability.
Valueformoney
5.25 Marketingandnetworkingeffortshaveprovenmoreresourceintensivethanenvisaged.
§ Nevertheless,theprizeappearstobedeliveringvalueformoneyinthefollowingways:
§ Winnersreportthattherecognitionofferedbytheprizehasofferedaneffectiveincentivetocontinueandexpandtheiractivities.
§ Thepotentialaddedvalueofdocumentingsolutions,providingthatthesearedisseminatedeffectively.Inacontextoflowsolvercapacitytocapturelessons,theapplicationprocesshasbeguntobuildapplicantcapacitytodocumenttheirwork.
5.26 Thenewmanagementarrangementswillimprovevalueformoney.ItisrecommendedthattheUK prize team consider well-established and networked institutions to enable the prize toleverageexistingnetworksandresources.
5.27 Otherwise,VFMmaybeimprovedmostefficientlybyaddressingtherisksidentifiedabove,withmitigatingactionsincludingsupportingwinners/Stage2applicantstoimplementtheirsolutionsand leverage funds,buildingnetworks includingcommunitiesofpractice,and strengthening‘knowledgetopolicy’linkages(see‘sustainability’below).
Page37of68
Impactandsustainability
Potentialimpactandsocialinclusion
5.28 Thereisearlyevidencethatsolutionsarecontributingtotheimprovedresilienceoftheimpactgroup,oftenworkingwiththemostmarginalisedthrough,forexample,communityusergroups.Successful innovations include improvements to water management through small scaleinfrastructureand‘climatesmartvillages’whichintegratearangeofadaptations.
5.29 NGOsandfemalecommunitymemberswereparticularexercisedaboutthegenderimpactsofsolutions:
“She[usedto]sendhersontohisschoolbutaskthedaughtertocollectthewaterandfirewood.
Butaswaterisnowavailableinthehouseholdbothofthemgototheschool.Asthewateris
availableintheirownhouses[women]areinvolvedinthecommunityactivitiesandmeetings.
And from that they are getting the respect from their husbands. In [project] community
groups…there is be more than 50% female participation. Through this, their height in the
communityhasbeenraised…Forexample,[aspecificcommunitymember]wasnoteducated
butnowshecantalkinandleadmassmeetings.Andafterbeinginvolved,shehasencouraged
her daughter to read…Even her husband has supported her. These two are farmers and
generallywehaveatraditionthatfemalesshouldnotgooutwithouttheirguardians,buttheir
husbandshaveallowedthemtocomeherewithustospeakwithyou.”
5.30 However, more concerted monitoring efforts will be required to verify that solutions areachievingpositiveoutcomesforthepoorestandmostvulnerable.ItwillalsobeimportanttocollecttangibledataontheadditionalityoftheprizemodalityduringStage2.
Sustainability
5.31 Notethatrisks,VFM,impactandsustainabilitymayallbeaddressedthroughastreamlinedsetofresponses,notleastinrelationtowinner/applicantcapacityandnetworks,asoutlinedabove.
5.32 AcoreobjectiveofA@Sistoinfluencegovernmentpolicyandpractice.Sustainabilitycouldbeimprovedbystrengthening‘knowledgetopolicylinkages’–ongoingconsultationsaroundtheNationalAdaptationPlanofferarealopportunitytoembedsuccessfulsolutionsingovernmentstrategy(e.g.ascasestudies).A@Sisparticularlywell-placedgiventhatICIMOD(whosestaffwere involved in A@S design) are coordinating the gender inclusion component ofconsultations.However, the timeframe for feeding into theprocess (c. twomonths) is tightgiven ongoing recruitment of a new implementing agent. Engagement of governmentstakeholderscouldalsobe improvedbye.g. involvingofficials inmonitoringandverificationprocesses.
Strategicrecommendations
Timeframe
5.33 A@Sisfacedwithaparticularlyshorttimescaleforachievingimpact.Theprizeisalsoseekingto target solvers with strong capacity building needs. A sixthmonth extension is therefore
Page38of68
recommendedtooffertheseapplicantssufficienttimetoimplementplansandcapturepositiveoutcomes.
Scopetoscaleupand/orreplicate
5.34 A@Smay not be themost obvious candidate for a) replication given the level of capacitybuildingrequiredforthisgroupofsolversandb)scalingwithinNepalgiventhegeographical/topographical specificityof solutions.The latterwas identifiedasa significantbarrier to thescaleupofadaptationstonationallevelbyanumberofinterviewees.
5.35 However,theprizepresentsarealopportunitytolearnlessonsaroundusingprizestodrawinanatypicalgroupofsolvers–theMTRteamsupportsI2I’srecentdecisiontomakethisafocusareaoftheprizeevaluation.
5.36 IfDFIDandotherswishtocontinueworkingwiththesetypesofsolvers,theymightconsiderhybridfundingmodalities–askingwhenthecatalyticeffectofprizesismost/leastvaluabletodeterminewhenandwhereprizesande.g.grantsaremostlikelytobeeffective.
Prioritisingprizeinvestment
5.37 Tomitigaterisks,andenhanceimpact,sustainabilityandvalueformoney,itisrecommended
thatthefollowinginvestmentsintheprizebeprioritised(indecreasingorderofpriority):
§ SolvercapacitybuildingtargetedatsmallercommunityorganisationsandgroupstobuildalevelplayingfieldforStage2applicants.Particularattentionshouldbepaidtocapacitytomonitorimplementationanddocumentresults,aswellastomobiliseadditionalfunding(H).
§ If practical, engaging with ongoing national policy development processes to embedsuccessfulsolutionsingovernmentstrategy(L).
Page39of68
Chapter6: UrbanSanitationChallenge
Background
6.1 TheUrbanSanitationChallenge(USC)isincentivisingMetropolitan,MunicipalandDistrictAssemblies(MMDAs)withapopulationofmorethan15,000toprioritisethedeliveryofimprovedurbansanitation,withtheaimofincreasingaccesstosustainablesanitationservicesforallhouseholds,withaparticularfocusonthepoor.
6.2 Thechallengeisformedof2stages:
§ Stage1(“Duapa”)offereda3monetaryprizesforthebestliquidwastemanagementstrategies,aswellas21honoraryawards.
§ Stage 2 (“the Dignified City Award”), in which 17 MMDAs are participating, will track theirperformanceinimplementingtheseplansandprovidemonetaryrewardtothosewhohavehadthegreatestimpact.
§ A baseline assessment is also being conducted to enable robustmeasurement of the impact ofimplementation.
6.3 USChasrecentlybeenawardedfundingbytheBillandMelindaGatesFoundationtoimplementasisterprize–asanitationawardschemetofundspecificthematicawardsundertheumbrellaoftheUSC–inpartnershipwiththeCommunityWaterandSanitationAgency.
Headlines
6.4 Keyachievementsinclude:
§ The development of innovative liquid waste strategies by 48 MMDAs – including examples ofinterventions across the value chain, and innovative service delivery and financialmodels – andprioritisationoftheirimplementationbythe17MMDAsparticipatinginStage2(i.e.exceedingtheexpectationthat15MMDAswouldaccessStage2).
§ Ashiftingovernmentpriorities–asdemonstratedbyaMinistryofFinancepolicydirectivebyfortheintegrationoftheUSCintobudgetguidelinesforliquidwastemanagement,andthecommitmentof17MMDAstousingassemblyfundstosupportimplementation.
§ Thecreationofaplatformfora“meetingofminds”28ontacklingsanitationchallenges.Specifically,theestablishmentofnascentpublic-privatepartnershipsbetweenMMDAsandarangeofprivatesectorpartners,withearlysignsthatthesearebeingputintopracticeasMMDAsbegintoimplementtheirstrategies.
§ MMDAsfeltthecommunicationandresponsivenessduringtheapplicationprocesswasoutstanding.MMDAs often mentioned how it was nice knowing they were notified as the application wasreceived.
28Applicants
Page40of68
6.5 Challengesinclude:
§ Orienting MMDAs on the differences between the prize modality and traditional fundingmechanisms,aswellasmanagingexpectationsaroundthevalueofprizesavailable.
§ Delayeddisbursementofcashprizestothe3winningdistrictassemblies(DAs).DAsreportthatthisislikelytoposeabarriertothetimelyimplementationofstrategies.
§ Capacitylimitationsoftheagentconductingthebaselinesurvey.
6.6 Opportunitiestostrengthentheprizeincludescopeto:
§ Engagewiththenewpoliticaladministration,includingMinistryofWaterandSanitation,tobuildontheGovernment’scommitmenttoimprovingsanitationservices.
§ BuildMMDA’simplementationcapacity,withaparticularfocusonmonitoring.
§ ContinuetosupportMMDA’stobuildandconsolidatepartnershipswith,andleveragefundingfrom,privatesectorpartners.
Application,judgingandaward
Accessibilityofapplicationprocessandqualityofapplications
6.7 Using urban communities with a population of 15,000 persons as a threshold for eligibility wasappropriate, and well-informed by consultations with key stakeholders and thorough scoping ofsanitationissuesinGhana.
6.8 Applicantsreportedthatrequireddocumentationandjudgingcriteriawereadequatelyexplained,andthattheyunderstoodthescoringsystemforeachoftheareastobeaddressedbytheapplication.Theydid,however,drawheavilyonsupportivematerialsavailabletoapplicants,includingresourcematerialsonthewebsite,helpdeskandfeedbacksystems.
6.9 Thequalityofapplicationswasvariable,inpartaccordingtothecapacityofindividualMMDAs.Overallthequalitywasdescribedbyteammembersandjudgesasaverage.However,certainapplicationswereofparticularlyhighquality,includingstrategieswhichspannedtheentiresanitationvaluechainand/orinnovativeservicedeliveryorfinancingmodels.Whilethesupportinadministeringtheapplicationwasstrong,MMDA’sdidseemtorequiremoresupportandpotentiallytimeincraftingtheirplans.For“some
ofthedistrictsitwasthefirsttimetheywerecomingoutwithastrategy”(MLDRG)andsomeMMDAssimplysubmittedtheiralreadyexistingplansthatdidn’texactlymeetthecriteria.ItmaybeusefultoprovidemorecapacitybuildingsupportonstrategycreationandtrytoaligntheawardtimelinewithanyexistingMMDAstrategycreationtimelines.Onejudgementioned“ForinstancesomeoftheMMDAs
hadtospendhugeamountstohireconsultanttohelpthemproducetheirproposals”.
Effectiveness,adequacyandefficiencyofjudgingprocess
6.10 Thejudgingprocesswasagreedtobeeffective,withtheuseofexternalexpertshelpingtoensureitscredibility.ThetransparencyoftheprocessandprovisionoffeedbackwasemphasisedasapositivebyseveralMMDAs:
Page41of68
“Thejudgingcriteriawasverygoodbecauseitguidedyouonwhattodoandshowedwhat
markswouldbeallocated foreachof thequestions in theapplication. Andthenwhenthey
finishedtheysentitbacktoustoshowourperformance.”
6.11 MMDAsalsoparticularlyappreciatedtheopportunitytopresenttheirstrategiesinthefinalphaseofjudging.Judgesandteammembersperceivedthisphasetobecriticaltotherigouroftheprocess.
Outreachandparticipation
Sufficiencyofoutreach/levelofpromotion
6.12 Theprizefoundisrelativelystraightforwardtoreachpotentialapplicants,giventhatitwasworkingwithaknowngroupof solverswith ready-made institutional structures to facilitatecommunications (theMinistryofLocalGovernmentsentoutacirculartoallMMDAs).Face-to-faceworkshopswere,however,importanttocreateunderstandingandgalvaniseinterestintheUSC.
6.13 Traditionalmediai.e.newspapersandradio,wereusedtoreachthoseoutsideofMMDAsandfoundtobeeffective.Widerawarenessraisingamongstbeneficiariescouldmakegreateruseofsocialmedia(particularlyWhatsApp which is popular amongst both the literate and non-literate population) togeneratedemandforimprovedsanitationservices.
Sufficiencyandappropriatenessofincentivesforapplicants
6.14 Theresponsethattheprizereceivedfromthecallforapplicationsisagoodindicatorofthesufficiencyof the incentive for applicants: “For the first time we have seen MMDAs prioritising liquid wastemanagement”.29Theprimary incentiveforMMDAs,aswellasprivatesectorpartners,wasgenerallyagreedtobethe ‘prestige’thatcomeswithassociationwiththeprize.MMDAsalsonotedthattheyotherwisefinditdifficulttosourcefunding–andthattheprizehasthepotentialtohelpthemleveradditionalresources.
6.15 SomeMMDAsexpresseddissatisfactionregarding thedecisionnot toawardanymonetaryprizes toMetropolitanandMunicipalAssemblies,arguingthattheirstrategiesweretailoredtowardsaddressingspecificgapsinliquidwastemanagementinpoorcommunities:
“Over time the goal posts kept changing. We were told that that there was going to be two
categories–oneformetropolitan/municipalandanotherforthedistrictsandthateachwillhave
threewinners.Sowhenwewereshortlistedamongstthelastsixwewereoptimisticthatwewillat
leastwinsomethingonlyfortherulestobechangedonthedayofprizeaward.”
6.16 Itisworthnoting,however,thatthisdecisionwasbasedonthequalityofapplicationsreceived.Whilstthis caused some confusion and disillusionment amongst applicants, it is clear that the risks wereconsideredandrationalecommunicatedappropriatelybytheteam.
6.17 Someprivatesectorpartnersfeltthatthattheyshouldalsobeabletogainfromtheirinvolvementintheshort-termbyreceivingaprize,aswellasthattheprizevaluewastoolow.However,basedonthe
29Implementingagent
Page42of68
levelandqualityofapplications,anddegreeofprivatesectorinvolvement,theteamappearsthehavepitchedthetypeandvalueoftheStage1prizeappropriately.
Prizemanagement
Governanceandrisk
6.18 In-countrymanagementbyIRCisofgoodquality,andcollaborationwiththeUK-basedteamiseffective.Theteam,inboththeUKandGhana,demonstrateastrongawarenessandunderstandingofdeliveryrisksandhavegivencarefulconsiderationtomitigation.Theserisksareasfollow:
§ Thechallengesposedbypoliticalchangesandprotractedpost-electiontransition.Withachangeintheparentministryforsanitationissues,thenewministerandseniorofficialsneedtobeengaged.Atlocallevel,untilDistrictChiefExecutive(thepoliticalheadsofMMDAs)areappointed,mostofthemajoractivitiesoutlinedinthestrategiesproposedbytheAssemblieswillnotprogress.AstheUSCrequires MMDAs to use their own funds to support implementation, reprioritisation and/oradministrativedelayscouldalsoaffectimplementation.
§ MMDAcapacitygapspertinenttoimplementation,mostnotablycapacitytoconductmonitoring/reportingactivities.
§ Compressedtimelinefordelivery.
.Monitoringandevaluation
6.19 Thecapacityof theagentcarryingout thebaselinehasbeenweak.The teamhave,however, takenappropriatemitigatingactions,byremovingthecommunity-basedelementofthesurveyi.e.restrictingscopetoassemblylevelinformation.
6.20 There is a consensus that the USC has, until now, placed greater emphasis on design andimplementationthanmonitoringand learning.However, IRChas recentlyestablisheda learningandsharingplatformwithMMDAsviaWhatsApp,heldalearningworkshoptocollateviewsonchallengesandlessonsfromapplicantsandstakeholders,anddocumentedlearnings.Applicantsagreedthattheseinitiativesarepositive,but thatmoreemphasisneeds tobemade toaddress their capacitygaps toenablethemtoapplylessonsandmonitorimplementation.
Partnershipsandcoordination
6.21 I2I’s relationship (MoU)with theMinistry of LocalGovernment andRuralDevelopmentworkswell,lending USC the authority and legitimacy needed tomobiliseMMDAs. That USC has recently beenawardedfundingbytheBillandMelindaGatesFoundationtoimplementasisterprizeinpartnershipwiththeCommunityWaterandSanitationAgency,demonstratestheeffectivenessofI2Iengagementwithgovernmentstakeholdersanddonors/fundersworkinginthesamespace.
6.22 Public-privatepartnershipsformedthroughtheUSCweredescribedasa“win-win”30forbothparties.
30MMDAs,privatesectorpartners
Page43of68
§ Privatesectorpartners learntaboutthechallenge independentlyofMMDAs,andreportthistohavestrengthenedtheirpartnerships(e.g.levelsofunderstanding,efficiency).Settingoutthenames of private sector partners on the project website enabled MMDAs to identify potentialcollaborators.
§ Privatesectorpartnershaveexpressedmostsatisfactionwiththeperformanceofpartnershipssofar,wheretheywereactivelyinvolvedintheapplicationprocessi.e.abletofeedintoandmobilizeresourcestosupporttheMMDAsstrategies.Thesepartnershipsarebeginningtobeoperationalisedasimplementationcommences.
§ Private sector partners see themselves as offering themost value in a) building the capacity ofMMDAs to deliver sanitation services via the development of a strong cadre of MMDA wastemanagement experts and b) marketing successful solutions and products developed duringimplementation.
6.23 Some MMDAs have also initiated contact with local NGOs working in the sector to support theimplementationoftheirstrategies.
Impact,sustainabilityandvalueformoney
Potentialimpactandsocialinclusion
6.24 Strategies appear to be targeting the sanitation challenges of poor and low-income groups. In allMMDAs interviewed, for example, strategies targetedpoor communitieswhere thepopulation lackdomestictoilets,resortingtoopendefecation,andwhereincomelevelsaretoolowtodevelopthebasicinfrastructurerequiredtocounterthis.Almostallstrategiesspannedtheentirevaluechainof liquidwastemanagement,includingconstructionoftoiletsinhomesandpublicfacilities(markets,transportterminals),collectionandtransportationoffaecalsludge,andcomposting.
Sustainabilityandvalueformoney
6.25 TheMinistryofFinancehasissuedapolicydirectivefortheintegrationoftheUSCintobudgetguidelinesfor liquid waste management, whilst most of the MMDAs that qualified for Stage 2 have passedresolutionsfortheuseofassemblyfundstoimplementtheirstrategies.ThatassemblieshavebeguntousetheirownfundstoimplementstrategiesbeforeUSCprizewinningshavebeendisbursedrepresentsstrongvalueformoney.
6.26 InallMMDAsthebudgetrequiredforthestrategiesproposedwasgreaterthantheprizemoneybeingoffered, and does not include budget for subsequent scaling of successful solutions. With Ghanabecoming a middle income country, grants as a mechanism for funding development activities ofgovernmentisonthedeclineandhenceisanunsustainablewayoffundingsanitation.Theroleplayedbytheprizeinleveragingprivatesectorfundingoffersawelcomealternative.
6.27 MMDAs awarded cash highlighted the risk of financing the gap between prize money and projectbudget.MMDAs not awarded cash but receiving an honour award asked for othermechanisms tosupporttheirproposals.Thereisanopportunityfortheawardprocesstoactasafilterforotherprivatepartners to support screened projects. This may also help reduce award fatigue and maximisebeneficiariesserved.
Page44of68
“TheriskreallyhastodowiththeMMDAsbecomingdisinterestedovertime.Nowitisa
newthingbutiftheresultisonlylookingatprizewinnerswithoutapplaudingtheworkthatisbeing
putinthenIfearthenon-prizewinnerswillbackout.”
6.28 ThereisevidencethatthedevelopmentofthestrategiesbytheMMDAsasaresultoftheSanitationChallengehasbegun toopenupopportunities to attract funding from theprivate sector andotheractors.Forexample,AtiwaDistrict,oneofthecashprizeawardwinners[fortheirliquidsmanagementplan] has since secured the support of the Ernest PeryerMemorial Foundation and [other] privateindividualstosupportcapitalcostsoftheimplementation.
Strategicrecommendations
Timeframe
6.29 Despitethecompressedtimeframe,theMTRdoesnotrecommendanextensiontotheprizetimeline.However,potentialdelaystoStage2asaresultofpost-electionchangesinGhanashouldbemonitoredcloselyoverthecomingc.3months,andthequestionofanextensionberevisitedshouldtheknockoneffectsprovesignificant.
Scopetoscaleupand/orreplicate
6.30 Itisrecommendedthat,giventheleveloftractionachievedwithgovernment,effectiveincentivisingofpublic-privatepartnerships,andpotentialforsustainabilitythatopportunitiestoreplicatetheprizeareinvestigated.InfacttheGatesFoundationisalreadyreplicatingpartsoftheprogramwithIRC.TheMTRrecommends that the prize is not scaled in-Ghana (given the appropriateness of the threshold foreligibilityaroundsizeofcommunityserved)butreplicatedelsewhere.Giventheuniquecharacteristicsoftheprize–particularlyitssuccessinbuildinglocalcoalitionsaroundsanitationproblemsinaspecificsetting–itshouldbereplicatedatcountrylevelratherthanscaledglobally.
Prioritisingprizeinvestment
6.31 Tomitigaterisks,andenhanceimpact,sustainabilityandvalueformoney,itisrecommendedthatthe
followinginvestmentsintheprizebeprioritisedequally:
§ IncreasecapacitybuildingforMMDAsthroughlearningandpracticeworkshops,includingcapacitytomonitorimplementation(H).
§ BudgetforthedevelopmentandmanagementoftheCWSAsisterprize(L).
§ ScopingforreplicationoftheUSCi.e.toidentifychallenging,‘prizeable’problemsandappropriatecountrysettingswithstrongprizeecosystems,notleastanenablingpoliticalenvironment(VH).
§ Duetothecreationofstrategicplans,thereisanopportunityfortheawardprocesstoactasafilterforotherprivatepartnerstosupportscreenedprojects.Thismayalsohelpreduceawardfatigue,maximizebeneficiariesserved,andincreasefollow-oninvestment.
§ MMDAsstrategicplansshouldalsoincludeafinancingplantoensurethattheMMDAcanleveragefunds,whetherthosebeinternal,external,andtheprizemoney,toimplementtheproject.
Page45of68
Chapter7: DreampipeII
Background
7.1 DreampipeisaglobalWASHprize(focusedonDFIDprioritycountries)thatseekstoencouragethedevelopmentofworkableandreplicableideasforexpandingthefinancingavailablefornon-revenue-water31(NRW)reductionactivitiesindevelopingcountries.
7.2 DreampipeIIisformedof3phases:
§ Phase 1 – awards will be made for the best business plans setting out how Stage 2requirementswillbecarriedout.
§ Phase2–applicantswillbeassessedoncarryingout(andfullydocument)ademonstrationproject to reduce NRW in a selected water utility, as well as an updated business planfocusingonfinancingandcontractingforamajorexpansionprojectinthesameutility.
§ Phase3–applicantswillsubmittermsheetsforallmajorprojectandfinancingagreementsneeded for the expansion project in the selected utility, including a performance-basedcontract.
7.3 Theoriginalprize–DreampipeI–waslaunchedinFebruary2016andawardedinJulyofthesame year. Dreampipe has since been redesigned to create a more focused, tournamentapproachwiththeintentionofminimisingrisk.
Headlines
7.4 Keyachievementsinclude:
§ Timelyadaptationinthedecisiontoclosedown,redesigntheprizeprogrammedrawingonlessonsfromDreampipeI,andlaunchingDreampipeII.
§ ExtensivepromotionofDreampipeIIinrelevantinternationalforaandsectorpublications,andpromisingapplicationnumbers.
7.5 Themostnotable challenges facedby theprizeprogramme related to thediverse solutionsgenerated by Dreampipe I applicants, which would not have allowed the team to form acoherentportfolio.However,asnotedabove,theteamhaslearntvaluablelessonsfromthisexperienceandputthesetogoodusewhenredesigningtheprize.
7.6 Opportunitiestostrengthentheprizeincludescopeto:
§ Refineamarketingmodelforglobaldevelopmentprizes–marketingforglobalratherthancountry-specific prizes poses particular challenges, but there are early indications thattargetinginternationalnetworks/communitiesofpracticehasbeenaneffectivemarketing
31NRWisdefinedaswaterthatisproducedbutlostbeforereachingtheconsumer,orisimproperlybilledfor.
Page46of68
tool.Inphases2and3theteamshouldcontinuetorefinetheirapproachandcouldcaptureusefullessonsaroundwhichtechniquesworkbestforwiderdissemination.
§ Considerloweringthethresholdforeligibilityrelatingtothe‘minimumnumberofpersonsimpacted’, placing slightly less emphasis on direct beneficiary numbers to ensure thatgenuinely innovativesolutionswith lowerbeneficiarynumbersbutthepotentialfor largeindirectimpactarenotruleout.
§ Consider – depending on the applications received for Phase I and type of applicants –facilitatingparticular individualapplicantsto jointogethertoforminterdisciplinaryteamswithcomplementaryexpertise.
DreampipeIandredesign
LimitationsofDreampipeI
7.7 DreampipeIproducedtoowideanarrayofsolutionsfortheteamtoformacoherentprofile,withtheparameterssetoutforapplicantsprovingtoobroadandunspecific.
7.8 Marketing a global prize has proven challenging (given the large applicant pool and limitedresources)comparedtoin-countryprizeswhichcanpursuemoreintensiveengagementwithasmaller pool of applicants. Application numbers were lower than hoped with only a smallproportion of those who had shown interest submitting applications. Approximately 400visitorspermonthtothewebsitewererecordedwithintheapplicationwindow,46registrationswere received,ofwhich20applied–a relatively steepdropoff rate.The spreadof typeofapplicantswas also limited,with themajority alreadyworking in thewater sector (i.e. notdrawinginnewordifferenttypesofsolvers).
Redesignprocess
7.9 Thedecision to redesignDreampipe represents a goodexampleof theprogrammeworkingadaptively andusingaproblem-centric approach. The teamconsultedwithawide rangeofstakeholders including prize judges, sector experts, and financiers working in developingcountries to develop a clear understanding of the issues/problems that needed to beconsideredfortheredesign.
7.10 Whilst the end goal of Dreampipe II remains the same, the redesign has narrowed theparametersoftheproblem,aswellasthepathwaysthroughwhichsolversareincentivisedtotackleit,withachangeoffocusfromfinancingmechanismstosourcesoffinancing.Theteamhavealsoactivelyencouragedsubmissionsfrominterdisciplinaryteams.
ImplementingDreampipeII
Outreachandsupportiveactivities
7.11 Theteamhaveadoptedamorestrategicmarketingapproach,andappeartohavehadsuccessinworkingthroughrelevant international forae.g.2016Water Ideasconference,andsectorpublicationse.g.GlobalWaterIntelligencemagazine,withnumbersofapplicationspromising.
Page47of68
7.12 Applicantsarenotprovidedwithtechnicalassistanceduringtheapplicationprocess,butaregivenadhocguidancewhennecessary.Thislevelofsupportislikelytobesufficientgiventhatapplicantsareexpectedtohaverelativelyhighcapacitylevels.
7.13 Asforeligibility,therequirementthatapplicantsmustbeworkingwithatleast150,000personscouldbelimiting.ThisisoneinstancewhereI2Icouldreconsiderthebalancebetweendirectandindirectoutcomes–thatis,betweenbeneficiariesimpacteddirectlybythesolutionsbeingdeveloped and implemented under the prizes, and inducing the development of reallyinnovative solutions with potential for largescale, albeit indirect, impact. Rather than athresholdforeligibilitybasedonbeneficiarynumberswhichrulesoutsmallerinitiatives,scoringcriteriacould includebeneficiarynumberstoensurethattheprogrammereaches its targetswhilstallowingsmaller,buttrulyinnovativeinitiatives,tocompete.
Prizemanagement
Governanceandrisk
7.14 As a global prize managed in-house by IMC, Dreampipe benefits from particular efficientmanagementarrangements.
7.15 Asmentionedabove,theredesignprocessrepresentsagoodexampleofadaptivemanagementwith the prize team placing particular value on the flexibility of the programme and the“problem-solvingenvironment”itenables.
7.16 Themain risk to implementation relate to applicant capacity – specifically, to run Phase 2demonstrationprojects andpreparing term sheetsduringPhase3. The successof theprizedependsontheapplicantsbeingableandwillingtoundertakelargeinvestmentswithrelativelylittlesupportandfeedback.
7.17 Itisworthmentioningthatcertainsector-specificrisksremainbeyondthescopeofsolutionsbeing inducedbytheprizeprogramme–mostnotably,poorsectorgovernanceandpoliticaleconomyaspectsofwaterutility(e.g.rentseeking).
Monitoringandevaluation
7.18 DreampipeisoneofonlytwoprizeswherethereisstillscopeforStage1learningtofeedintolaterstages(or ‘phases’ inDreampipe’scase). If theteamdochoosetocarryoutevaluationafterphases1and/or2, this shouldbe carriedout in timelyway to feed intodesignof thesubsequentphase.Amodestincreasetotheevaluationbudgetwillbeneededtoresourcethis.
7.19 There is consensus that the team has learnt important lessons from the limitations ofDreampipeI–relatingtodefiningtheparametersoftheproblemtobetackledandhowglobalprizes canbemosteffectively andefficientlymarketed. Theprogrammeshouldensure thattheselessonsarefedintoprogrammelearningstreamstoinformlearningproducts.
Page48of68
Strategicrecommendations
Timeframe
7.20 GiventhetimethatwasrequiredtoredesignandrelaunchDreampipe,theMTRrecommendsasixmonthextensiontoenablesufficienttimetoachieveandevaluatepositiveoutcomesforbeneficiaries.
Scopetoscaleupand/orreplicate
7.21 Dreampipeworkswithrelativelywell-capacitatedsolversandsomaybeagoodcandidateforscaling or replication in specific contexts. There is, however, not yet sufficient evidence tosupport such a decision. If the prize is able to attract a sufficient number of high qualityapplicationsforPhaseIandstarttobuildcoalitionsofdiverseactorstotackleNRWreduction,thiswillprovideanearlyindicationofpotentialimpactandsustainability.
Futureinvestment/Scopetoscaleupand/orreplicate
7.22 Toenhanceimpact,sustainabilityandvalueformoney, itisrecommendedthatthefollowing
investmentsintheprizebeprioritised(indecreasingorderofpriority):
§ CapacitybuildingforapplicantstosupportthemtorunPhase2demonstrationprojectsandpreparetermsheetsduringPhase3.Facilitatingpartnershipsbetweenparticipants,withafocusonoutreachtofinanciers(M).
§ Responsive end phase evaluations should the team choose to run evaluations followingphases1and/or2(M).
Page49of68
Chapter8: GlobalLEAPRefrigerationPrize
Background
8.1 I2I issupportingaDFID/USAIDcollaborationontheGlobalLEAP(LightingandEnergyAccessPartnership) awards – an existing programme operated by the NGO CLASP – by providingfunding to the prize purse for its off-grid refrigeration prize and running field testing ofshortlistedappliances.
8.2 I2I is also developing a sister prize for larger scale refrigeration – the Off-Grid Cold ChainChallenge(OGCCC)–forwhichtheconceptnoteshasrecentlybeenapprovedbyDFID.Theprizewillbeformedoftwostages,thefirstfocusingonideationandtheseconddemonstrationofresults.
8.3 The original design of I2I’swork in the energy access sectorwas formed of a trio of prizes(unrelatedtotheGlobalLEAPawards)tosupportashiftinpolicytosupportLPGusageinGhana.Ofthese,oneprize–‘theCylinderPrize’–toidentifyappropriatemethodstorecycleandreuseoldLPGcylindersthatdidnotmeetnewsafetystandardswasimplemented,buttheprogrammewasdiscontinuedowingtochangesinthepolicyenvironmenti.e.thegovernment’sdecisionnottoimplementtheLPGpolicy.I2I’senergyaccessworkhassincebeenredesignedarounditsengagementwiththeGlobalLEAPawards.
Headlines
8.4 Keyachievementsinclude:32
§ DevelopingastrongandeffectivepartnershipwithNGOCLASP,theleadimplementeroftheGlobalLEAPAwards.
§ Timely launchoftherefrigerationprizeandeffectiveoutreachtosolvercommunity,withnominationsexceedingtargets.
8.5 Challengesinclude:
§ ResourceconstraintsslowedtheprocessofscopingandpreparingaconceptnotefortheOGCCCgiventhecomplexityofthesubjectmatter.AbetterresourcedprojectmanagerwasappointedbyE4IinOctober2016toaddresstheseconstraints,withtheconceptnotenowapprovedbyDFID.
8.6 Opportunitiestostrengthentheprizeincludescope:
§ Toensure that the Energy 4 Impact prizemanager is fully involved inadhoc cross-prizelearning,giventhedifferentmanagementarrangementsforthisprize(withI2I’sinvolvementintheGlobalLEAPawardsmanagedentirelybyEnergy4Impact).
32ItisworthnotingthatadaptingtothechallengesfacedintheCleanCookingPrizeinGhanabyidentifyingitsinabilitytoprogress
Page50of68
Partnershipsandcoordination
8.7 GlobalLEAPisrunbyCLASP,andfundedprimarilybyUSAID.Thenumberofdifferentdonorsandimplementersinvolveddoessometimespresentchallengesastheresultofslightlydifferingexpectations.However,overallthestrengthofI2I’sinvolvementhasbeentoaddvaluetoanexistinginitiative.
8.8 Therefrigerationprizeoffersausefulexampleofthepotentialvalueofdonorandimplementerpartnershipsoninnovationprizes–aswellasofembeddingprizeswithinexistingprogrammesorinitiatives.Inadditiontoincreasingtheprizepurse,withCLASPalreadywell-networkedwithacommunityofsolversandworkingto identify ‘best inclass’products, I2Ihasbeenable tosupportthemtointroduceastrongerinnovationfocus,mostnotablybyrunningupcomingfieldtesting(forwhichCLASPdidnothavetheresources).
8.9 FacilitatingpartnershipsamongstpotentialsolversundertheOGCCCprizecouldhelptoensureimpact i.e. between manufacturers of technology and suppliers who understand marketdynamicstoenabledisruptionofexistingsupplychainmodalities.
Outreachandparticipation
Drawinginsolvers
8.10 ThepartnershipwithCLASPhasprovidedaccess to a known communityof solvers (existingnetworksworkingonoff-gridrefrigerationinitiatives),whichhasbeencriticaltoachievinggoodlevelsofparticipationintherefrigerationprize,particularlygiventhechallengesassociatedwithmarketing global prizes. The number of nominations has exceeded expectations, with 55nominations received from 28 different companies based in 14 countries (of which 7 aredeveloping).
8.11 Theteamhavelearntfromthisexperienceandadoptedastrategicapproachtothepromotionofthesisterprize,bypursuingpotentialpartnerships(e.g.withtheShellFoundation,theGlobalColdChainAlliance)thatcanprovideaccesstostrongsolvernetworks.
8.12 I2Ihasaddedvaluetotheprize’sabilitytodrawinsolvers,withtheofferofvaluableconsumerfeedbackthroughfieldtestingformanufacturersanddistributorsreportedtohaveattractednewentrants.
Diversityofsolutionsproposedandadditionalityofprizemodality
8.13 Applicationsincludeadiversityofappliancesandproducts,including45commerciallyavailableproductsand9latestageprototypes.Therearesomeindicationsthattheprizehasplayedacatalyticeffect,withsomebusinessesreportingrushingprototypesinordertobeabletoenter.
Page51of68
Supporttosolvers
8.14 Smallerorganisations’abilitytocompeteeffectivelywouldlikelybenefitfromcapacitybuildingactivities.However,suchactivitiesarelikelytobeimpractical(anddisproportionatelycostly)inthecontextofaglobalprize,butcouldbeconsidered ifthesetypesofsolvershappentobeclusteredinaparticularregion.AswithotherI2Iprizes,theprioritywouldbetosupportsolverstoengagewithothersourcesoffunding.
Prizemanagement
Governance
8.15 I2I’s involvement in the Global LEAP Awards is managed entirely by Energy 4 Impact. Themanagement relationship between the Programme Management Unit and implementersappearstobecollaborativeandeffective.Withtheprizeteamrelativelynewtotheprogramme,there were some teething issues with the setting up and communication of programmemanagementsystems.However,withcommunicationswithPMUeffective, such issueshavealwaysbeeneasilyrectified.
ManagementoftherelationshipwithCLASP
8.16 Theprize teamappear tobemanaging the relationshipwithCLASP, includingdisbursementrequests,effectively.ThereportingdataprovidedbyCLASPissufficient,butregularI2I-CLASPmeetingsarenowhappeninglessfrequentlythanpreviously–giventhattheserepresentthemainforumforreportingdeliveryrisks,itmaybeusefultoreintroducemoreregularmeetings.
Monitoring,evaluationandlearning
8.17 I2IhasinvestedresourcesinlearningfromtheexperienceoftheGhanaLPGProgramme.Whilstitisclearthatthesecondandthirdprizesbecameunviableastheresultofchangesinthepolicyenvironmentand thedecision to closedown theprogramme represents a goodexampleofadaptation,theteamshouldensurethatanywiderlearningsarefedthroughintoprogrammelearningstreams.
8.18 I2I’s redesigned energy access stream candemonstrate several examples of effective cross-prize learning.Most notably, the team is currently applying learning from elsewhere in theprogrammetoinformthedesignoftheOGCCC.TheseincludelessonsfromDreampipearounddefiningparametersoftheproblemtobeaddressedinthecontextofaglobalprize,andfromtheexistingGlobalLEAPrefrigerationprizearoundusingpartnershipstoidentifyandaccessthe‘right’solvers.
8.19 Nevertheless,as IMChasbeenrequiredtoabsorbmoreof themanagementof the I2Iprizeportfoliotherehasbeenadropininformationsharingbetweentheprizes.TheIMCteamshouldbear in mind the need to engage the Energy 4 Impact team in opportunities for internalinformationsharing,giventhatmostcross-prizelearningtakesplaceonanadhocbasis.
Page52of68
8.20 Field testingwill beused to inform learningproducts – and (although the technology focusdiffers fromother prizes in the I2I portfolio) offers a good example of capturing prize levellearningaroundsolutions forglobalpublicgoods.Byplacing shortlistedproductswith smallretailersinUgandaandprovidingthemwithanappropriatesolarpowersupply,theteamwillassessefficiencyaswellasthevaluepropositionfortheuser:
“Sothatwecanthencomparepotentialeconomicvaluewiththecostoftheproduct,andhelp
themarket understandwhat sort of price points andwhat sort of paymentplanswouldbe
necessaryinordertomakethistypeofproductaffordabletoatypicaluserinanoff-gridarea.”
Strategicrecommendations
Timeframe
8.21 The team plan to use a rolling entry system under the OGCCC to compress timeframes byenablingStage2tobeginassoonasStage1hasbeenawarded.Nevertheless–giventhattheGlobalLEAPrefrigerationandsisterawardareatanearlierstagethanotherprizeprogrammesintheI2Iportfolio–theMTRrecommendsasixmonthextensiontoallowsufficienttimeforsolverstoachievepositiveoutcomesforbeneficiaries.Anextensionwillalsohelptoopenuptheprizetonewsolvers;theshorterthetimeframe,themorelikelytheprize isto‘lockout’organisationswhicharenotwell-establishedand-capacitated.
Scopetoscaleupand/orreplicate
8.22 Theteamisalreadyeffectivelysupportingandscalingtheexistingrefrigerationprizethroughthedesignofacomplementarysisterprizefocusingonlargerscalerefrigerationforcommercialpurposes.
Prioritisingprizeinvestment
8.23 Toenhanceimpact,sustainabilityandvalueformoney,itisrecommendedthatthefollowinginvestmentsintheprizebeprioritised(indecreasingorderofpriority):
§ InvestmentintheOGCCCprizepurseneededforI2Itomoveforwardwiththedevelopmentoftheprize(VH).
§ LighttouchcapacitybuildingofOGCCCparticipantstosupportthetransitionfromStage1toStage2,includingfacilitatingpartnershipsamongstpotentialsolvers(M).
8.24 Capturinganddisseminating learningfromtheGlobalLEAPrefrigerationprizewillalsobeofcritical importance.AlthoughfallingbeyondI2I’spurview, theprogrammeshould, ifneeded,influencethewiderGlobalLEAPprogrammetoensurethatsuccessfulsolutionsarepromotedeffectively(inthecasethatGlobalLEAPisabletoclearlydefinetheparametersofanappliancethatrepresentsagoodvaluepropositionforthemarketandbeneficiaries).
Page53of68
Chapter9: Programmemanagement
Governanceandmanagementarrangements
Managementstructure
9.1 Leadershipandmanagementoftheprogrammeisverystrong,andperceivedassuchbyteammembers throughout the programme’s organisational structure. Restructuring centralprogramme management arrangements to bring prize managers in-house in 2015/16strengthenedthequalityofcross-prizeinteractionsandresultedinefficiencygains.
9.2 Both Itad and implementing agents report relationships with the PMU and in-house prizemanagerstobeeffective,openandcollaborative,whilstin-countryprizemanagersaresatisfiedwithlevelsofoversightanddelegation.
9.3 However,workingwithlocalimplementingagentshasinvolvedhighermanagementcoststhananticipated. Delegating additional responsibilities to in-country prize managers has provenchallengingowingtocapacitylimitations.
Adaptivemanagement
9.4 I2Iwasnotexplicitlydesignedasanadaptiveprogrammebuthasbecomesoorganicallyowingto the innovative nature of the programme and its activities. DFID has provided effectivesupport to this approach by offering a degree of flexibility onmanagement arrangements,includingbudgeting,whilstretainingeffectiveoversightofimplementation.However,workingflexiblyandresponsively,includingeffectivemanagementofResultsFramework,hasrequiredmoremanagementtimethanoriginallyenvisaged.
9.5 Initsearlierstages,theprogramme,attimes,found‘failingfast’tricky(i.e.identifyingandeitherhaltingoradaptingspecificapproacheswhicharenotworkingwell)–likelyasaresultofthechallengesassociatedwithstructuringprizesaroundparticularlyintractableissues.However,itcanalsopointtoeffectiveexamplesofadaptationinresponselessonslearnedandcontextualchanges – most notably, closing down the Ghana Clean Cooking prize and refocusingDreampipe.
9.6 The programme could domore to systematically capture and share lessons about workingadaptivelyintheinnovationsectorandonprizesspecifically.Atpresent,instancesofadaptationaredocumentedbutnotproducedforanexternalaudience.
Riskmanagement
9.7 ThePMU,in-houseandin-countryprizemanagersdemonstrateasoundawarenessofmajorriskstodelivery,whichtendtocentreonthefollowingissues:
§ Operating environment (e.g. changes to political environments, security and corruptionconcerns,andnaturaldisasters).
Page54of68
§ Insufficientsolverinterest(lowregistrationnumbers,poorqualitysubmissions)
§ Participant attrition (as a result of e.g. misjudged risk/reward curve, inadequate solvercapacity)
8.1 Riskismanagedeffectivelyatprizelevel,withteamsgivingcarefulconsiderationtothescopeformitigationandmakingalterationstoimplementationaccordingly.
8.2 Thereispotentialtoformaliseandcentraliseriskmanagementprocesses.Riskisownedbyprizemanagers rather than centrally and not currently reviewed as part of regular programmemanagementprocesses.Significantchangestotheriskregisterarecurrentlyreportedformallyonanannualratherthanquarterlybasis.Itisrecommendedthatalighttouchupdate–forexample,intheformofanissueslog–beincludinginquarterlyreporting.
Valueformoneyandfinancialperformance
9.8 Economy:
§ Managementcostshavebeenhigherthanoriginallyenvisaged,buttheteamhaveworkedhardtomaintaintheproportionofbudgetallocatedtoprogrammemanagementrelativetothe prize purse and other in-country activities. In 2016 central administration andmanagementconstituted8%oftotalspend.
§ Theproportionof junior (vs. senior)days is reported tohave increasedover time, albeitincrementally.Theprogrammehasalsoderivedeconomyfromthefact thatallUK-based(andmostin-country)programmestaffarepart-time.
9.9 Efficiency:
§ Therelativeefficiencyofdeliveryacrossthedifferentprizeshasbeendrivenprimarilybytheinterface between prize managers and implementing agents – specifically, the level ofoversightthathasbeenneeded/ofdelegationappropriate.Theglobalprizesunsurprisinglyoffer themost efficientmanagement arrangements,33whilst valuable lessons have beenlearntaroundtherelativeefficiencyofcontractingandimplementingagentandrecruitingin-countrystaff.
§ Itshouldbenotedthatefficiencymaydecrease,iftheprogrammeistoengageinthesolvercapacitybuildingactivitiesnecessarytomitigaterisksaroundimplementationofsolutionsandStage2award,andensuresustainableimpact.
§ Theprogrammehasalsobenefitedfromflexibilityofstaffresourcesacrosstheprizesatprizemanagerlevel.
9.10 Effectiveness:
33Theefficiencyoftheglobalprizesisalsoenhancedbysimplerprizeecosystemsanddesigns,althoughmarketingcoststendtobehigherthanisthecaseforcountryspecificprizes.
Page55of68
§ Programme documents currently suggest a wide range of measures of effectiveness –certain ofwhich, for example, those relating to variance between forecasted and actualbudgets or planned and actual delivery dates for key outputs, may create unhelpfulincentivesfortheprogramme,particularlygivenitsadaptivemodeofworking.
§ Effectiveness should be measured more straightforwardly, with reference to logframeoutcomes–i.e.O1)thedevelopmentanddeploymentofinnovationsolutionsandachievingofpositiveoutcomesforbeneficiaries,O3)influencingotherstoimplementprizes‘better’andO4)developingarobustevidencebasecentredonprogrammelearningstreams.
§ The first of thesewill be best demonstrated through evaluations’measurement of prizeeffects,withcompletedStage1evaluationspromising.Moreattentionshouldbedevotedto knowledge sharing and influencing to ensure that the programme not only tests butcaptures and disseminates lessons around its core hypothesis (and in doing so deliversagainstoutcomes3and4).
9.11 Equity:
§ Women’srepresentationamongstsolversvariesconsiderablyacrosstheprizes.Inthecaseof CIP, for example, significantly fewer women submittedWazo applications than men.However,womenappeartosubmitrelativelystrongerapplications,withaconversionrateof32%(winnersasapercentageofcompletedapplications)comparedto7%formen;almosthalfofwinnerswerewomen.Bycontrast,inthecaseofA@Snoneofthe15prizewinnerswerewomen’sorganisations,whilstnoneof thewinners sentwomen representatives toparticipate in the presentation of their solutions after award. Barriers include capacitylimitationsandpersistentsocio-culturalbarriers.
§ Bycontrast, there isemergingevidencethatsolutions inducedacrossdifferentprizesaregeared towardsbenefittingpoorandvulnerablegroups.BothCIPandA@Ssolutionsareworkingthroughwomen’sgroupstotacklethespecificvulnerabilitiesfacedbywomen(e.g.time and information poverty), aswell as to strengthen their voice and decision-makingpowerwithincommunities.
Page56of68
Chapter10: Strategicrecommendations
Programmeduration
10.1 Delays to implementation have resulted from the innovative nature and complexity ofprogramme design (and consequent programme adaptations), as well as unforeseeablecircumstances – not least the April 2015 earthquake in Nepal. It is recommended that theimplementationtimelineforcertainprizesbeextendedbysixmonthstoallowsufficienttimeforthedevelopmentanddeploymentofsolutions,anddemonstrationofpositiveoutcomesforbeneficiaries.
10.2 Specifically,thefollowingprizeswouldbenefitfromasixmonthextension:
§ AdaptationatScale
§ DreampipeII
§ EnergyAccess
10.3 Inaddition,potentialdelaystoStage2oftheUrbanSanitationChallengeasaresultofpost-electionchangesinGhanashouldbemonitoredcloselyoverthecomingc.3months,andthequestionofanextensionberevisitedshouldtheknockoneffectsprovesignificant.
10.4 ThiswouldbringthedateforthefinalclosureofprizestoJune2019(ratherthanDecember2018),afterwhichtheevaluationteamwillneedninemonthstoconductplannedevaluationand research. Given that three months are already provided for in existing programmetimelines,thiswouldbringtheprogammeenddatetoMarch2020,sixmonthsafterthecurrentcontractenddate34–i.e.atotal(no-cost)extensionofninemonths.
10.5 Managementcostsassociatedwithprizeextensionsshouldbekeptascloseaspossibletothe‘low’bracketofinvestment.
Targetingprogrammeinvestments
10.6 Itisrecommendedthat,atprizelevel,theprogrammeinvestsavingsfromreducedprizepursein thoseareaswhichmitigatemajor risks toStage2delivery i.e. theeffectivedevelopment,deploymentandtestingofsolutionsandpositiveoutcomesforbeneficiaries.Thiswillhelptoensurethattheprizesachievesustainable impactandmaximisevalueformoney.Prize levelfindingssetoutspecificsforeachprize,butsolvercapacitybuildingisaconsistenttheme–notleastsupporttoengagewithpotentialfunderstohelpleveradditionalinvestment.
10.7 Attheprogrammelevel,I2Ishouldprioritiseinvestmentin:
a) Knowledgesharingandinfluencing–includingthegenerationanddisseminationofbothprize(e.g.addressingspecificproblemsthroughprizes,workingwithdifferent
34TheprogrammeheadcontractextendssixmonthsbeyondthecurrentenddatefortheI2Iprogramme,totakeintoaccountdeverytimelinesfortheFrontierTechaddon,whichfallsoutsideofthescopeofthisMTR.
Page57of68
typesofsolveretc.)andprogramme(operatinginnovationprizesfordevelopment)levellearning(VH).
b) Scoping for scale up / replication of specific prizes where concrete opportunitiespresent themselves, with the Urban Sanitation Challenge the strongest contender(VH).
10.8 TheMTRsupportsexistingevaluationplanstomeasurespecificprizeeffectsandadditionalityaspossiblewithinexistingresourceenvelopes.Theprogrammecouldconsiderincreasestotheevaluationbudget,forthoseprizeswherethereisstillscopeforflexible,responsiveevaluationsupport to feed intoStage2design,namelyDreampipe II and theGlobal LEAP refrigerationsisterprize.Otherwise,relevantopportunitiesforsuchadaptivelearningarelimited.Acrosstheboard, itmaybenecessarytomakemodest increasestoevaluationbudgetstoevaluatetheeffectivenessof,previouslyunplanned,capacitybuildingactivities(L).
10.9 Whereverpossible,I2Ishouldseektomaximiseimpact–withoutplacingsignificantadditionalburdenontheprogrammebudget–bybuildingpartnershipswith,andleveragingtheresourcesandnetworksof,existingandwell-established institutionsandprogrammesoperating inthesamesectors.
Opportunitiesforscalingandreplication
10.10 TheexperienceofdesigningandestablishinginnovationprizesfordevelopmentthroughI2Ihasbeenasteeplearningcurve–butoffersvaluablelessonsandlikelyeconomiesofscalewhenscaling/replicating.
10.11 Thoseprizeswherebarrierstoscalingsustainablyarehighest–includingwheresolvercapacityismostlimited–maynotbethemostobviouscandidatesforreplication.
10.12 Criteria for identifying the most suitable candidates for scaling or replication include: theinnovativenessofsolutionsdeveloped;effectivenessofcoalitionsorpartnershipsbuiltaroundthe prize problem; successful transition to Stage 2, including effective implementation ofsolutionsandsubmissionofprizeworthyapplications;anddemonstrationofpositiveoutcomesforbeneficiaries,whicharelikelytobesustainable.
10.13 Atpresent,thestrongestcandidatesappeartobe:theUrbanSanitationChallenge,owingtothelevel of traction gained with government and early indications that effective public-privatepartnershipsarebeingfacilitated;andtheClimateInformationPrize,whichisworkingwithamotivated and relatively well capacitated solver group, and hence faces lower barriers tosustainability.Itshouldbenoted,however,thatitistooearlytoassesswhetheracorrelationexists betweena group’s capacity and the innovativenessof solutionsdeveloped (although,basedonconsultations,theMTRteamsuspectsthatthismaywellbethecase).
10.14 Inpractice,specificopportunitiesforreplicationhavesofaremergedinadhocwayatlevelofindividualprizes.I2Ishouldbepragmaticandpursueconcreteopportunitieswheretheyarise,providing that the prize in question remains on a trajectory towards delivering sustainableimpact.
Page58of68
10.15 However,thisdoesnotmeanthatotherprizesshouldbede-prioritised.I2Iisfundamentallyaresearchprogramme, and soopportunities to capture learning about, for example,workingwithdifferentsolversandindifferentsectorsshouldbemaximised.
Page59of68
AnnexA: StatusofI2Iprizes
ClimateInformationPrize,Kenya
Milestone Progress
AwardofStage1(Wazo)
Achieved:3winnersand10runnerupswereawardedon6thApril2016.
LaunchofStage2(Tekeleza)
Achieved:Successfullylaunchedon6thApril2016.
AwardofTambua(RecognitionPrizes)
Achieved:Totalof12applicationsreceived,fromwhich7wereshortlistedforjudging.Aprizepurseof£4,550wasawarded.
CloseofStage2(Tekeleza)
Ontrack:3submissionshavebeenreceivedfortheTekelezaPrize,closing28thFebruary.TheprizeclosestonewapplicantsinDecember2017,andwillcloseinMarch2018.
AwardofStage2(Tekeleza)
TobereviewedatcloseofStage2
AdaptationatScale,Nepal
Milestone Progress
LaunchofStage1(ProtsahanPuruskar)
Achieved:theprizewassuccessfullylaunchedon2ndMay2016).
CloseofStage1 Achieved:Theprizewasclosedon5thOctober2016(rescheduledfrom30thSeptember).
AwardofStage1 Achieved:Atotalof15winnerswereeachawarded£10,000on16thDecember2016.Atotalof59applicationswerereceived.
LaunchofStage2(KarynwayanPuraskar)
Ontrack:TheStage2prizeisduetobelaunchedbyMarch2017.All15winnersfromStage1arebeingcommunicatedwithtoinvitethemtoapplyforStage2.
AwardofRecognitionPrize(PahichanPuraskar)
TheRecognitionPrizeisduetobeawardedbytheendofthethirdquarterof2017.
CloseofStage2 Stage2isduetocloseinAugust2018.
AwardofStage2 TobereviewedatcloseofStage2
UrbanSanitationChallenge,Ghana
Milestone Progress
CloseofStage1(DuapaAward)
Achieved:Stage1closedonApril30th2016.Atotalof91/139solversregisteredtheirinterest,resultinginatotalof48MMDAssubmittingtheirliquidwastemanagementstrategies.
Page60of68
Milestone Progress
AwardofStage1 Achieved:Atotalof21honoraryawardsweredisbursed.AllwinnersweregiventheopportunitytoenterStage2.
LaunchofStage2(DignifiedCityAward)
Achieved:Stage2waslaunchedinJune2016.17MMDAshaveaccessedStage2(15anticipated).Supporttoapplicantsisprovidedthroughongoinghelpdeskfacility
VerificationofStage2
Ongoing:Poorperformancebycontractedagenthasresultedindelaystotheassessment.Contractwithagenthasbeenre-negotiated.Theappointmentofanadditionalfirmtosupplementtheworkisplanned.
CloseofStage2 Stage2isduetocloseinJuly2018.
AwardofStage2 TobereviewedatcloseofStage2
Dreampipe,Global
Milestone Progress
CloseofStage1DreampipeI
Achieved:DreampipeIclosedinApril2016.Atotalof20applicantsfrom14countriesreceived.Prizeredesignedtonarrowprizeparametersandmitigaterisk.
AwardofStage1DreampipeI
Achieved:3winnerswereawardedatotalof£30,000onJuly31st2016.
LaunchofPhaseIDreampipeII
Achieved:DreampipeIIwaslaunchedonOctober21st2016.ThefirststageofDreampipeIIwillcloseinMarch2017.
CloseofPhase1DreampipeII
DuetocloseMarch2017
AwardofPhase1DreampipeII
TobereviewedatcloseofPhase1
LaunchofPhase2DreampipeII
DuetolaunchJune2017
CloseofPhase2DreampipeII
Duetocloseearly2018
AwardofPhase2DreampipeII
TobereviewedatthecloseofPhase2
LaunchofPhase3DreampipeII
Duetolaunchearlysummer2018
CloseofPhase3DreampipeII
Duetocloselatesummer2018
AwardofPhase3DreampipeII
TobereviewedatthecloseofPhase3
Page61of68
GlobalLEAPRefrigerationPrize
Milestone Progress
LaunchofPrize Achieved:SuccessfulpartnershipestablishedwithGlobalLeap,hasresultedinnominationsbeingopenedon21stSeptember2016.
CloseofPrize Ontrack:Thedeadlinefornominationsclosedon20thJanuary2017.55nominationshavebeenreceivedfrom28differentcompaniesbasedin14countries(ofwhich7developingcountries).45arecommerciallyavailableproductsand9latestageprototypes.FieldworkisscheduledforQuarter4,andlabtestinginQuarter5.
AwardofPrize Tobereviewedatcloseoftheprize
Page62of68
AnnexB: Documentsconsulted
ProgrammeDocumentation
AnnualReport2015AnnualReport2016I2IDesignReportProgrammeLogframeOrganogramQuarter4ReportQuarter5ReportQuarter6ReportFinancetrackerBusinesscasesFebruary2017BeneficiarynumbersCostperbeneficiaryVfMSummaryandIndicatorsProgrammeandPrizeRiskMatrix
UrbanSanitationChallenge
USCDesignDocumentStageOnePresentationsStageOneEvaluationFindingsJudgingCriteria
Dreampipe
StageOneEvaluationFindingsDreampipeDesignDocumentTheoryofChange
AdaptationatScale
TheoryofChangeandM&EFrameworkDesignDocumentJudgingCriteria
ClimateInformationPrize
DesignDocumentTambuaPrizeDesignDocumentWazoPrizeDesignDocumentStageOneEvaluationFindingsJudgingCriteria
Page63of68
EnergyAccess
StructureandComponentTitlesofGLEAPAwardsGLEAPAwardFactsSheet
AdditionalDocumentation
Bruntetal.(2012).Inducementprizesandinnovation,JournalofIndustrialEconomics,XL(4),pp.657-696.Everett, B., Wagner, E. and Barnett, C. (2012). Using innovation prizes to achieve the MileniumDevelopmentGoals.Innovations,7(1),pp.107-114.Everett,B.(2011)Evidencereview–environmentalinnovationprizesfordevelopment.London:DEWPoint.Goldhammer,J.,Mitchell,K.,Parker,A.,Anderson,B.andJoshi,S.(2014)Thecraftofincentiveprizedesign:lessonsfromthepublicsector.DeloitteUniversityPress.Hanson(1998)Patternsofpatronage:whygrantswonoverprizesinscience.California:UC,Berkeley.Hendrix,M.(2014)ThePowerofPrizes.WashingtonDC:USChamberofCommerceFoundation.McKinsey&Co(2009).“Andthewinneris…”Capturingthepromiseofphilanthropicprizes.Sydney:McKinsey&Co.NestaCentreforChallengePrizes(2014)Challengeprizes:apracticeguide.London:Nesta.NestaCentreforChallengePrizes(2013)Thegivingchallengeprizes.London:Nesta.XPRIZE(n.d.)WhatisanXPRIZE?CulverCity:XPRIZE.
Page64of68
AnnexC: Interviewees
Name Organisation RoleUK/remoteMagdalenaBanasiak
DFID SeniorResponsibleOwner–SeniorInnovationAdviser,ClimateEnvironmentandWaterTeam,ResearchandEvidenceDivision
DavidWoolnaugh DFID TeamLeader,ClimateEnvironmentandWaterTeam,ResearchandEvidenceDivision
DonaldMenzies DFID InnovativeAidInstrumentsAdviser/PBRLeadBryonyEverett IMCWorldwide TeamLeaderGabrielleMinkley IMCWorldwide ProgrammeManagerJontySlater BlueGlobeConsulting PrizeLeadMichaelLoevinsohn IndependentConsultant ResearchLeadRobbieGregorowski ItadLtd Evaluation&LearningLeadCherylBrown Independentconsultant
(contractedbyItad)EvaluationCoordinator
AlexMason IMCWorldwide PrizeManager(CIPandA@S)LarsOttoNaess IDS ThemeLead(A@S) VeronicaDiBella IMCWorldwide PrizeManager(USC)SimonCollings Energy4Impact ThemeLead(EnergyAccess)MakenaIreri Energy4Impact PrizeManager(EnergyAccess)FroeydisGording IMCWorldwide PrizeManager(Dreampipe)ChrisShugart Independentconsultant
(contractedbyIMC)PrizeDirector(Dreampipe)
Nepal(A@S)PurusottamManShrestha
IMCWorldwide ImplementingPrizeManager
JamesWalton IMCWorldwide NepalCountryManagerDilBadhwarBhattarai(andcommunitymembers)
MachhapuchhreDevelopmentOrganization(MDO)
Applicant/winner
NagdavYadav(andcommunitymembers)
CDAF Applicant/winner
SabitraGhimire HIMAWANTI-Nepal Applicant/winnerKanchanLama Independentconsultant
(contractedbyIMC)Advisorypanel
MadhavKarki CGED-Nepal Advisorypanel/judgeTopKhatri JudgeAnjalPrakash ICIMOD Programmecoordinator(roleinprizedesign)NareshSharma NepalClimateChange
SupportProgramCoordinator,NationalAdaptationPlan
AnnikaOlsson DFIDNepal EconomicAdviser,ClimateChangeandNaturalResourceManagement
Kenya(CIP)
NickiSpence Cardno(implementingagent)
ImplementingPrizeManager
Page65of68
BeatriceKinyanjui Cardno RegionalManagerStephenMutimba Camco StakeholderEngagementExpertDanielMbeyah Applicant/winnerMercyAgosa Applicant/winnerJohnKioli KenyaClimateChange
WorkingGroupJudge
VictorOrindi AdaConsortium JudgeGeorgeOtieno ICPAC JudgeGhana(USC)
VidaDuti IRC(implementingagent)
TeamLeader
KwameAsubonteng
IRC SanitationExpert
AbubakariWumbei IRC CommunicationsLeadMawuenaDotse MAPLEConsult AdviserHenriettaOseiTutu MLGRD MinistryFocalContactAnthonyMensah AccraMetropolitan
AssemblyApplicant/winner
HamiduOsman AtiwaDistrictAssembly Applicant/winnerAbu OffinsoDistrictAssembly Applicant/winnerRichard MampongMunicipal
AssemblyApplicant/winner
TheresaBaahEnnumh
PlanningDeptKNUST,Kumasi
Judge
FrankRomeoKettey WaterandSanitationforUrbanPoor
Privatesectorpartner
Dr.KwakuAddai KITAInstitute,EjisuJuabeng
Privatesectorpartner
DanielOsei AdjeiKumEnterprise Privatesectorpartner
Page66of68
AnnexD: Exampleinterviewprotocol
INTERVIEWPROTOCOL4of7:LOCALIMPLEMENTINGAGENTS
RoleinrelationtoI2I
1. Whatdoesyourroleintheprogrammeentail?
Overviewofprizeobjectivesandprogress
2. Howwouldyoudescribetheprize’smainobjectivesandisitontracktoachievethem?3. Whatevidenceistherethattheprizeisstimulatingthedevelopmentof[energy/WASH/climate
adaptation]solutionsforlow-incomeconsumers?4. Whatevidenceistherethattheprizeisresultinginsolutionsorapproachesthatareneworhaven’t
beentraditionallyappliedinthisarea?
Headlinesreprizedesignandimplementation
5. Howdidtheprizegoaboutidentifyingtheproblemtobeaddressed?6. Whatdoyouthinkismostinnovativeabouttheprize?7. Whatwouldyouidentifyastheprize’skeyachievementssofar?8. Andwhathavebeenthekeychallengesand/orlessonslearned?9. Whatopportunitiesexisttostrengthentheprize?
Applicationandawardprocess
10. Whataretheincentivesforapplicants?Arethesesufficient/appropriate?11. Howappropriatearetheprize’sjudgingcriteria?Howeffectivelyhavethesebeenapplied?12. Isthejudgingprocessefficientandeffective?Coulditbeimproved?Ifso,how?13. Aretheproposalsofsufficientdetailanddepthtojudgepotentialandenableassessmentof
viability?Iftherearesignificantnumbersofproposalsthatcannotbeadequatelyjudged,dotheyfalldownincommonareas?
Outreachandparticipation
14. Whoarethetargetgroupfortheprize?Howweretheyselected?15. Howmanyproposalsweresubmittedandbywhom?Whichwereawarded?16. Aretherepatterns(e.g.demographic/organisational)inthetypesofsolvers(applicantsand
winners)thatstandout?Howdoesthiscomparewithyouroriginalexpectations?17. Whatevidenceistherethatmarketingactivitiesarereachingsolversthatmightotherwisebe
challengingtoreach?Couldmarketingactivitiesbeimproved(e.g.reach,quantity,qualityandmeansofcommunication)?
18. Isthelevelofsupportprovidedtoapplicantsappropriate?Shouldnon-prizebasedsupportbeusedtosupportapplicants?
19. What typeof risksdo solvers faceduring theapplicationprocess?What could theprizedo tominimisethese?
20. Howwellhastheprizeanticipatedthetypesofresourcesorcapacityneededbysolversandsetareasonablerisk/rewardcurvei.e.aremilestonesappropriate?
21. Arethereanypatternsinthesolutionsbeingproposedthatareunexpected,surprisingorunwanted?
22. Whatishappeningtoproposalsthatweresubmittedbutnotawarded?
Page67of68
Prizeecosystem
23. Aretherepatternsinthesolutionsbeingproposedorcommunicationsfromsolversthatmightindicatedeficitsintheprizeecosystemi.e.theresourcesavailabletosolvers?Ifsohowcouldthesebeaddressed?
E.g.:§ workforce/capacityneededbysolution§ information,data,statisticsetc.,relatingtotheproblem§ facilities(testing,IT,officeetc.)neededbysolvers§ infrastructure(utilities,equipmentetc.)neededbysolution§ materialsavailability(ifsolutionrequiresspecificmaterials)§ collaboration/networkingwithothersolverstofindpartners§ seedfinance,potentialinvestors,marketresearch,neededbysolvers§ localisation(language,culturalreferences)neededforsolvers§ regulatory/legalassistanceneededbysolvers§ qualityofeducation/disseminationnetworkifsolutionrequiresit§ other
Programmemanagement
24. Aremanagementarrangements–governance,levelofinput,degreeofoversight/delegation,riskmanagement–effectiveandappropriate?Howcouldthesearrangementsbestrengthened?
25. Whatarethemajorriskstothesuccessoftheprize?26. Hastheprizechangedandadaptedinresponsetochangingrisksand/orlessonslearned?27. Hasthesizeandstructureoftheprizebudgetbeenappropriate?Coulditbeimproved?28. Howwouldyoudescribetherelationshipsbetweentheprizeteamandprogrammemanagement
unit/in-houseprizemanagers?Couldthesebeimproved?29. How would you describe coordination with i) other prize teams and ii) other relevant donor
initiatives?Couldthisbeimproved?
M&Eandresearch
30. Are monitoring and verification systems appropriate for applicants’ capabilities and resourceconstraints,aswellastheneedsoftheprogrammeforverification,judgingandaward?
31. Howwouldyoudescribethebalanceandrelationshipbetweenprizedesign/implementationandM&E?
32. What data is being collected for prize levelM&E?How can youmeasure the impact of prizesrelativetootherfundingmechanisms?
33. Istheprizecapturinglessonsforwiderknowledgesharing?Whathasbeendocumentedsofarandhow?
Valueformoney
34. Doestheprizeoffergoodvalueformoney?35. Howisthismeasured?Aretheretypesand/orinstancesofVFMnotcurrentlybeingcapturedby
thesemetrics?36. Howcouldtheprize’sVFMbeimproved?
Impactandsustainability
37. Whatevidenceistherethatprizeoutcomesareimpactfuland/orsustainable?Includinganyevidenceofadditionalityi.e.achievementsthatwouldnothavehappenedinprize’sabsence.
38. Whatevidenceisthereofsocialinclusioni.e.howsolutionsimproveaccessforpoorandvulnerablegroups?
Page68of68
39. Whatelsecouldtheprizedotoenhanceimpactand/orensuresustainability?Shouldtheprogrammeshouldbroadensupportiveactivitiestofacilitatethesegoals?
40. What opportunities are there for prize winners to secure inward investment? How could thelikelihoodofthisbeimproved?
Programmedurationandtargetingfutureinvestment
41. Isthetimeframeforprizeimplementationandevaluationsatisfactory?Ifnot,whatwouldbethespecificbenefitsofextendingthetimeline?
42. Istheresufficientevidencetoinformadecisiontoscaleuporreplicatetheprize?Ifso,mightscaleupand/orreplicationbeasuitableoptionfortheprize?Ifnot,atwhatpointofevidencecollectionshouldscaleuporreplicationbeconsidered?
43. IstherescopetoleverageinwardinvestmentfromsourcesotherthanDFIDtosupportreplicationand/orscaleup?Ifso,whatsourcesoffundingwouldbemostappropriate?
44. Inyourview,shouldfutureinvestmentsintheprizeprioritise:§ Prizemanagementandadministration
Ø BetterprocessesØ LearningandknowledgesharingØ Supporttoimplementingagents
§ Replicationand/orscaleup§ Capacitybuildingofapplicants§ Donorandinvestorengagementtohelpleveradditionalinvestment§ Other