DevelopmentalEvaluationandSocialInnovationMichaelQuinnPatton

SymposiumonDec10,2017inTokyoEvaluationthatSupportsSocialInnovation– NewHorizonbroughtbyDevelopmentalEvaluation

Organizedby GraduateSchoolofGovernanceStudies,MeijiUniversityCo-organizedbyCSONetworkJapanInstituteofProgramEvaluation,MeijiUniversitywiththesupportofJapanEvaluationSociety

2

Evidence-based Practice

Evaluation grew up in the “projects” testing models under a theory of change that pilot testing would lead to proven models that could be disseminated and taken to scale:

The search for best practicesand evidenced-based practices

3

OriginalPrimaryOptions

Formativeand

SummativeEvaluation

(Mid-termandEnd-of-ProjectReviews)

4

EvaluationTraditionalEvaluating…• Projects&Programs• LogicModels• TheoriesofChange• SMARTGoals• Implementation• Outcomes&ImpactsGenerating...• Judgments,Lessons• Recommendations

InnovativeNewDirections:Evaluating…

• Missionfulfillment• Strategy• Advocacycampaigns• Policychange• SystemsChange• Complexdynamicinterventions

5

Innovations&Challenges:Evaluating…

• Communityimpacts• Regionalinitiatives• Environmentalecosystemsustainability• Networksandcollaborations• Leadership• Inclusivenessanddiversity• Innovation• Collectiveimpact• Scaling

6

Principles

INNOVATION

8

DevelopmentalEvaluationDefined

Purpose:Developmentalevaluation(DE)informsandsupportsinnovativeandadaptivedevelopment incomplex

dynamic environments.

DEbringstoinnovationandadaptationtheprocessesofaskingevaluative questions,applyingevaluationlogic,andgatheringandreportingevaluativedatatosupportproject,program,product, and/ororganizational development withtimelyfeedback.

KeyDECharacteristics• Focusondevelopment (versusimprovement,accountabilityorsummativejudgment)

• Takesplaceincomplexdynamicenvironments• Feedbackisrapid(asrealtimeaspossible).• Theevaluatorworkscollaboratively withsocialinnovatorstoconceptualize,designandtestnewapproachesinalong-term,on-goingprocessof adaptation,intentionalchange,and development.

9

KeyDECharacteristics• TheDEevaluatorcanbepartoftheinterventionteam.

• Theevaluator'sprimaryfunctionsaretoelucidatetheinnovationandadaptationprocesses,tracktheirimplicationsandresults,andfacilitateongoing,real-time,data-baseddecision-makinginthedevelopmentalprocess.

• DEbecomespartoftheintervention.

10

Complexityconcepts&Evaluation

• Emergence:Self-organizing,Attractors•Nonlinear:Smallactionscanhavelargereactions. “TheButterflyWingsMetaphor

•Uncertainty;unpredictable;uncontrollable;unanticipatedconsequences

•Coevolution:Processuses;interdependence•Adaptation:Changingcontext

11

12

Purposes

Purpose Challenge Implications

1 Ongoing development

Implemented in complex & dynamic environment

No intention of becoming fixed;identifies effective principles

2 Adapting effective principles to new contexts

Innovative initiatives:Develop ‘their own’ version

Knowledge disseminated;sensitivity to context, capabilities & priorities

FundamentalIssue:HowtheWorldIsChanged

Top-downscalingof“provenmodels”withFidelityEvaluation

versusBottoms-upadaptivemanagement

andDevelopmentalEvaluation

13

Models vs. Principles

Identifying proven principles for adaptive management

(bottoms-up approach)versus

Identifying and disseminating proven models

(top down approach)14

15

16

5 Purposes

Purpose Challenge Implications

3 Pre-formative development of model

Dynamic situations require innovative solutions; Model does not exist

New models move into evaluation & others remain developmental

4 Major cross-scale developmental evaluation

Disrupt existing system; scale & systems change add levels of complexity

Adaptive cross-scale innovations assume complex dynamics: agility, responsiveness

17

5 Purposes

Purpose Challenge Implications

5. Developing a rapid response in turbulent environment:

Existing responses no longer effective as conditions change:

Planning, execution & evaluation occur simultaneously.

Rapid feedback, adaptation, & agile responses.

natural disasters,civil unrest,political turmoil,famine.../

turbulence,instability,dynamic,high uncertainty…/

DEPrinciples

1. Developmentalpurpose2. Evaluationrigor3. Utilizationfocus4. Innovationniche5.Complexityperspective6.Systemsthinking7.Co-creation8.Timelyfeedback

18

19

Globalproblemstranscendnationalandagencyboundaries

• Climatechange• Economicturbulence• Refugees• Virulentinfectiousdiseases• Dyingoceans• Globalcyber-terrorism• Internationaldrugcartels• Humantrafficking• Weaponstrafficking• Povertyandinequality• Multi-nationalcorporatecollusion

Innovationto AddressGlobalProblems

ØThedefinitionsoftheproblemsaredisputedØThe“facts”areamatterofintensedebateØPoliticsandspecialinterestsdominate:

-- nationalinterests-- multi-nationalcorporateinterests-- agencyagendas-- competitionforresources

ØThestakesarehugeØInnovationiscrucialØDEisappropriate

23

TRANSFORMATION

GlobalSystemsAnalysisSkills:InterconnectedSDGs

Two-EyedSeeing

TheGuidingPrinciplebroughtintotheIntegrativeScienceco-learningjourneybyMi'kmaw ElderAlbertMarshall.

Etuaptmumk istheMi'kmaw wordforTwo-EyedSeeing.

GlobalKnowledge

Local&indigenousknowledge

DE Interdependence

28