WICKEDPROBLEMS:PEACEBUILDINGEVALUATIONETHICSDeterminingWhatisGoodandRight

DevelopedbyReinaC.Neufeldt,PhD

forthePeacebuildingEvaluationConsortium

August2016

Thisreportwasmadepossibleby

theCarnegieCorporationofNewYork

AboutthePeacebuildingEvaluationConsortiumThePeacebuildingEvaluationConsortium(PEC)isaprojectofAllianceforPeacebuilding(AfP)inpartnershipwithCDACollaborativeLearningProjects,MercyCorpsandSearchforCommonGround(SFCG).TheprojectisfundedbytheCarnegieCorporationofNewYork(CCNY)andisfield-wideefforttoaddresstheuniquechallengestomeasuringandlearningfrompeacebuildingprograms.ThePECconvenesdonors,scholars,policymakers,localandinternationalpractitioners,andevaluationexpertsinanunprecedentedopendialogue,exchange,andjointlearning.Itseekstoaddresstherootcausesofweakevaluationpracticesanddisincentivesforbetterlearningbyfosteringfield-widechange through threestrategicand reinforcing initiatives:1)DevelopingMethodologicalRigor;2) Improving theCultureofEvaluationandSharedLearning;and3)FosteringtheUseofEvidencetoInformPeacebuildingPolicy.

AbouttheAuthorDr.ReinaNeufeldtisanAssistantProfessorinthePeaceandConflictStudiesProgramatConradGrebelUniversityCollege, University of Waterloo. She has engaged in program design, monitoring, evaluation and learning inpeacebuildingandconflictresolutionforoverfifteenyears.Herpublicationsinmonitoringandevaluationinclude‘“Frameworkers”and“Circlers”–ExploringAssumptionsinImpactAssessment’(AdvancingConflictTransformation:TheBerghofHandbookII,2011),“Interfaithdialogue:assessingtheoriesofchange”(PeaceandChange,36,2011),aswellastheco-authoredReflectivePeacebuilding:APlanning,MonitoringandLearningToolkit(CRSandtheKrocInstitute, 2007) and “Building blocks for peacebuilding impact evaluation” (Journal of Peacebuilding andDevelopment,2,2005). HerrecentbookEthicsforPeacebuilders:APracticalGuide (Rowman&Littlefield,2016)providesamorecarefulexplorationofethicsandpeacebuildingforpractitioners.

ReinahasanMAinSocialPsychology(YorkUniversity)andaPhDinInternationalRelations(AmericanUniversity);she is trained in both qualitative and quantitative researchmethods. Reina’s current research focuses on fieldlearningandexplorestheroleofreflectivepractice in improvedpeacebuilding.She isalsoamemberofthePECAdvisoryBoard.

Author’s Note: Thank you to Diana Chigas and Colleen Duggan for insightful comments, careful reading andthoughtful engagement in our discussions of ethics in peacebuilding evaluation; thanks also to the larger PECadvisoryboardforactivelywrestlingwithquestionsofwhatconstitutesgoodinpeacebuildinginanongoingway.Finally,thankstoBrianAdiengeandSarahMcLaughlinattheAllianceforPeacebuildingfortheireditingworkandtoJazmineWalovitchatCDACollaborativeLearningProjectsforthefinallayoutofthedocument.

TableofContents1.Introduction.............................................................................................................................................12.Values,EthicsandEvaluation...................................................................................................................2

3.TheWhat:Determining“Good”and“Right”Peacebuilding....................................................................2

3.1GoodEnds:Consequences.................................................................................................................3

3.2RightMeans.......................................................................................................................................4

3.3VirtuesandGoodCharacter...............................................................................................................5

3.4Relationality.......................................................................................................................................6

3.5SkillstoWorkOn:IdentifyingMoralValuesinEvaluation.................................................................7

4.TheWho:WhoDecidesWhatisGoodandRight?...................................................................................7

5.TheHow:HowDoWeEnsurePeacebuildingEvaluationsAreDoneEthically?.......................................9

5.1HowFindingsAreGenerated.............................................................................................................9

5.2HowanEvaluatorConductsHer-orHimself?..................................................................................10

5.3HowSensitivetheEvaluationProcessistotheConflictEnvironment.............................................10

6.Conclusion:CreatingSpaceforMoralValues........................................................................................11

7.AdditionalResources..............................................................................................................................12

8.APPENDIX:EthicsQuestionsduringanEvaluationCycle.......................................................................15

8.1.QuestionsontheWhat(What’sGoodandRight)..........................................................................15

8.2QuestionsRegardingtheWho(WhoDecides).................................................................................16

8.3QuestionsofHowtoConductanEvaluationEthically.....................................................................17

1

1.Introduction

Anexchangelikethisiscommoninorganizationsandcanbefrustrating.Theprojectmanagerfeelslikeheorsheisbeingaskedfordatathatwillnothelpimproveimplementationinthecommunity,althoughitmightbeimportantformanagementandthereforenecessary.Theadministratoristryingtogatherdatatohelpjustifyorganizationaldecisions,andlikelyfeelingpressurefromboardmembersandfunderstodemonstrateimpact.

Thetensioninthedialogueinvolvesnotonlycompetinginstitutionaldemandsbutalsocompetingmoralclaims–that is, there are competing claims about what is good and right in peacebuilding as well as peacebuildingevaluation.Themoralclaims,however,arecloakedinlanguagethatmakesthemappearasiftheyarevalue-freedecisions,whichmakesthemhardtodetect.Acommonvalue-freeassertiongoeslikethis:ifwecollectthedatathenwewillknowwhattodo.Thisideaofbeingabletocollectdatathatwillsimplytelluswhattodoispopular,and is rooted ina schoolof thought thatespousesvalue-free social scienceas theway toanswermanagementquestions.

However,whenweaskwhatconstitutesgoodresultsoraskquestionsaboutwhatwasdeterminedasimportantinordertogatherthedata,wecatchaglimpseofthemoralvaluesuponwhichclaimsarebeingmade.Forexample,inthedialogueabove,weseemoralclaimsthatevaluationsaregoodandworthwhileif:

§ theygeneratedataonimpactforscaling-up,§ thepeopleinvolvedintheinitiativelearnfrom,andmakeadaptationsbasedontheevaluationdataand

process,§ theyhelptheprojectofficermakefield-leveldecisions,§ theyhelpadministratorsmakedecisionsaboutallocatingfuturefunding,or§ theyjustifytheprogramtoexternalfunders.

Thesevariousclaimsoftenexistside-by-sideandcancomplement,orbeintensionwitheachotherwhenchoosingevaluationapproachesandmethodologies.

Alsonestledwithinthisexchangearemoralclaimsaboutwhatwillconstitutegoodendswithinthepeacebuildinginitiative. That is,whattheprogramshouldbeevaluatedupon inordertodetermine itsworthormerit. Theseclaimsaresignaledbywordslikeimpact,effective,scaling-up,changethingsforthebetter,responsive,andbythecharacteristicsofbeingcourageous,dedicatedorpassionate.

“We want to know that our indicators are collecting data that will determine our impact,” the senioradministratorsays.“Wereallywanttoknowifourpeacebuildingmodelwaseffectivesowecanscale-up.”

“Right,”saystheprojectofficer.“And,ourinitiativeisreallyaboutmakingadifferenceinpeople’slivesinthe

community,sowedon’twanttojustshowimpact,wewanttohaveimpactandunderstandhowthathappened

sowecontinuetochangethingsforthebetter.Theyouthandwomen’sgroupleadersweareworkingwithare

terrific – they are really courageous, passionate and dedicated to building peace in their communities. The

situationisvolatilebutwearerespondingasbestwecan.”

“Great! Wewillalsoneedtousethat impact informationtoallocateourtimeandfundinginthisprogram

areainthefuture,anddevelopanewproposalforwhentheprojectwrapsup,”respondsthesenioradministrator.

“Okay,”repliestheprojectofficer,worryingaboutwhetherornotthereisenoughoccurringintheparticipatoryassessment thatwill constitute “impact” in theadministrator’s eyes –or inanexternal funder’s eyes –andwhetherornottheircurrentideasabouthowtoutilizeevaluationdatawithintheprojectwillbeunderminedbythesenioradministrator’sdemands.

2

Thepurposeofthisbriefingpaperistosupportevaluatorsinproducinggoodevaluationsbyhelpingtothinkaboutvalues and ethics consciously and carefully. To do this, we’ll first look at the relationship between ethics andevaluation.Wewillthenlookatthebigpictureofwhatconstitutes“goodpeacebuilding”orpeacebuilding“doneright.”Wewillalsoexploreethical issues involved inwhodetermines thevaluesandcriteriabywhichwe judgeprograms.And,finallywewillexploreethicalissuesinvolvedinhowevaluationsareconducted–anareaforwhichtheretendstobemoredevelopedguidanceavailable.Thepaperconcludeswithadditionalresourcematerialforfurtherexploration.

2.Values,EthicsandEvaluation

Evaluationinvolvesmakingvaluejudgementsaboutwhatisgoodandright.Wecollectempiricaldata,analyzeitandtheninterpretitinordertomakejudgementsabout“theworth,meritorvalueofsomething.”1AsDeborahFournier notes in her definition of evaluation for the Encyclopedia of Evaluation, “It is the value feature thatdistinguishesevaluationfromothertypesofinquiry,suchasbasicscienceresearch,clinicalepidemiology…”andsoforth.2Thevaluefeatureofevaluationmakesit,inpart,ethicaldeliberation–weighingandchoosingamongstwhatwillbeconsideredgoodandright,orworthyandofvalue,inagivenpeacebuildinginitiative.

Whenwethinkaboutevaluationtraining,however,ourskilldevelopmenttendstofocusontechniques,methodsandthebusinessaspectsofevaluation.3 Forexample,writingornegotiatingthetermsofreference,developingfocusedlinesofinquiry,gatheringdataandanalyzingitusingparticularmethodologies.Theseareimportantand,aswill be noted below, also part of an ethical evaluator’s practice. However, the skills for thinking about andweighingtheethicaldimensionsofevaluation–suchashowtoidentifyandweighthevalue-basedelementsofanevaluation–arefrequentlymissingevenwhenrecognizedasimportant.4

Our lack of attention to ethics often feeds into problems in evaluations. Peacebuilding initiatives occur inenvironmentswithconflictedparties,divergentinterestsandneeds,whichpeacebuildinginitiativesareattemptingtochange.Evaluationsinvolveknowledgegenerationandaffectresourcedistributionamongststakeholdersandsostakeholdersareveryconcernedabouthow“worth”isdeterminedandhowitwillreflectonthem.5Evaluationsarecommissionedbypartieswhowanttofundworkthatmakesadifferencefromtheirperspective(e.g.greatestimpact).Thosebeingevaluatedhaveastakeintheevaluationandtheconflictcontext–implementerswanttodogoodworkandgetfundstocontinue,whilecommunitymemberswantqualityprogramsthatsupporttheirvisionfor their community. When there aredivergent values, they can contribute to conflict and thepoliticizationofevaluations–particularlyifthosevaluedifferencescoincidewithconflictcleavages.We,asevaluators,maythenunintentionallycontributetoconflictandunderminethepeacebuilding initiativeweareevaluating ifwearenotattendingtomoralvaluesaspartofconflictandevaluation.Butwhataredifferentwaysofunderstandingwhatisgoodandright?

3.TheWhat:Determining“Good”and“Right”Peacebuilding 1Mathison,"ValueJudgment,”444.2ibid.,140-1.Thosewhoarguethatempiricaldatacanbegatheredinordertoinformrationaldecisionsinavalue-freewayarerootedincontemporarysocialsciencethinkingfromMaxWeberthroughtoAmericanPsychologistDonaldT.Campbellandtotoday’s“evidence-based”managementsystems;foranoverviewseeQuinnPatton,UtilizationFocusedEvaluation.3Ethics,whentaught,tendstoberelativelyshortoranadd-on.Forexample,theAmericanEvaluationAssociationhasdevelopeda1.5-2hourtrainingonitsethicsguidelines,whichisavailableathttp://www.eval.org/p/cm/ld/fid=105.Thereareexceptions,suchasBushandDuggan,“AddressingEthicalandPoliticalChallengesinEvaluation.”4Forexample,theAfricanEvaluationAssociation’s“AfricanEvaluationGuidelines-StandardsandNorms”documentnotestheimportanceofvaluesidentificationinasub-pointunderthediscussionoftheprincipleofutility.SeeAfricanEvaluationAssociation,“AfricanEvaluationGuidelines-StandardsandNorms,”5.5MichaelQuinnPattonhasacleardiscussionofthepoliticsofevaluationandhowtomanagethepoliticsofevaluationinChapter14ofUtilizationFocusedEvaluation.

3

Wetendtoassumeweorthosedesigningthetermsofreferenceknowautomaticallywhatisgoodandright,andthatwe—evaluators,evaluands,andotherstakeholders—allagreeuponit.However,thisisoftennotthecase.Isuspectoneofthereasonsthatpeopledon’tlikeevaluationsisbecausetheyfeeltheevaluationsarejudgingthemby criteria they don’t think are appropriate and are not “good” or “right”— in other words, they feel like theevaluatordoesnotunderstandorknowtheirrespectiveneedsandvalues.Howcanwedobetter?

Evaluation involvesaskingbigquestions:“Wasthis initiativetherightthingtodo?Didgoodpeacebuildingoccurhere?Whatchanged?Whatistheevidencethatgoodpeacebuildingoccurred?”Thefirsttwoquestionsinthissetdemandethicalthinking.

Whenwe think ethically, wemake judgements aboutmoral claims; it is the process of reasoning through andassessingwhatmakesforgoodendsandrightmeansgiventhelegitimateneedsandexpectationsofourselvesandothers.6Thisischallengingwork.

Thereis,however,alongtraditionofmoralphilosophythatpresentsmoraltheoriesthatcanhelpevaluatorsexplorearangeofvaluesthatcanbeusedtodeterminewhatisgoodandrightinpeacebuilding,andfromwhichcriteriaandstandardstojudgeempiricaldatacanbedrawn.Inthesectionbelow,we’llbrieflylookatbigideasfromfourmoraltheory families to help evaluators identify different moral values that might be prioritized by stakeholders inevaluations:goodends,rightmeans,embodyingexcellenceaspeacebuilders(andevaluators)andenactingcaringrelationships(aspeacebuildersandevaluators).Thislistisnotacompletelistofallpossiblevaluesofgood,butastartingpointforimportantconsiderations.

3.1GoodEnds:ConsequencesPeacebuildinginitiativesusuallypursuepeaceasagoodend–thismayincludestoppingovertviolence,addressingstructuralviolence,andbuildingpeacewithjustice.Theseare“bigends”orgoalsthatreflectthemoralvaluesthatguidewhywedopeacebuildingwork.

Feeding into the big ends are more specific moral values about ends and desired consequences that framepeacebuildingevaluations.Thesemoralvaluesappearinthewaythetermsofreferencepresentthepurposeoftheevaluationaswellasitsobjectivesandthelinesofinquirythatidentifyhowaninitiativeshouldbejudged.Termsof referenceusuallyemphasize thecriteriaandstandards thatalignwith thevaluesof thegroup funding theevaluation.

Forexample,termsofreferenceforpeacebuildingevaluationsoftendrawontheOECD-DACcriteria,whichdefinesgoodpeacebuildingintermsofits:

1. relevancetodrivingfactorsofconflict(asdeterminedbyaconflictanalysis),2. effectiveness vis-à-vis intended objectives and the degree to which results were achieved (outputs,

outcomesandtheconnectiontoimpacts),3. impact intermsof intendedandunintendedpositiveornegativeeffectsonthewiderenvironmentand

conflictcontext,4. sustainability or the “continuation of benefits” when funding is discontinued (can include things like

communityownershipandresilienceinsettingsofconflict),and5. efficiencyintermsofhoweconomicallytheresourcesthatareputintoaninitiativeproduceresults.7

TwoadditionalOECD-DACcriteriathatarementioned,butnotonequalfootingastheprecedingfive,arecoherence,andcoordinationamongstintervenorsorfunderstoproducemoreeffectiveresults.

6Weston,A21stCenturyEthicalToolbox,86.ForacarefulexplorationofvaluesinevaluationseeSchwandt,"TheLandscapeofValuesinEvaluation:ChartedTerrainandUnexploredTerritory."7OECD,"EvaluatingPeacebuildingActivitiesinSettingsofConflictandFragility:ImprovingLearningforResults,”65-71.

4

Thereisastrong,unifyingmoralclaimbehindthesecriteria:peacebuildingprogramsorprojectsaregoodiftheyachievethegreatestgoodforthemostnumberofpeopleorproduceatleastasgreatagoodasanyothercourseofactionwouldproduce. Ourbestaction is therefore theone thatachieves thebestend. Thismoral claim isparticularlyimportantforthecriteriaofimpactandtheeffectsonthewiderenvironmentandcontext,butalsokeytothedefinitionsofrelevance,effectiveness,efficiencyandsustainability.

Intheexchangebetweenthemanagerandtheadministrator(above),theadministratorisemphasizingthattherightthingto do is the one that produces the greatest impact – thegreatestgoodforthemostnumber,soitcanbe“scaled-up”and produce even greater good for more people. This isends-based consequentialist moral thinking, and is intensionwith the program staffer’s thinking about processandpeople.Theefforttoscale-upcanproducethe“greatestgood”bybenefittingmorepeople,yetalsocanunderminethequalityandnatureofthechangeprocessforthoseintheinitialproject(andstillproducethe“greatestgood”becausemore people are involved who receive some benefit). Inconsequentialistthinkingsomeharmsalongtheway–likeover-stretchedstaffor fieldpersonnelandpartners–areconsideredacceptableaslongastheoutcomeisatleastasgoodasanyothercourseofaction.

The emphasis on ends is strongly represented inpeacebuilding evaluation criteria such as the OECD-DACcriteria.Whenweaskquestionsaboutendsthenweweighthe harms and the benefits and look to determine if thebenefits togetheroutweigh theharmsvis-à-visourcriteria(e.g. was our impact overall positive? Was our initiative,overall,relevant?).

There are, however, other ways to think about whatconstitutes good peacebuilding (e.g. principles, character,relationships),whicharesometimesintensionwithends-basedthinking.And,asnotedabove,thesedifferencescanalignwithstakeholdergroupsandcontributetomisunderstandingandconflictwithinanevaluation.

3.2RightMeansWhileends-basedthinkingmaydominateevaluationdiscourse,itisneverthelesscommonforpeacebuilderstothinkaboutthewayinwhichwedoourworkasimportant,particularlyimportantintermsofmodellingwhatwehopetoachieve.Pursuing“peacebypeacefulmeans”isthereforeacommonphraseinthefield–aswellasthetitleofabook.8

Ourmeansandtheprinciples thatguideouractionscanbeseenasgood inandof themselves. Forexample,ImmanuelKantarguedweshouldalwaystreatpeopleasendsandneverasameanstoanend.Anyactionfollowingthisprinciplewasgoodinandof itselfregardlessofwhathappenedasaresult.Peoplemightrecognizethisasaformulationof“thegoldenrule”ofreciprocitythatappearsinmanyreligioustraditions.9Respecteveryone’shumanrightsisanoutgrowthofthisimperativeofKant’s.Othermeansthatareemphasizedinpeacebuildingevaluation 8Galtung,PeacebyPeacefulMeans:PeaceandConflict,DevelopmentandCivilization.9Kant,GroundworkfortheMetaphysicsofMorals.

EXAMPLE

Evaluations of “Quick Impact” Projects (QIPs)reflectconsequentialistreasoning.QIPsareoften

promoted in post-accord environments because

theyprovidepositive“peacedividend”benefitsto

many, such as rebuilt schools, which may help

stabilize a community as a base for other

interventions in a short period of time. These

projects are therefore typically evaluated based

onthespeedandscaleofthebenefits (e.g.how

manyvillagesreachedinashortperiodoftime).

Given this reasoning, they have been seen as

producing a better overall consequence (more

villages reached) than slow, intensive

participatory processes that might rebuild a

schoolinacommunityoveratwo-yearperiodand

generategreater communityownership,butdue

to its intensive nature does not reach as many

people as quickly. Note, however, if the time-

horizonofconsequencechanges,theassessment

of benefits and harms shifts and so QIP

evaluations may initially be positive but then

become negative if short-term benefits did not

producelong-termbenefits.

5

guidanceistheprinciplethatactionsmust“donoharm”ormustpromotenon-discriminationasabasisforinclusivesocieties.10

Ifwe–evaluatorsandthosebeingevaluated–decidetofocusonthe“rightmeans”thataredefinedbyourprinciplesandmost important indeterminingwhetherapeacebuilding initiativewasgood,weexamine theways that thepeacebuildinginitiativewasimplemented.Didtheprocess“donoharm”?Didittreatpeoplewellthroughout–alwaysasendsandneverasmeanstoadesiredend?Diditmodelhumanrightsprinciples?Wastheinitiative

participatoryandinclusive?Wasitgender-sensitive?Didtheapproachembodynonviolentprinciples?Thisisquiteadifferentsetofquestionsthandiditproducethegreatestgoodforthemostnumberofpeoplebecauseourevaluationquestions includeassessmentsofproblematicactions thatmighthaveaffectedonlyaminorityofpeople,orgroupsofpeoplewhohavebeenseenas less importantoverall (e.g.gender-based violence has often been overlooked inpeacebuildingprojectsthatfocusondemobilizingmilitants).When we shift to thinking about the right means andprinciples,thenwealterourevaluationfocustoincludetheprocessbywhichaninitiativepursueditsends.11

Notethatattimes,someprinciplesmaybeatoddsor inconflict.Forexample,valuingindividualsandindividualrightsovercommunitiesandcommunityrightscanbeproblematic.Inthesesituations,theevaluatormaybemakingjudgementsbasedondisputedvalues,whichisdiscussedfurtherinsectionfour(below).

Therearetwomoreperspectivesonhowtojudgewhatconstitutesgoodthatarelesscommonthanthefirsttwo.Lesscommon,however,doesnotmeanthattheydonothavemerit,particularlywhenthinkingaboutpeacebuilding.Oneistothinkaboutthepurposeofpeacebuildingasdefinedbythepursuitofpersonalexcellence.Asecondistothinkaboutthepurposeofpeacebuildingasbasedonrelationships–thatis,thinkingaboutapersonbeingapersononlythroughandwithotherpeoplewithinarelationalnetwork,asoccursinUbuntuorcareethicsthinking.12

3.3VirtuesandGoodCharacterIfwejudgepeacebuildingbyitspromotionofpersonalexcellence,thenwereframeourevaluationfromfocusingon activities that produced communal peaceoutcomes to looking at howpeacebuilding initiatives supportedcharacterdevelopmentandexcellencewithinsettingsofconflict.Thismightbeturnedintooutcomesandoutputs,but ifdonemechanically itwilldefeat thecentral idea thatexcellence involvesone’scharacter. So, rather than

10Forexample,Anderson,DoNoHarm:SupportingCapacitiesforPeacethroughAid.SeealsoOECD,"EvaluatingPeacebuildingActivitiesinSettingsofConflictandFragility:ImprovingLearningforResults."11Therearesomeevaluationmethodsthatwereenvisionedtodrawmoreattentiontomeansandprocessinevaluation,suchasempowermentevaluationortransformativeevaluation,whichcanbehelpful.SeeforexampleFetterman,Kaftwarian,andWandersman,Empowerment

Evaluation,orMertens,TransformativeResearchandEvaluation.12ForanintroductiontoethicsofcareseeHeld,"TheEthicsofCare."ForanoverviewofUbuntuethicsseeforexampleMunyakaandMotlhabi,"UbuntuandItsSocio-MoralSignificance."

EXAMPLE

Evaluation questions that focus on particular

principles, suchaswhetherornotgenderequity

was achieved in a community-based youth arts

project, may look at levels of gender parity or

increases in participation by young women in

peacebuilding activities. Evaluations may also

look at project staffing and whether or not the

principle of gender equity was evident in the

processofimplementation.

6

emphasizetheendorthinkaboutgoodvirtuesasameanstoanend,virtuesthinkingemphasizesalifewell-livedandseescollectiveflourishingemergeaspart-and-parceloflivingliveswell.

Personal excellence is the focusof virtueethics, and ispresentwithina varietyof religious traditions, includingChristianity, BuddhismandTaoismaswell as inAristotle’swork. Theparticular virtues that are important in a givencontextandculturemayvary.Forinstance,inthedialogueat the start of this paper, theprogrammanger commentsthat thewomenandyouth involved in theprojectpossessparticular character traits that she sees as excellent: theparticipantsarepassionateanddedicated.Therearelikelyother virtues that are also important in this context thatwouldbepartofalifewell-lived.Ifweweretothinkaboutplacinganintentionalfocusoncharacterdevelopmentinourevaluations,wewouldlookforvirtuesattheimpactslevel–whichmayseemcounter-intuitivebecausewetendtothinkofpersonalchangeasanecessarystepforotherchangestooccur,andthereforealower-leveloutput.13

3.4RelationalityAnotherwaytothinkaboutjudginggoodinpeacebuildinginvolvesseeingreconfiguredrelationshipsasmoralgoodsthemselves.14Conflicttransformationapproachestendtoassumethisgood.Relationalethicsfocusesonlisteningtovoicesofthosewithwhomone’slifeisintertwined(nearandfar)andrespondingtothoseneedsinwaysthatarereceivedbythepersonbeingcaredfor(toavoidpatronizingbenevolence).ThisorientationisevidentinUbuntuethics,whichwerepartofthephilosophyunderpinningtheSouthAfricanTruthandReconciliationCommission.Inthistypeofassessment,ourhumanityisintertwinedandsowebecomegoodonlyasweactwithandrespondtoother’shumanity.

Ifwejudgepeacebuildingbyrelationality,thenwejudgeitbytheways inwhichpeoplearecaredfor inandbythepeacebuilding initiative – those working in the initiative,thosereceivingthe initiativeaswellasthose inthe largercirclesaroundtheinitiative.Weretheirneedsrespondedto?Weretheirvoicesheard?Reframingourjudgementofgood to assess relationality requires the evaluation beconceptualizedwithinarelationalcontextaswell.Inotherwords,relationshipsarenotassessedintermsoftheirsocialcapital–whichwouldmake themmeans toanend–butratherasgoodsinandofthemselves.

13ThisisalsoanemphasisoftheReflectingonPeacePracticeProject,whichprioritizes“greatestgood”thinkingintermsofsystemic-levelchange,seeAndersonandOlson,ConfrontingWar:CriticalLessonsforPeacePractitioners.14Thisapproachisbasedonanethicofcare,andalsofiguresprominentlyinUbuntuethics.

EXAMPLE

Youth sports and peacebuilding projects often

containastrongfocusoncharacterdevelopment.

Thisinvolvesdevelopingleadershipskills,strength

ofcharacter,respect,trust,empathyandsoforth.

Thistypeofprojectmodelbuildsontheideathat

characterexcellence isa necessarypartof living

well. Evaluations then correspondingly assess

charactervirtues.Evaluationsmayalsoassessthe

character virtues of those implementing the

projectsandwhetherornotgoodcharacterwas

modelledinimplementation.

EXAMPLE

Restorativejusticeprogramsfocusonbuildingor

restoringrelationshipsbetweenthoseconsidered

victims and offenders. Youth victim-offender

programs, for example, work in communities to

restore relationships between youth who have

committed crimes and those affected by the

crimes.Inthesetypesofprograms,restorationof

relationships are highly valued goods and

evaluationofthese initiativesnecessarily lookto

assessthenatureandqualityoftherelationships

built.

7

3.5SkillstoWorkOn:IdentifyingMoralValuesinEvaluationExpandingourthinkingaboutwhatisgoodandrightinpeacebuildinghelpsustorecognizethatwhenwegointoevaluationswearenecessarilymakingjudgementsaboutwhatisgoodamongarangeofdifferentconceptionsofthe“good”andthe“right”—whichincludemanymorethanthefourperspectivespresentedabove.Asevaluatorsareaskedtomakejudgementsofworthandmerit,wecanbemoreexplicitandcarefulinidentifyingwhatisbeingvalued,toenhancethelikelihoodthattheevaluationitselfisgoodandright.

Importantskillsforevaluators,then,includeidentifyingmoralvaluesembeddedindocumentsliketheTermsofReference(TOR)foranevaluation,andmanagingdiscussionofthevalueswithstakeholderstoensurethatpeopleagreewiththedifferentmoralvaluesthataregoingtobeusedtojudgethepeacebuildinginitiative.Forexample,ifanevaluatoridentifiedthevaluesintheconversationabove,heorshewouldquicklyseethattherewasatensionbetweenwhatwasvaluedasgoodbytheadministrator(impact,scaling-up)andwhatwasvaluedasgoodbytheprogram staffer (doing good work, supporting the community). The community itself is not involved in thisconversation,andlikelywouldpossessadditionalvalues.Theevaluatordoesn’tneedtopre-judgethevalues,butrathertohelpstakeholdersidentifyandagreeonwhatismostimportant,worthyandmeritoriousthroughdialogue,inordertoconductanevaluationthatwillbeuseful,goodandcontributetolearning(andperhapsevenflourishing).

Thefollowingtext-boxcontainsquestionstoasktohelpanevaluatorthinkaboutmoralvaluesandwhatconstitutesgoodandrightinpeacebuilding.ThesequestionsaremappedontoanevaluationcycleinAppendix1,whichincludesthecontractingandentrypointoftheevaluation,evaluationdesign,datacollection,analysisandreportingstages.15

TEXTBOX3.5QUESTIONSTOASKABOUTWHATCONSTITUTESGOODANDRIGHTPEACEBUILDING

Herearesomegeneralquestionstohelpevaluatorsthinkaboutthegoodandrightinpeacebuilding:

§ HowarepeoplejudgingwhatisconsideredgoodandrightintheTORorinconversations?Whatisvaluedasmorallygood(e.g.ends,means,virtues,relationships)?

§ Doallofthestakeholdersagreewiththevaluesbeingusedtojudgetheinitiative?

§ Aretheretensionsbetweenvaluesthatdifferentstakeholderspossess?Ifso,whatarethey?

4.TheWho:WhoDecidesWhatisGoodandRight?Inpeacebuildingprojectsorprogramsthatarefundedbyanexternaldonor,therearetypicallythreecommunitiesofstakeholderswhoareinterestedinthequestionofwhatconstitutesgoodandrightpeacebuilding.16First,thereisthecommunityinconflict,inwhichthepeacebuildinginitiativeoccursthatisdirectlyvested-inandaffected-bythepeacebuilding.Second,thereistheimplementingorganization,whichmightbelocally-basedorinternationally-based.Andthird,thereisthefundingorganizationorthefundingcommunity.Anevaluatoristypicallyconnecteddirectlytothesecondand/orthirdcommunitiesbyvirtueoftheircontractagreement,andonlyindirectlytothefirstcommunity(unlessitisthecommunityitselfthatiscommissioningtheevaluation).

15ThankstoColleenDugganforthissuggestion.IextendDugganandBush’sethicalquestionsintheAppendixillustrationsfordifferentconsiderationsofthegoodandrightinpeacebuildingevaluation.Formoredetailedexploration,particularlyaroundthehowquestions,seeBushandDuggan"EvaluationinConflictZones:MethodologicalandEthicalChallenges,”5-25;BushandDuggan,“AddressingEthicalandPoliticalChallengesinEvaluation”;and,DugganandBush,"TheEthicalTippingPointsofEvaluatorsinConflictZones,"485-506.16Fast,Neufeldt,andSchirch,"TowardEthicallyGroundedConflictInterventions:ReevaluatingChallengesinthe21stCentury,"185-207.

8

So,whatroleshouldeachofthesegroupshaveindeterminingthegoodandtheright?Is itfairandrightforanoutsidefundertohavethefinalsayinwhatis“goodpeacebuilding?”Isitfairandrightfortheaffectedcommunityto have the final say in what is “good peacebuilding?” Likewise, is it fair and right if it’s the implementingorganization?Theseareimportantethicalquestionsthatevaluatorsmayfeelareoutsideoftheircontrol,especiallyifcoming-inonacontractthatseemstohavelimitedflexibility. Thisdynamicofpoweramongststakeholdersindetermining what is valued, however, can reinforce power differentials between funders and receivers, and isimportanttoconsiderandaddressifwereallydowanttobeethicalinourevaluationpractice.

TheAmericanEvaluationAssociationspeakstothebroaddomainofvaluesandtheresponsibilityofevaluatorstoconsidermultiplestakeholdergroups,includingsocietyasawhole,intheirGuidingPrinciplesforEvaluatorsunderthetopicheadingof“responsibilitiesforgeneralandpublicwelfare.”17TheAfricanEvaluationAssociation(AfrEA)similarly emphasizes the importance of recognizing and involving vulnerable groups as well as communityparticipants inevaluation.18 Theseare importantconsiderations,particularly toaddress themoralcritiquethatpeacebuildingisanewerformofcolonialdomination.

Howmightevaluatorsaddressconcernsaroundpowerdifferentialsethicallywithintheirowndomain?Oneconcretewaytodothisistoset-upareferencegroupfortheevaluationthatincludesallofthekeystakeholders.Forexample,MichaelQuinnPattonadvocatesareferencegroupmodelinUtilization-FocusedEvaluationinordertoensurethatthe evaluator is not “some stakeholder’s political puppet”.19 Patton’s “Evaluation Task Forces” involve majorstakeholdersinhelpingtomakedecisionsaboutthepurposeandfocusofanevaluationaswellasthemethodsused.Henotesthatifstructuredwell,suchagroupcanhelptoincreasetransparency,involvediverseperspectivesandvalues and increase openness to the evaluation process, among other things.20 There are other forms ofconsultationthatcanalsobeutilized(usingskillsthataresimilartothoseinvolvedinmediation).Onefinalnote,referencegroupsortaskforcesneedtobewell-facilitated,tobalanceinputs,andinvolvestakeholdersiftheyaretoworkwell.

Anotheroptionforevaluatorsatthecontractingstageistofindoutifthekeystakeholdersareopentohavingtheirprogramtheoryofchangeandthevaluesunderlyingitre-assessed.Inthisprocess,askingquestionslike“whatdoessuccessmean”or“howdoyouknowthingsareimproving”canidentifyvaluesthatpeopleholdinpeacebuilding,andhelp identifywaystoevaluatethem. Forexample,arestakeholderswillingtoconsiderthatgoodoutcomesmightmeanmoralcharacterdevelopmentratherthandevelopingsocialcapital.Reassessingvaluesmayalsohelpimplementersidentifywhetherornotatheoryofchangethatisinoperationisitselfaccidentlymarginalizingpeopleor groups, and can help make sure the evaluation process itself doesn’t compound problems (e.g. by usingcommunitiesorcommunitymembersasanendtostability).

Asevaluators thinkabouthow to structureevaluationsethically, thequestionofwhodecideskeyvalue-relatedquestionsarecritical.

17“AEAGuidingPrinciplesTrainingPackage,”AmericanEvaluationAssociation,accessedAugust1,2016,http://www.eval.org/p/cm/ld/fid=105.18AfricanEvaluationAssociation,"AfricanEvaluationGuidelines-StandardsandNorms."19QuinnPatton,Utilization-FocusedEvaluation,529.20Ibid.,537-9.

9

TEXTBOX4.1.QUESTIONTOASKABOUTWHOMAKESDECISIONS21

Herearesomegeneralquestionstoasktohelpevaluatorsthinkaboutethicsrelatedtothe“who”:

§ Whoarethemajorstakeholders?Howaretheyinvolvedindecision-making?

§ Arethemajorstakeholdersopentohavingtheirtheoryofchangereassessedbyabroadgroupofstakeholders?

§ Howcanwestructureaprocessthatcanidentifyandaddresssignificantmoralvaluedifferences?Addresspowerimbalances?Managecompetingclaimsandinterests?

§ Ifwesetupareferencegroup,whoshouldbeonit?Whatrolewillreferencegroupmembershave?Howwillpowerdifferentialsbemanagedinthereferencegroup?Shouldsomepeoplebeleftoffofthereferencegroupbecausetheyholdinordinatepower?

5.TheHow:HowDoWeEnsurePeacebuildingEvaluationsAreDoneEthically?

Abigconcern forevaluators isensuring theprocessbywhichwedoevaluations isethical. Beingethical inourconductisamajorfocusofethicsguidanceforpeacebuildingevaluators.Thissectionwillbethereforeshorterasthereareamplematerialsavailabletoexplorethesepointsingreaterdetail(seesuggestedresourcesattheend).However,concernsrelatedtoensuringwearedoingevaluationsethicallyaretypicallyraisedatthreelevels:1)howarefindingsgenerated?2)howdoesanevaluatorconducthimorherself?and,3)howsensitiveistheevaluationprocessitselfistotheconflictenvironment?

5.1HowFindingsAreGeneratedGatheringdataandanalyzingthefindings isthecentraltaskforevaluations. Inorderfortheevaluationtohaveintegrity,theprocessbywhichfindingsaregeneratedneedstohaveintegrity.Wecanthinkaboutthisintermsofthemethodsthatarechosentogatherdataandaskquestionslike:arethemethodsappropriateforthetaskathandand is the evaluator capable of implementing the appropriatemethods? Are there values implicit in the data-gatheringmethodologythatconflictwithvaluesheldbythestakeholders?

We can also think about the question of howwe collect data in terms of diligence and trustworthiness, askingwhetherevaluatorsarethoroughintheirwork(collectingandanalyzingdata),responsibleandup-frontwiththeevaluation’sstrengthsandlimits.Herequestionstoaskare:isthedatatrustworthy(collectedandanalyzedinwaysthat consider rigor and thoroughness that areappropriate to themethodology)? Am Imaking claimsbasedonsufficientdata?Whatarethelimitsofmydataandinferences?

Finally,weneedtothinkabouthowpeopleandrelationshipsarerespectedaspartofthedata-gatheringprocess.Herewecanaskquestionslike:doestheevaluationprocessrespectpeople?Doesitrespectrelationships?AmImakingsureIprotectpeople(interviewees,assistants,stakeholders)intheprocess(e.g.dopeoplefeelsafe?Isthereanypotentialcoercionoccurring?Isconfidentialitymaintained?)?AmItakingstepstoensurenoharmsoccur?AmIsharingmyfindingsbackwiththecommunitiesandstakeholdersinvolvedintheproject?

21Toseewaysinwhichquestionsabout“thewho”canmapontotheevaluationcycleseeAppendix1(Box2).

10

TEXTBOX5.1.QUESTIONSTOASKABOUTHOWANEVALUATIONISCONDUCTED22

Generalquestionstoconsidertoensureanevaluationisdoneethically:

§ Aretherevaluesimplicitinthedata-gatheringmethodologythatconflictwithvaluesheldbythestakeholders?Arethereothermethodsthatmightbemoreappropriate?

§ Isthedatatrustworthy?Arethelimitsofthedataandinferencesstated?

§ Doestheevaluationandpost-evaluationreportingprocessrespectpeople,relationshipsandtheculturalcontext?

§ Are people (interviewees, assistants, stakeholders) protected against coercion? Isconfidentialityensured?

§ Am I embodying important personal and professional character virtues (e.g. honesty,integrity,trustworthiness,competence)?WhoelsecanhelpmedeterminewhetherornotIamembodyingthesevirtues?

§ Howmighttheevaluationcontributetoharminthiscontext(e.g.worsenedethnic,political,socio-economicdivisions)?HaveIlimitedthepossibleharms?

§ HowcanIdotheevaluationinawaythatcontributestogood?

5.2HowanEvaluatorConductsHer-orHimself?Ethicsguidanceforevaluatorsuniformlyspeakstotheimportanceofanevaluator’sintegrity,whichinvolvebothcharacterexcellenceandasetofbehaviorsthatresultfromhavingintegrityorpropriety.Behaviorstypicallyincludebeing honest, transparent, trustworthy, competent (with respect to communication skills and evaluationmethodology), culturally sensitive, and declaring any personal conflicts of interest. 23 As an evaluator, the bigquestionstoaskinvolvethinkingthroughone’sownconduct–individuallyorwithmentorsandadvisors.AmIactingwithintegrity?AmIcontributingtotheoverallsocialenvironmentandpublicwelfare(oratleastnotcompromisingit)?

5.3HowSensitivetheEvaluationProcessistotheConflictEnvironmentThethirdlevelofconsiderationforensuringpeacebuildingevaluationsareethicalfocusesonthecontext.Conflictcontexts are volatile, peacebuilding addresses delicate issues and people are very vulnerable (physically andpsychologically).Thismeansitisveryeasytocontributetoharminandthroughtheevaluationprocess.Todogoodevaluation, therefore, requires considering theways inwhich the evaluation processwill do no harm by beingconflict-sensitive.24Thismeansbeingawareofwaysinwhichtheevaluationmayfeedintotensionsordivisionswithin the conflict context andmitigating them. Conflict sensitive framing tends to focuson reducingnegativeconsequences,howevermoral thinkingpushesus toalso consider thepositiveandalsoaskhow theevaluation

22SeeAppendix1(Box3)forhowallthreeofthesesetsofquestionscanbemappedontotheevaluationcycle.23Thislistwasgeneratedfrom“Ethics,”CanadianEvaluationAssociation,accessedAugust1,2016,http://www.evaluationcanada.ca/ethics;"AmericanEvaluationAssociationGuidingPrinciplesforEvaluators,"accessedAugust1,2016,http://www.eval.org/p/cm/ld/fid=51;AfricanEvaluationAssociation,AfricanEvaluationGuidelines-StandardsandNorms;Widmer,Landert,andBrachman“EvaluationStandards:ofSEVAL,theSwissEvaluationSociety(SEVALSTANDARDS)”;SociétéFrançaisedel’Évaluation,“Chartedel’evaluation:despolitiquespubliquesetdesprogrammespublics”;and,AustralasianEvaluationSociety,“CodeofEthics.”24See,forexample,OECD/DAC’sguidanceforhowtobeconflictsensitiveintheevaluationprocessinOECD,“EvaluatingPeacebuildingActivitiesinSettingsofConflictandFragility:ImprovingLearningforResults”.SeealsoJayawickrama,"’IfTheyCan'tDoAnyGood,TheyShouldn'tCome:'NorthernEvaluatorsinSouthernRealities,"26-41.Otherauthors,suchasEsserandVanderkampin“ComparableandYetContext-Sensitive?ImprovingEvaluationinViolentlyDividedSocietiesThroughMethodology,”42-56;andKennedy-Chouanein“DevelopingOECDDACGuidanceonEvaluatinginSettingsofViolentConflictandFragility,”110-115addressrelatedtopics.

11

processmightcontributetogood,suchasdesigninganevaluationitselfinawaythatcontributestoconstructiverelationshipbuildingbetweengroupsthatmistrusteachotherinaconflictsetting.

6.Conclusion:CreatingSpaceforMoralValuesTo conclude, we’ll go back to the beginning and our conversation between the program staff member andadministratorandaddanethicalevaluator:

“Thedatawilltelluswhattodo,”theadministratorsaysconfidently.

“Notsofast”theevaluatorinterjects,“Whatyouchoosetodohastodowithwhatyouvalue,what

you judgeasmattering,and thenhowyou interpret thedata inmakingyour choice. Let’shavea

conversationaboutwhatmatters–toyou,andtotheotherstakeholders.”

An ethical peacebuilding evaluator, just like an ethical evaluator more generally, develops reflexive skills inidentifyingand listeningtothevalue judgementsandclaimsthatstakeholdersaremaking. Werecognizeattheoutsetthatevaluationinvolvesmakingvaluejudgementsaboutthingsthatmattertopeople.Andtherearelayersofjudgmentsthatoccurinpeacebuildingevaluations,whichrequirecarefulandopenthinking,particularlyastheyoftenoccurinfragileordividedsettings.

Asaskilledandethicalevaluator,beingabletoaskimportantquestionsandcreatethespacetotalkaboutwhatis valued as good and right in peacebuilding is critically important. This isn’t a one-time event, but rather aperspectiveorsetofmusclesthataredevelopedwithuseovertime.Thequestionsexploredaboveaboutmoralvalueperspectives,whodecideswhatmattersinanevaluation,andhowtoimplementpeacebuildingevaluationsareintendedtobeahelpfulstartingplace.Itis,however,onlyabeginning.

12

6. ReferencesandAdditionalResourcesAmericanEvaluationAssociation.“AEAGuidingPrinciplesTrainingPackage.”AccessedAugust1,2016.

http://www.eval.org/p/cm/ld/fid=105.

Austin,Beatrix,MartinaFischer,andHansJ.Giessmann(eds.)AdvancingConflictTransformation:TheBerghof

Handbook.Secondedition.Leverkusen:BarbaraBudrichPublishers,2011.

Bush,Kenneth,andColleenDuggan.“AddressingEthicalandPoliticalChallengesinEvaluation.”PresentedattheEvaluationConclave,Kathmandu,February2013.

Esser,DanielE.,andEmilyE.Vanderkamp.“ComparableandYetContext-Sensitive?ImprovingEvaluationinViolentlyDividedSocietiesThroughMethodology.”JournalofPeacebuildingandDevelopment8,no.2(2013):42-56.

Fast,LarissaA.,andReinaC.Neufeldt.“EnvisioningSuccess:BuildingBlocksforStrategicandComprehensivePeacebuildingImpactEvaluation.”JournalofPeacebuildingandDevelopment2,no.2(2005): 24-41.

Fetterman,David,ShakehKaftwarian,andAbrahamWandersman.EmpowermentEvaluation.Secondedition.ThousandOaks,CA:Sage,2015.

Fournier,DeborahM.“Evaluation,”inEncyclopediaofEvaluation,editedbySandraMathison,161-162.ThousandOaks,CA:SAGEPublications,2005.

Galtung,Johan.PeacebyPeacefulMeans:PeaceandConflict,DevelopmentandCivilization.ThousandOaks,CA:SagePublications,1996.

Held,Virginia."TheEthicsofCare,"inTheOxfordHandbookofEthicalTheory,editedbyDavidCopp.NewYork,NY:OxfordUniversityPress,2006.

Kant,Immanuel.GroundworkfortheMetaphysicsofMorals.TranslatedbyArnulfZweig,editedbyThomasE.HillJr.andArnulfZweig.NewYork,NY:OxfordUniversityPress,2002.

Kennedy-Chouane,MeganGrace.“DevelopingOECDDACGuidanceonEvaluatinginSettingsofViolentConflictandFragility.”JournalofPeacebuildingandDevelopment8,no.2(2013):110-115.

Lederach,JohnPaul,ReinaNeufeldt,andHalCulbertson.ReflectivePeacebuilding:APlanning,Monitoringand

LearningToolkit.Mindanao:TheJoanB.KrocInstituteforInternationalPeaceStudies,UniversityofNotreDameandCatholicReliefServicesSoutheast,EastAsiaRegionalOffice,2007.

Mathison,Sandra(ed.)"ValueJudgment,"inEncyclopediaofEvaluation.ThousandOaks,CA:SAGEPublications,2005.

Mertens,DonnaM.TransformativeResearchandEvaluation.NewYork,NY:GuilfordPress,2007.

Munyaka,Mluleki,andMokgethiMotlhabi."UbuntuandItsSocio-MoralSignificance,"inAfricanEthics:AnAnthologyofComparativeandAppliedEthics,editedbyMunyaradziFelixMurove.Scottsville:UniversityofKwaZulu-NatalPress,2009.

Neufeldt,ReinaC.EthicsforPeacebuilders:APracticalGuide.Boulder,CO:Rowman&LittlefieldPublishers,2016.

Neufeldt,ReinaC.“Interfaithdialogue:assessingtheoriesofchange.”PeaceandChange36,no.3(2011):344–372.

OECD."EvaluatingPeacebuildingActivitiesinSettingsofConflictandFragility:ImprovingLearningforResults,"inDACGuidelinesandReferenceSeries.Paris:OECD,2012.

QuinnPatton,Michael.UtilizationFocusedEvaluation.Fourthedition.ThousandOaks,CA:Sage,2008.

Schwandt,ThomasA."TheLandscapeofValuesinEvaluation:ChartedTerrainandUnexploredTerritory,"inProgressandFutureDirectionsinEvaluation:PerspectivesonTheory,Practice,andMethods,editedbyDebraJ.RogandDeborahFournier.SanFrancisco,CA:JosseyBassPublishers,1997.

13

Weston,Anthony.A21stCenturyEthicalToolbox,thirdedition.NewYork,NY:OxfordUniversityPress,2013.

I.ProfessionalEvaluationAssociationGuidesonEthics

AfricanEvaluationAssociation(AfrEA)."AfricanEvaluationGuidelines-StandardsandNorms."AfrEA,2006/7.

AmericanEvaluationAssociation."AmericanEvaluationAssociationGuidingPrinciplesforEvaluators."AccessedAugust1,2016.http://www.eval.org/p/cm/ld/fid=51.

AustralasianEvaluationSociety(AES).“CodeofEthics.”Carlton:AES,2013.

CanadianEvaluationAssociation."Ethics."AccessedAugust1,2016.http://www.evaluationcanada.ca/ethics.

SociétéFrançaisedel’Évaluation.“Chartedel’evaluation:despolitiquespubliquesetdesprogrammespublics.”Paris:SociétéFrançaisedel’Évaluation,2006.

Widmer,Thomas,Landert,Charles,NicoleBrachman.“EvaluationStandards:ofSEVAL,theSwissEvaluationSociety(SEVALSTANDARDS).”Basel:TheSwissEvaluationSociety,2000.

II.ExplorationsofEthicalChallengesforEvaluatorsinConflictAreas

Bush,Kenneth,andColleenDuggan."EvaluationinConflictZones:MethodologicalandEthicalChallenges."JournalofPeacebuildingandDevelopment8,no.2(2013):5-25.

Bush,Kenneth,andColleenDuggan,eds.EvaluationintheExtreme:Research,ImpactandPoliticsinViolently

DividedSocieties.NewDelhi:SagePublicationsandInternationalDevelopmentResearchCentre(IDRC),2015.

Duggan,Colleen,andKennethBush."TheEthicalTippingPointsofEvaluatorsinConflictZones."AmericanJournal

ofEvaluation35,no.4(2014):485-506.

Jayawickrama,Janaka."'IfTheyCan'tDoAnyGood,TheyShouldn'tCome:'NorthernEvaluatorsinSouthernRealities."JournalofPeacebuildingandDevelopment8,no.2(2013):26-41.

Therearealsobriefdiscussionsofethicsrelatedtoevaluationin:

Church,Cheyanne,andMarkRogers.DesigningforResults:IntegratingMonitoringandEvaluationinConflict

TransformationPrograms.Washington,D.C.:SearchforCommonGround/UnitedStatesInstituteforPeace,2006.

Lederach,JohnPaul,ReinaNeufeldt,andHalCulbertson.ReflectivePeacebuilding:APlanning,MonitoringandLearningToolkit.Mindanao,PH:TheJoanB.KrocInstituteforInternationalPeaceStudiesandCatholicReliefServicesSEAPRO,2007.

III.EthicsandPeacebuildingMoreGenerally

Anderson,MaryB.DoNoHarm:SupportingCapacitiesforPeacethroughAid.Boulder,CO:LynneReinnerPublishers,1999.

Anderson,MaryB.,andLaraOlson.ConfrontingWar:CriticalLessonsforPeacePractitioners.Cambridge,MA:CDACollaborativeLearningProjects,2003.

Barry,Bruce,andRobertJ.Robinson."EthicsinConflictResolution:TheTiesThatBind."InternationalNegotiation7(2002):137-42.

Cohen,Cynthia.WorkingwithIntegrity:AGuidebookforPeacebuildersAskingEthicalQuestions.Waltham,MA:BrandeisUniversity,2001.

Fast,LarissaA.,ReinaC.Neufeldt,andLisaSchirch."TowardEthicallyGroundedConflictInterventions:ReevaluatingChallengesinthe21stCentury."InternationalNegotiation7,no.2(2002):185-207.

InternationalAlert."CodeofConduct:ConflictTransformationWork."London:InternationalAlert,1998.

14

Kelman,HerbertC.,andDonaldP.Warwick."TheEthicsofSocialIntervention:Goals,Means,andConsequences."InTheEthicsofSocialIntervention,editedbyGordonBermant,HerbertC.KelmanandDonaldP.Warwick,3-33.Washington,D.C.:HemispherePublishingCorporation,1978.

Laue,James,andGeraldCormick."TheEthicsofInterventioninCommunityDisputes."InEthicsofSocialIntervention,editedbyGordonBermant,HerbertC.KelmanandDonaldP.Warwick,205-32.Washington,D.C.:HemispherePublishingCorporation,1978.

Murithi,Tim.TheEthicsofPeacebuilding.Edinburgh:EdinburghUniversityPressLtd,2009.

Neufeldt,Reina.EthicsforPeacebuilders:APracticalGuide.Lanham,MD:Rowman&LittlefieldPublishers,2016.

Sawatsky,Jarem.JustpeaceEthics:AGuidetoRestorativeJusticeandPeacebuilding.Cambridge:TheLutterworthPress,2009.

Slim,Hugo."DealingwithMoralDilemmas."InPeacebuilding:AFieldGuide,editedbyLucReychlerandThaniaPaffenholz.Boulder,CO:LynneRiennerPublishers,2001.

Warfield,Wallace."IsIttheRightThingtoDo?APracticalFrameworkforEthicalDecisions."InAHandbookofInternationalPeacebuilding:IntotheEyeoftheStorm,editedbyJohnPaulLederachandJaniceMoomawJenner,213-23.SanFrancisco,CA:Jossey-Bass,2002.

15

8.Appendix:EthicsQuestionsduringanEvaluationCycle

8.1.QuestionsontheWhat(What’sGoodandRight)

Entry/Contracting

Evaluation

Design

Data

Collection

DataAnalysis&Interpretation

Communication

ofResults

Utililization

ofResults

§ Doestheevaluationdesignreflecttherangeofvaluesthatwillbeusedtojudgegoodandright?

§ Willtherebemethodslimitations?§ Aretheeffectsofcultureand

contexttakenintoconsiderationvis-à-visthegoodandrightinthedesign?

§ Arethedatacollectionmethodologiescapturingthedepthandrangeofconsiderationsofgoodandrightintheprojectbyitsdiversestakeholders(e.g.areyoumeasuringrelationshipsusinganends-basedsurvey?)

§ Aretheeffectsofcultureandcontexttakenintoconsiderationvis-à-visthegoodandright?

§ Doesanalysisandinterpretationtakeintoconsiderationtherangeofvaluesheldaboutgoodandrightbydiversestakeholders?(E.g.areyoudefaultingtoends-onlythinking?)

§ Doyourevaluationproducts(writtenandverbal)reflecttherangeofconsiderationsofgoodandrightbasedonthevaluesofitsstakeholders?

§ Aremoralvaluesexplicitlydiscussed?

§ Doyouanticipateanymisuseoffindingsiftherearediversevaluesincludedintheevaluation?Howcanyouaddressthis?

§ Arecommissionersopentodiscussingwhatisconsideredgoodandrightintheevaluation?(Isitanends-onlyfocus?Areconsiderationsofmeansincluded?Virtues?Relationships?)

§ Is“good”predeterminedbythosecommissioningtheevaluation?§ Doallstakeholdersagreeonthevaluesbeingusedtojudge?§ Aretheredifferencesbetweenwhatthecommissioningorganizationvalues

andwhatvulnerablepopulationsintheinitiativevalue?Ifso,whatarethey?

16

8.2QuestionsRegardingtheWho(WhoDecides)

• Whoarethemajorstakeholders?Howaretheyinvolvedindecision-making?(e.g.outsidersorinsiders?)

• Arethemajorstakeholdersopentohavingtheirtheoryofchangereassessedbyabroadgroupofstakeholders?(E.g.asking“whatwouldsuccesslookliketoyou?”)

• Aresomevoicesmoreorlessrepresentedinyourdatacollection?Isthisfairandappropriate?

• Istheevaluationstructuredtoidentifyandaddresssignificantpowerimbalancesandmoralvaluedifferencesbetweenstakeholders?

• Areyouusingareferencegroup?Doesitincludediversevoices?Howwillyoumanagepowerdifferentials?

• Doestheanalysisandinterpretationtakeintoconsiderationtherangeofstakeholdersviewsappropriately?

• Isthereundueinfluenceofsomeviews(e.g.thecommissionersoryourown)ontheanalysis?

• Doyouanticipateanymisuseoffindingsbymajorstakeholders?Howcanyouaddressthis?

• Canyoudomoretoensureevaluationfindingsareused?

• Inyourcommunications,doyoureflectconsiderationsofabroadgroupofstakeholders?

• Arevulnerablegroupsandcommunitygroupsincludedinthecommunication?

• Areresultsshared(includingmadepublicasappropriate)?

Entry/Contracting

Evaluation

Design

Data

Collection

DataAnalysis&Interpretation

Communication

ofResults

Utililization

ofResults

17

8.3QuestionsofHowtoConductanEvaluationEthically

Entry/Contracting

Evaluation

Design

Data

Collection

DataAnalysis&Interpretation

Communication

ofResults

Utililization

ofResults

§ Isthedatacollectedinatrustworthyway?

§ Doestheprocessofdatacollectionrespectpeopleandrelationships?Arepeopleprotectedfromcoercion?Isconfidentialityensured?Areallgroupsrepresented?

§ Areyouembodyingpersonalandprofessionalcharactervirtues?(e.g.trustworthy,havingintegrity)

§ Isthedatacollectionprocessconflictsensitive?

§ Isthedataprotectedandhandledcarefully?

§ Areyoucompetentintheevaluationmethodsproposed?

§ Arethedatacollectionmethodologiesinconflictwithothervaluesheldbystakeholders?

§ Isthedesignconflictsensitive?(Areyoureinforcinganystereotypes?Areyouavoidingissues?)

§ Haveyouconsideredcultureandcontext?

§ Whatarethelimitsofyourdesign?

§ Areyouactingwithintegrity(honest,transparent,trustworthy,competent)?Doyouhaveanyconflictsofinterest?

§ Cantheevaluationbedoneethically?Willthisevaluationcontributetothecommongood(doesitneedto)?

§ Willthecommissionerdiscussethicaldilemmas?§ Areyoupreparedtodecline?§ Haveyouconsidered“theWho”and“theWhat”questions?

§ Doyouanticipateanymisuseoffindings?Howcanyouaddressthis?

§ Canyoudomoretoensureevaluationfindingsareused?

§ Isthecommunicationstrategyconflictsensitive?

§ Arevulnerablegroupsandcommunitygroupsincludedinthecommunication?

§ Areresultsshared?§ Inyourcommunication,doyou

actwithintegrity(e.g.honest,transparent,includelimitations)

§ Isconfidentialitymaintained?§ Aretheclaimsbeingmadebaseduponsufficientdata?§ Whatarethelimitsofthedataandinferences?§ Doesyouranalysisconsidermultipleviewpoints?Isitcareful?

Areyoutransparentinyourprocess?§ Isconfidentialitymaintained?