arts evaluation toolbox - changing the story€¦ · ref level 3 takes the five pillars of our...
TRANSCRIPT
www.britishcouncil.org
1
ARTS EVALUATION TOOLBOX April 2018
EvaluationCoreEssentialsPage
Thispageisaquickstarttowhatyouneedtodevelopyourprojectevaluation
• QuickonepageintroductiontoEvaluation• JoiningtheDotsPresentation• ArtsTheoryofChangeTemplateforyourproject• TheArtsEvaluationPlanningTemplate• ArtsImpactSummaryTemplatetoreportattheendofyourproject
ThefullArtsEvaluationToolboxismadeupof:-
• Introductionandwhywedoevaluation• Planningyourevaluation• ResultsandEvidenceFrameworkGuidance(REFLevel2andLevel3)• EvaluationtoolkitstosupportlookingatparticularREFthemessuchsoftpower• Communicatingyourevaluationfindings• ODAfundedprojects• ValueforMoneyapproach• Somepointstoconsiderwhenthinkingaboutyourevaluationplan• Bankofsurveyquestionstohelpmeasureimpactandgatherfeedback• ArtsandCultureEvidenceResearchBank• ArtsEvaluationTrainingResources,PodcastsandWebinar• ArtsEvaluationPlan2016-2020
FurtherSupport
RegionalEvaluationManagershttp://intranet.britishcouncil.org/Site/SPI/Pages/Contact-us.aspxIanThomas–HeadofEvaluation,Arts,[email protected]
Introduction
“Moreimportantlygetbetteratevidencinghowbrilliantweare–showing,measuring,monitoring,evaluatingandtellingourstoryincrediblybrilliantly-theneverythingelsewilllookafteritself,includingthemoney.’”CiaránDevane,July2016
Collectively,wecantellapowerfulstoryaboutthevaluableimpactofpublicandprivateinvestmentintheartsglobally.
www.britishcouncil.org
2
• Realise-Thevalueofyourprojectorprogramme;• Compare-Connect,benchmark,andsharetostrengthenyourprojectorprogramme;• Communicate-Havericherconversationsaboutyourimpactandvalueofyourprojector
programme;• Grow-Useactionableinsightstodriveyourgrowthinyourarea.
Evaluationisaprocessthattakesplacebefore,duringandafteranactivity.Itincludeslookingatthequalityofthecontent,thedeliveryprocessandtheimpactoftheactivityorprogrammeontheaudience(s)orparticipants.
Takingtheopportunitytounderstandwhetheryouachievedwhatyousetoutto,howwellyoudidit,whatimpactyouractivityhashadandtoreflectcriticallyonboththeactivitiesandprocesseswillbenefityouandyouraudiences.Thisknowledgecanbeusedinternallybyyourteamtodriveimprovementandexternallytodemonstrateachievements.
Thebigquestionishowdoestheartsdeliveranddevelopanevidencebaseagainst:security,prosperityandinfluence?
TheBritishCouncil’sArtsStrategysetsoutfivepillarsforourglobalartsprogramme:-
• ArtsforSocialChange;• Sharingartsworkwiththeworld;• Capacitybuilding;• Fosteringcollaborationandnetworks;• Policyandresearch
Evaluationhelpsusjudgeandhelpsusshowwhetherwe’vemetourgoals/outcomesagainstourArtsStrategyandformspartofyourevaluationplanandthinkinghowyourworkrelatesbacktotheArtsStrategyandtotheCorporatePlan.
• AcopyoftheArtsStrategycanbedownloadedfromherehttps://www.dropbox.com/s/cjl87t9ryh2xjhy/BC013_Arts%20Strategy.pdf?dl=0
www.britishcouncil.org
3
Evaluationshouldhelpyoutomakesurethatwhatyouaredoingisasgoodaspossibleforthosetakingpart.Takingtimetofocusonwhatisworkingwell,whatneedschanging,andwhathassurprisedyou–alladdstothelearningandstaffdevelopmentofyourteam.Theevidenceyoucollectintheprocesswillbeinvaluablewhenitcomestolobbyingandcampaigningforyourworkandprogrammesandsharingandlearningfrombestpracticeacrossourteamsandprogrammes.
• IntroductiontoEvaluationWebinar-gettingstartedandtheoryofchangehttp://publicservicetransformation.org/resources/evaluation-and-analysis/718-evaluation-webinar-getting-started-and-theory-of-change
The Life Cycle of your Evaluation Plan
Anevaluationshouldprovideinformationthatiscredibleanduseful,enablingtheincorporationoflessonslearnedintothedecisionmakingprocess.
Evaluationatitsheartisasimpleprocess.Itinvolves:
1. Statingwhatyouintendtoachieve,whataretheoutcomesyouwanttoachieve,whatisyourevidenceofneed/baselineforyourproject?
2. Decidinghowyouwillshowwhetherthishasbeenachieved3. Gatheringthenecessaryevidence4. Summarisingandanalysingtheevidence5. Comparingfindingswithwhatwasoriginallyoutlined,decidingontheimplicationsand
providingrecommendationsandideasforfuturework
TheBritishCouncilisgoodatcapturingoutputsandreachatanactivitylevel,throughourScorecardsystem.Manyprogrammesarealsocollectingevidenceofshort-termchangethroughendofprojectevaluations.Wealsoneedtocapture,throughthearts,thelongtermimpactsandchangeswearemakingthroughourprojects.
www.britishcouncil.org
4
Core Evaluation Standards
Sevenessentialstandardsthatshouldguidethemonitoringandevaluationofallprojects,largeorsmall:-
1. Youhaveaclearideaofthechangesyouwanttosee,andhowyourprojectwillachievethemsuchasthroughaTheoryofChangeandyourevaluationplan;
2. Youhaveaclearplanforhowyouwillmonitoractivitiesandmeasurewhetheryouareachievingchange;
3. Youunderstandthecurrentsituationwellenoughsothatattheendoftheprojectyoucantellwhathaschangedsuchasbydoingabaselineandevidenceofneedforyourproject;
4. Youanalyseyourdatainastructuredandtransparentway;5. Whenmeasuringchangeyoulookatanissuefromdifferentperspectivesinorderto
improvetheaccuracyofyourresults;6. Youonlycollectdatayouintendtouseandisdoneinanethicalway;7. Yousharethefindingsandlearningfromyourevaluation.
Thinking about your evaluation
Therearethreeaimsofevaluation:
• Tosupportthedevelopmentofyouractivity(formativeevaluation).• Toensureyoumanageitbetternexttime(evaluationofyourprocesses).• Toassessthefinalimpactofyouractivity(summativeevaluation).
Allprojectsshouldhaveanevaluationplaninplacefromtheearlystagesofthedevelopmentoftheproject,beclearinwhatoutcomesyouwanttoachieve,and,whereappropriate,aTheoryofChange(suchasiftheprojectisODAfunded).Thedepthoftheevaluationplanwilldependonthescaleandambitionsoftheproject.
• BritishCouncilwebinarrecordingonmonitoringandevaluationclickonthislink"Everythingyoueverwantedtoknowaboutmonitoringandevaluationbutwereafraidtoask".(requiresfreeregistrationlogin)https://cc.callinfo.com/cc/playback/Playback.do?id=2c4lto
• BritishCouncil’sStrategyPerformanceandInsightTeam’sEvaluationToolkitisontheIntranetherehttp://intranet.britishcouncil.org/Site/MonitoringandEvaluating/Pages/Default.aspx
• OnlineoutcomestoolTheOutcomesMatrixisatooltohelporganisationstoplanandmeasuretheirsocialimpact.Itincludesoutcomesandmeasuresfornineoutcomeareasand15beneficiarygroups.http://www.goodfinance.org.uk/impact-matrix
www.britishcouncil.org
5
Thisapproachcanbesummedupas‘PROPER’evaluation:
• Planned
• Reflective• Organisation-wide
• Pragmatic
• Enablingandenergising
• Responsive/re-iterative
www.britishcouncil.org
6
The Results and Evidence Framework
FromApril2017weareintroducinganewResultsandEvidenceFrameworktomeasuretheimpactofourculturalrelationsmodeltakingathematicapproach.Itisdesignedtoshowouroverallimpactasclearlyaspossibleusingthedatawecollectandtohelpteamsdesignevidencestrategiesforthemainoutcomeswearetryingtoachieve.
TheREF8Themes
ArtsREFLevel2
Atlevel2oftheREF,inarts,wearelookingtocountviadataentryontoSAP:-
• Howmanypeoplewereachthroughourglobalartsprogrammesviafacetoface,online/digital,publicationsandbroadcasts.
• Wealsoarecountingthenumberofartistsandartsorganisationsweworkwith,brokendownbyUKbasedartistsandartsorganisationsandthosewhoarenotbasedintheUKtohelpustellourUKValuenarrative.
• DigitalfiguressuchassocialmediaandwebsitesandbecollectedusingtheBritishCouncil’sdigitalmonitoringtoolshttps://digital.britishcouncil.org/skills/web-analytics/planning/scorecardInordertohavetraffictoyourproductcountedtowardsscorecard,itneedstobelistedonthedigitaldashboard.Ifyouareregisteringawebsite,itwillneedtobetrackedusingWebtrendsinordertoreportdataintothedashboard.
www.britishcouncil.org
7
Everymonth,wereportonallwebsitetrafficandbasicengagement.Everyquarter,wereportonthequarterlyperformanceofallsocialmediaproductsandwebsites.
• OurdataisalsobrokendownbygendertohelpmeetourreportingrequirementsaroundODAandGender.
ArtsREFLevel3
REFLevel3takesthefivepillarsofourglobalartsstrategyandthroughevaluationandresearchlookstodevelopanevidencebaseandnarrativearoundourglobalartsprogrammes:-
• Thedevelopmentofsocialoutcomesoftargetedindividualswhichcontributestowardsgreatersocialinclusionandsocialengagementintargetedcountries.
• Culturalprofessionalsandartistsdeveloptheirknowledge,artisticpractice,skillsandnetworkstocontributetowardsgreaterprosperity.
• ArtsandculturalorganisationsintheUKandtargetcountriesraisetheirprofile,showcaseUKartsandincreasetheeconomicvalueoftheirpartnershipscontributingtowardsUK'sinfluenceandattractionintheWorld.
• TheartsandculturalinfrastructureandcreativeeconomiesoftheUK,citiesandtargetcountriesarestrengthenedthroughinvestmentandcollaboration.
• TheUK'sinternationalreputationisenhancedthroughexperiencesofartsandculturecontributingtoourculturalrelationsandsoftpowerapproaches.
Artsprogrammesmayalsocontributeevidenceofimpactatlevel3towardsotherresultsareaswithintheframeworksuchasSkills,WomenandGirls,YoungPeopleandSociety.
TosupportartsmanagersworkindevelopingtheirevidencebaseatREFLevel3wehavecommissionedanevidencestrategyfortheartswhichlooksatthevariousevaluationapproaches,evaluationtoolsandquestionsforeachofthefivethemesfortheartsatlevel3tohelpartsmanagerschoosetheappropriateapproachfortheirevaluation.
Thereareavarietyofevaluationtoolslaterinthistoolboxwhichmayalsohelpwithyourevaluation.
Furtherreadingandsupport
TheArtsEvidenceStrategycanbedownloadedhere:-https://www.dropbox.com/s/1dd6sl8c5z6dbzm/171002%20British%20Council%20Evidence%20Strategy%20for%20the%20Arts%20Final%20Report.docx?dl=0
ThereisFAQ’ssheetontheResultsandEvidenceFrameworkonlinehere:-https://www.dropbox.com/s/50rse4xehxplv81/All%20you%20need%20to%20know%20about%20the%20Results%20and%20Evidence%20Framework.docx?dl=0
TherearehelpvideostosupportyourplanningandenteringdataontoSAPforREFLevel2here:-http://intranet.britishcouncil.org/Site/SPI/Performance/Pages/SAPEasyStepGuidesandVideos.aspx
www.britishcouncil.org
8
Some evaluation tools to support your evaluation plan
Youareencouragedtoadaptthesetoolstothespecificsocio-culturalcontextinwhichyouareworkingandtobeoutcomesledbasedonevidenceofneedforyourproject.
Planningyourevaluation
• BritishCouncilArtsEvaluationPlanningTemplatestarthereyourevaluationplanningandthinkingwiththistemplate.https://www.dropbox.com/s/lzt7o6tqstj9vtl/Arts%20Evaluation%20Planning%20Template.docx?dl=0
• BritishCouncilArtsTheoryofChangeTemplate-completingaTheoryofChangeTemplatehelpsyoutomapoutthenecessarystepsyouneedtotaketoachieveaparticulargoal.Itcanhelpyoutoidentifythepotentialimpactandrisksofyourplan,andconnectyourworktoabiggergoalandisarequirementforALLODAfundedprogrammes.https://www.dropbox.com/s/8058lndj8vyji65/Arts%20Theory%20of%20Change%20Template.docx?dl=0
• NESTATheoryofChangeVideohttps://www.youtube.com/watch?v=6zRre_gB6A4• DevelopingaTheoryofChangeGuideshowsaproject’spathfromneedstoactivitiesto
outcomestoimpact.Itdescribesthechangeyouwanttomakeandthestepsinvolvedinmakingthatchangehappen.http://www.thinknpc.org/publications/theory-of-change/
• NESTA’sStandardsofEvidenceframeworktohelpstructurehowevidenceisgathered,interpretedandassessedhttp://www.nesta.org.uk/publications/nesta-standards-evidence
• Onlinewebinar-Whatisgoodevidence?http://www.alliance4usefulevidence.org/event/webinar-what-is-good-evidence
• Howtocostanevaluationhttp://www.ces-vol.org.uk/tools-and-resources/planning-for-monitoring-evaluation/guidance-paper-one-tools.html
• Whattoincludeinanevaluationbriefhttp://www.ces-vol.org.uk/tools-and-resources/planning-for-monitoring-evaluation/guidance-paper--two-eval-brief-tools.html
• NESTADIYToolkit-theDIYToolkithasbeenespeciallydesignedfordevelopmentpractitionerstoinvent,adoptoradaptideasthatcandeliverbetterresults.
• http://diytoolkit.org/
EvidenceofNeed/developingyourbaselinesupportmaterial
Youunderstandthecurrentsituationwellenoughsothatattheendoftheprojectyoucantellwhathaschanged.Ifyoudon’tknowwhatthesituationisatthebeginningofyourproject,itwillbeverydifficulttoassesschangeattheend.Allprojectsshouldcollectbaselinedatanearthestartofaproject.Youcanthencomparethiswithdatayoucollectlateron,toseeifyourprojectismakingadifference.Baselinedatashouldbecollectedasnearaspossibletostartoftheproject,butonlywhenyouhaveaclearideaofwhatchangeyouwanttoseeandhowyouintendtogetthere.
www.britishcouncil.org
9
• CultureSegmentsReports-CultureSegmentsisauniversal,sector-specificsegmentationsystemforcultureandheritageorganisations.TheprincipalobjectiveofCultureSegmentsistoprovidethesectorwithasharedlanguageforunderstandingtheaudience,withaviewtotargetingthemmoreaccurately,engagingthemmoredeeply,buildingrelationshipsandmaximisingloyalty.https://www.dropbox.com/sh/4estavlze0zqak1/AACv7nwUsw7CObWTA4GYkbAJa?dl=0
• PortlandSoftPowerTop30CountriesIndexhttp://softpower30.portland-communications.com/ranking/-includingtheculturesubindex
• WorldBankCountryProfileshttp://www.worldbank.org/en/country• CivicusCivilSocietyIndex(2009)
-CSIIndicatordatabasehttp://www.civicus.org/csi/-HollowayR(2001)UsingtheCivilSocietyIndex:AssessingtheHealthofCivilSociety.https://www.civicus.org/view/media/IndexHandbook.pdf
• InternationalFederationofArtsCouncilsCountryProfileshttp://worldcp.org/index.php• BritishCouncilCulturalSkillsUnitResearchreportsinculturalskillsgaps
https://www.britishcouncil.org/cultural-skills-unit/research• AudienceFinderisafreenationalUKaudienceinsightsdataanddevelopmenttool,enabling
culturalorganisationstounderstand,compareandapplyaudienceinsightacrossEngland.http://www.audiencefinder.org/
• AmericansandtheArtsSurveyhttps://www.dropbox.com/s/luyjvfinhfgxqxv/Americans%20and%20the%20Arts.pdf?dl=0
• VisitEnglandInternationalMarketsSegmentsandCountryProfileshttps://www.visitbritain.org/markets-segments
• AssociationofIndependentMuseumsVisitorVerdictoffersreal-timeresultsonaonlinedashboardandprovidesinformationaboutwhovisitsyourmuseum,whytheyarethere,whattheydoontheirvisitandwhattheythinkoftheirexperience.Theresultsyoureceivearegivenindustrycontextbyprovidingabenchmarkagainstsimilarmuseums.http://www.aim-museums.co.uk/content/aim_visitor_verdict/
• MeasuringyourSocialMediaImpactandTheSocialMediaMonitorhttp://intranet.britishcouncil.org/news/Pages/Your_Social_Media.aspx
• EUCulturalImportandExportDatasetsproducedeachyear.http://ec.europa.eu/eurostat/statistics-explained/index.php/Culture_statistics_-_international_trade_in_cultural_goods
• WorldCitiesCulturalForumDataset(2018)70globalcitiesculturalinfrastructureandconsumptionindicators.http://www.worldcitiescultureforum.com/data
www.britishcouncil.org
10
Generalevaluationtools
Developingyourevaluationquestions
Thefourprinciplesofagoodevaluationquestionhttps://www.dropbox.com/s/1rho1ved1b7wg2r/4%20Principles%20of%20a%20good%20evaluation%20question.docx?dl=0
Developingyourcasestudies
ISPOSpresentationoncasestudieshttp://intranet.britishcouncil.org/uk/sites/KPI/Evidence%20Strategies%20Training/Case%20studies%20-%20Kelly%20Beaver.pptx
AHRCGuidanceonCaseStudieshttp://www.ahrc.ac.uk/documents/guides/case-studies-guide/
Developingyourbaselineandendofprogrammesurveys
Introductiontoundertakingsurveysincludingsamplingapproacheshttp://intranet.britishcouncil.org/uk/sites/KPI/Evidence%20Strategies%20Training/Intro%20to%20Survey%20Methods%20-%20Olivia%20Ryan.pdf
ISPSOSpresentationonsurveysandfocusgroupshttp://intranet.britishcouncil.org/uk/sites/KPI/Evidence%20Strategies%20Training/Qualitative%20Methods%20-%20Fay%20Sadro%20and%20Ellie%20Thompson.ppt
Guidancenotesonundertakingsurveyshttps://www.dropbox.com/s/93qqf7mo52cuvpp/Developing%20Surveys.docx?dl=0
https://onthinktanks.org/articles/how-on-earth-do-you-measure-the-impact-of-your-events/
Evaluationtoolkitslookingatparticularthemes/issues
ArtsandSocialChange
• Arts,HealthandWellbeingEvaluationFrameworkdevelopedbyPublicHealthEnglandhttps://www.gov.uk/government/publications/arts-for-health-and-wellbeing-an-evaluation-framework
• Toolsformonitoringandevaluatingchildren’sparticipationhttp://www.savethechildren.org.uk/sites/default/files/images/ME_Toolkit_Booklet_5.pdf
• GuideonmeasuringSocialReturnonInvestmentandmeasuringSocialValuehttp://socialvalueuk.org/what-is-sroi/the-sroi-guide
• ArtsEducationEvaluationWebinarNationalEndowmentfortheArtsEducationstaffhostedawebinarprovidinganoverviewtotheBostonYouthArtsEvaluationProjecthttps://www.arts.gov/video/arts-education-webinar-past-current-and-future-needs-youth-arts-organizations-using-data
www.britishcouncil.org
11
• CitizenengagementscaleWorldVision’sInfluenceandEngagementmatrixthisscalecouldbeusedtomeasureincreasesincitizenlevelsofengagementincommunitydecisionmakerprocesses.Itissuitedtocommunitylevelinfluenceandchange.https://www.dropbox.com/s/7996yfu86bxxhtp/citizen%20engament%20scale.docx?dl=0
• LifeEffectivenessScale-measuringpersonalchangeatanindividuallevel(aperson'scapacitytoadapt,surviveandthrive)http://www.wilderdom.com/leq.html
• WellbeingScale-measuringindividualchangesinwellbeinghttps://www2.warwick.ac.uk/fac/med/research/platform/wemwbs/andalsotheMappniessApptohelpmonitorWellbeinghttp://www.psyt.co.uk/
• Child&YouthResilienceMeasuredesignedasascreeningtooltoexploretheresources(individual,relational,communalandcultural)availabletoyouthaged12to23yearsold,thatmaybolstertheirresilience.ThemeasurewasdesignedaspartoftheInternationalResilienceProject(IRP)http://cyrm.resilienceresearch.org/
• PolicyChangeMeasureprogressininfluencingaspecificpolicyaspartofaproject.TheVSO‘AdvocacySuccessScale’consistsofeightpointsandisusedformonitoringpolicyinitiativeswithpartners.WiththeVSOtoolthemainbenefitisintheprocessofreviewingandchallengingevidencetogetherwithvolunteersandpartners,andthenreviewingplansforthefollowingperiod.https://www.bond.org.uk/data/files/Effectiveness_Programme/research/ScalarApproaches.pdf
• PartnershipsforLearningguidetoevaluatingartseducationprojectshttp://webarchive.nationalarchives.gov.uk/20160204101926/http:/www.artscouncil.org.uk/advice-and-guidance/browse-advice-and-guidance/partnerships-for-learning-a-guide-to-evaluating-arts-education-projects
Furtherreadingandsupport:-
ThefullCultureandDevelopmentEvaluationToolkitisonlinehere:-https://www.dropbox.com/s/jcelbyy0il5976j/Final%20M%26E%20Toolkit%20for%20C%26D%20Itad%20180714.pdf?dl=0
TheexternalevaluationoftheCultureandDevelopmentGoldenThreadProgrammeisonlinehere:-https://www.britishcouncil.org/arts/culture-development/about
CreativeEconomyandEconomicImpacts
• Measuringtheeconomicbenefitsofartsandculturehttp://www.artscouncil.org.uk/sites/default/files/download-file/Measuring_the_economic_benefits_of_arts_and_culture.pdf
• CulturalEconomicImpactToolCalculator-thisonlinetoolkitisdesignedtoestimatetheeconomicimpactofculturaleventsandactivitiesthattakeplaceinEngland,ScotlandandWales.ThetoolcanalsobeusedforprojectsbasedinNorthernIrelandhttp://www.eitoolkit.org.uk/
www.britishcouncil.org
12
• MappingtheCreativeIndustriestoolkithttp://www.acpcultures.eu/_upload/ocr_document/BRITISH%20COUNCIL_MappingCreativeIndustriesToolkit_2-2.pdf
• EUCulturalImportandExportDatasetsproducedeachyear.http://ec.europa.eu/eurostat/statistics-explained/index.php/Culture_statistics_-_international_trade_in_cultural_goods
• DCMSculturalexportdatasethttps://www.gov.uk/government/collections/dcms-sectors-economic-estimates
Furtherreadingandsupport
BritishCouncil’sValueforMoneyToolkit(arequirementforallODAfundedprogrammes)http://intranet.britishcouncil.org/uk/sites/ES/ADPages/ValueForMoneyToolkit.aspx
CreativeEnterpriseProgrammeCaseStudieshttp://creativeconomy.britishcouncil.org/media/uploads/files/British_Council_CEW_Case_Studies_FINAL_18-05-2016.pdf
CapacityBuildingandFosteringNetworks
• Networkingmappingtoolshttps://kumu.io/freeonlinetoolwhichmapsyournetworksintoanonlinevisualnetworkmap
• GraphCommonsisacollaborativeplatformformapping,analysingandpublishingdata-networks.Ittransformsdataintointeractivemapsanduntanglecomplexrelationsthatimpactthemandtheircommunities.Guidanceonnetworkingmappinghttps://blog.graphcommons.com/mapping-networks/
• CulturalPartnershipsEvaluationTool-offersawaytoreportthecomplex,multi-layeredoutcomesofapartnership.Itexamineskeyareastomeasurereturnoninvestmentincludinginputs,leverage,community,culturalandbusinessbenefits.http://culturehive.co.uk/resources/a-cultural-partnership-evaluation-toolandalsohttps://www.artsandbusinessni.org.uk/media/1207/2012-05-15-12-53-09-82-how-to-evaluate-partnerships-a-guide-final.pdf
• Measuringtheimpactofyourevents–surveyquestionsforposteventfeedbackhttps://onthinktanks.org/articles/how-on-earth-do-you-measure-the-impact-of-your-events/
• Howtoevaluateonlinesuccess-detailstheonlineanalyticsmethodology,researchandkeyfindingsofamajorstudyintohowartsandculturalorganisationscanevaluateonlinesuccesshttp://culturehive.co.uk/resources/lets-get-real-how-to-evaluate-online-success
• Usingsocialmediadatatounderstandtheimpactofaconference/eventsonsocialnetworks-http://www.nesta.org.uk/publications/net-effect-using-social-media-data-understand-impact-conference-social-networks#sthash.rk5V5DBZ.dpuf
• SocialNetworkAnalysisToolsSocialnetworkanalysisisthemappingandmeasuringofrelationshipsandflowsbetweenpeople,groups,organisations,computersorother
www.britishcouncil.org
13
information/knowledgeprocessingentitieshttp://www.kstoolkit.org/Social+Network+Analysis
• MeasuringSocialMediaImpactToolkithttps://www.dropbox.com/s/zwz6yl2823g3fry/social_media_evaluation_published_on_af_site.pdf?dl=0
Furtherinformationandsupport
10YearsoftheCreativeEntrepreneurProgrammeEvaluationReporthttps://www.dropbox.com/s/faph92xb98q3l2h/10%20years%20of%20the%20YCE%20programme.pdf?dl=0
IstanbulCreativeHubsOnlineMappinghttps://graphcommons.com/graphs/21cc69ad-86d7-489d-99ac-524a3aeb15b8?auto=true
CulturalRelationsandSoftPower
BritishCouncilCulturalRelationsModel
Evaluationtools
• ReturnonInfluenceGREATCampaignguidanceonmeasuringReturnonInfluencehttps://www.dropbox.com/s/jfu5hgpga268z9x/Return%20on%20Influence.docx?dl=0
www.britishcouncil.org
14
• Contributionanalysisapproachsetsouttoverifythetheoryofchange,butalsotakesintoconsiderationotherinfluencingfactorsCausality/contributionisinferred‘beyondreasonabledoubt’https://www.betterevaluation.org/en/start_here/contributeandalsoslides23&24ofthispresentationhttp://intranet.britishcouncil.org/uk/sites/KPI/Evidence%20Strategies%20Training/BC%20Theory-based%20IE%20MK.PPTX
• OUCulturalValueModelTheCVMisatoolforrethinkingandassessingculturalvaluefrommultipleperspectivesandcouldbeusedforaSeasonorFestivalsuchastheIranSeasonandShakespeareLiveshttp://www.open.ac.uk/researchprojects/diasporas/cvp/cultural-value-frameworkExamplefromtheIranSeasonhttps://www.dropbox.com/s/pbdl1xuc3agv22h/%5BMME%5D%20Perception%20Study%20Report%20on%20UK%20Culture%20%20Arts_20161130.pdf?dl=0
• DigitalSentimentAnalysisMonitoringofdigitalplatformsandmediatodeterminereach,tonalityandsentimentbuildingontheworkUniversityofLiverpoolandtheOpenUniversitydidaroundShakespeareLivesdigitalandmediaanalysisandalsotheworkoftheUKIndonesiaSeason.ExamplefromUKIndonesiaSeason:-https://www.dropbox.com/s/km7szneiezsmlod/UK-ID%20Baseline%20Report_Oct2015_Sept2016.pdf?dl=0
• PerceptionSurveysbuildingontheworkofShakespeareLives,UKKoreaSeasonandtheUKUAESeasontargetedperceptionsurveysofsmallgroupsofkeystakeholdersand/orinfluencersyouwanttotracktheirchangesintheperceptionoftheUKbefore,duringandafteraSeasonorFestivalforexample.https://www.dropbox.com/s/n2a6xt4ewhd8kpl/ICC%282017%29SLReport-Appendices.pdf?dl=0andUKKoreaSeasonPerceptionSurveyBaselineReport2017https://www.dropbox.com/s/pbdl1xuc3agv22h/%5BMME%5D%20Perception%20Study%20Report%20on%20UK%20Culture%20%20Arts_20161130.pdf?dl=0
• SurveyparticipatingartsandculturalorganisationstohelpbuildtheevidenceforValuetotheUKofyourprogramme.ExamplefromShakespeareLives:-https://www.surveymonkey.co.uk/r/6TNX8JG
• MediaAnalysis–buildingontheworkandapproachoftheICCforShakespeareLivesandLiverpoolEuropeanCapitalofCulturemeasuringmediaimpactshttps://www.dropbox.com/s/jct04mbck6052dw/Media%20Toolkit%20%281%29.pdf?dl=0andalsohttps://www.dropbox.com/s/n2a6xt4ewhd8kpl/ICC%282017%29SLReport-Appendices.pdf?dl=0
Furtherreadingandsupport
TheUniversityofWestScotlandReportintohowwemightevaluateartsandsoftpowerisonlinehere:-https://www.dropbox.com/s/y0xzy7xsqhzyly8/Final%20report%20for%20British%20Council%2031032017%20%28march%202017%29.docx?dl=0
ApaperonourdevelopingBritishCouncilapproachtoevaluatingartsandsoftpowerisonlinehere:-https://www.dropbox.com/s/n2a6xt4ewhd8kpl/ICC%282017%29SLReport-Appendices.pdf?dl=0
www.britishcouncil.org
15
ConnectingGroundShakespeareLivesandPerceptionsoftheUKinRussia,ChinaandtheHornofAfrica,UniversityofLiverpoolhttps://www.dropbox.com/s/wuz1x43i4gona5z/ICC%282017%29SLReport-Final.pdf?dl=0
UKRussiaYearofCultureMediaAnalysisReporthttps://www.dropbox.com/s/m2x4fyuru7ilpne/Russia%20UK%20Yr%20of%20Lit%20%26%20Lang%20Media%20Coverage%20Analysis.pdf?dl=0
Audiences
• FamilyArtsEvaluationandAudienceResearchToolkithttp://www.familyarts.co.uk/wp-content/uploads/2015/05/Family-Arts-Festival-Evaluation-Toolkit.pdf
• Researchingaudiencesatoutdooreventsandfestivalshttp://culturehive.co.uk/resources/researching-audiences-at-outdoor-events-and-festivals
• CapturingtheAudienceExperience-Theaimofthishandbookistohelptheatreorganisationsunderstandhowaudiencesexperiencetheirwork.Itpresentsanewmodelfordescribingtheaudienceexperience,alongwithstandardsurveytemplatesandguidanceonhowtousetheminthecontextofaprojectevaluation.https://www.itc-arts-s3.studiocoucou.com/uploads/helpsheet_attachment/file/23/Theatre_handbook.pdf
• CulturalEventsImpactEvaluationResourceshttp://www.eventimpacts.com/• Measuringlargescaleartseventsaudiencesnumbers
https://www.dropbox.com/s/brrkaxtu8qf2o7o/attendancetoolkit.pdf?dl=0
Communicatingyourevaluationfindings
• GuidancenoteonwritinganArtsEvaluationReporthttps://www.dropbox.com/s/6d7uzkx5noz6as1/Arts%20Evaluation%20Report%20Sample%20Structure.docx?dl=0
• BritishCouncilArtsImpactsTemplateworkingwithyourCommunicationsLeadPerson,ashortPowerPointonusingtheArtsImpactTemplateisonlinehere:-https://www.dropbox.com/s/0w4ao9cf4ythfvy/ArtsImpactTemplate_Presentation.pptx?dl=0
• ArtsCouncilEngland’sAdvocacyToolkithttp://www.artscouncil.org.uk/why-culture-matters/making-case
• Howtocommunicateyourresultsguidehttp://www.thinknpc.org/publications/talking-about-results/
• UNICEFEvaluationToolkitonmonitoringAdvocacyandCommunicationshttp://www.unicef.org/evaluation/files/Advocacy_Toolkit_Companion.pdf
• LexisNexisisanews-trackingservicethatoffersoneoftheworld’slargestsearchabledatabasesofcontentfromnational,state,andlocalprintandbroadcastmedia.Contentanalysisthenhastobedoneonthemediaarticles.http://www.lexisnexis.com/en-us/products/nexis.page
www.britishcouncil.org
16
• NFPCharityParliamentaryTrackerMonitorsquarterlywhichcharitiesandcampaignshavebeennoticedbyMPsandhowMPsratetheeffectivenessoftheircampaigns.http://nfpsynergy.net/nfpsynergy-monitors/charity-parliamentary-monitor
Upload your evaluation reports & Arts Impacts Templates to our British Council Arts Dropbox https://www.dropbox.com/request/7bhWlphzwaPWpcMYdwN9?oref=e
After Action Reviews
TheAfterActionReview(AAR)isashortfocusedmeeting,fortheteam,conductedbytheteam,lastinghalfanhourorless.AARsallowtoyoucaptureusefuloperationalknowledgewhichisofimmediateshorttermbenefit,andwhichcanbeploughedbackintothenextshift,orthenextday'soperation.Thisallowsyoutomakecoursecorrectionsduringactivitybasedonwhatyoulearn,itallowsyoutoaddressandoptimisethewayyouworkasateam,anditallowsyoutostarttobuildyourcollectiveoperationalknowledge.Theprocessconsistsofaskingfourquestions:
• Whatwassupposedtohappenintheactivitywearereviewing?• Whatactuallyhappened?• Whatwerethepositive/negativethingsthatcontributedtotheresult?• Whathavewelearnedfromthis?
Furtherguidanceondoinganafterprojectactionreviewcanbefoundonlinehere:-https://www.dropbox.com/s/7uiqd3c8a6fqcos/After%20Action%20Review%20for%20Projects.rtf?dl=0
Triangulation
Animportantconsiderationwhenconductingresearchandevaluationiswhetheryoucanrelyonyourdataasbeingatrueportrayaloftheviewsorpositionofyourpopulationofinterest.Itisimportanttobesurethatthefindingsbeingpresentedhavenotbeenunintentionallyskewedbecauseofhowthequestionswereasked,orsomethinguniqueabouttherespondentsthatyoumayhaveoverlooked.
Triangulationisaboutusingarangeoftechniquestoprovideinsightintoatopicthatisparticularlyusefulasawayofcross-checkingtheresearchfindings.Itisparticularlyusefulasthereisoftenneveroneidealwaytoconductresearch;therearemanywaystofindinformation,andeachmethodhasitsadvantagesanddisadvantages.
Triangulationisalsoreferredtoasamixedmethodsapproachtoresearch.Itenablesyoutovalidatedatathroughcross-verificationfrommorethantwoevidencesources.Byusingtriangulationyouaretestingtheconsistencyofthefindingsandincreasingyourchanceofcontrollinganythreatstothevalidityoftheresults.
www.britishcouncil.org
17
Judgingstrengthofevidenceandextentofcontribution‘Strengthofevidence’relatestotheinternalvalidityoftheevaluationfindings.
• Theextentoftriangulationacrossstakeholders,participants/non-participants,and/ordatasources.
• Aconsiderationoftheposition,knowledge,analyticalcapacity,reflexivity,andpotentialbiasesofprimaryinformants–recognisingthatindividualsarepositionedinuniquewaysinrelationtotheprogramme,withdifferentlevelsofknowledgeandcapacity/willingnesstoanalyseandreflect,aswellasdifferentincentivesthatmayleadtobias.
• Aconsiderationofthebroadercontext–helpingtoensurethatexplanationsofchangearegroundedinanunderstandingofthepoliticalcontextandarenotover-reliantontheexplanationsofprogrammeparticipants
Sampling
Inlargescalestudiestherearevariousmethodsforrandomsamplingthatarebestadvisedbyexperiencedresearchers.Onetechniqueisquotasampling,whereevaluatorsidentifyspecifiednumbersofpeopleaccordingtopredefinedcharacteristics,suchasageandsex,inordertomakeupatotalsamplethatreflectsthepopulation.Thiscanworkwellinlargerprojectswhereevaluatorshaveagoodlevelofknowledgeaboutprojectparticipants.Thefollowingchecklistshouldhelpyouconsidertheykeyissueswhenreviewingthequalityoftheinformationthatyouhavegatheredforevaluationpurposes.
• Samplingandrecruitmentofparticipants:doyourparticipantsrepresentthevarietyofcharacteristics,rolesandviewspresentinyourpopulation?
• Responseratesandmissingdata:haveyoutakenappropriateactiontoensureahealthyresponserateandthatmissingdataarekepttoaminimum?
• Appropriatemethod:haveyouchosenamethodthatisappropriatetothecontextandparticipantsandthatisfeasibleforyoutodeliverwithintheproject’stimelineandbudget?
• Bias:haveyouconsideredwherebiascouldbeintroducedandtakenactiontominimisethis?
• Ethicalconsiderations:haveyougainedtheappropriateconsentfromyourparticipants?
Goodpracticesamplingguidehttps://www.theaudienceagency.org/insight/good-practice-guide-to-samplingTocalculateyoursamplesizemoreaccurately,thereareonlinesamplesizecalculators.https://www.surveysystem.com/sscalc.htm If your project is ODA funded? MonitoringandEvaluationaroundODAfundedprojectswillneedtocapturetheeconomicandsocialdevelopmentalimpactofourwork.M&Eshouldalign,testandreflecttheoriginalbusinesscaseandtheoryofchange/logframe.
AsBritishCouncilandFCOhavesigneduptotheinternationalaidtransparencyinitiative,wehavetopublishallourODAexpenditureandachievementsontheFCO/IATIwebsiteforpublicscrutinyevery
www.britishcouncil.org
18
quarter.ThereportlistseachindividualODAprojectbycountry,ashortdescription,expenditureprofileandprojectachievements.Itisupdatedeveryquarter.Itisimperativethattheinformationinthisreportisaccurateandthatsupportingdocumentationisavailableinprojectteamsasevidencetosupportanyquestionsthatarisefromanypublicorprivatebody.Strategy,PerformanceandInsightteamhavethereporttemplateandproducethereportwiththesupportfromcorporatefinance.
ForanyactivitylinkedtoOverseasDevelopmentAssistance(ODA)spend,wehavealegalrequirementtocomplywiththeUKgovernment’sInternationalDevelopment(GenderEquality)Actandtodemonstratethatourprogrammesseektoreducegenderinequalities.
TheBritishCouncil’sGenderToolkitisonlinehere:http://intranet.britishcouncil.org/uk/sites/ES/SO%20%20Programming%20Tools%20and%20Resources/Gender%20Toolkit.docx
Value for Money Approach
ThebroadprinciplesforhowUKgovernmentdepartmentsshouldensurevalueformoneyaresetoutinHMTreasurypublications,particularlytheGreenBookandManagingPublicMoneyThekeyrequirementsincludeensuringthat:
• Thereisacleareconomiccaseforeachspendingprogramme.• Interventionsareevidence-based• Risksaremanagedeffectively.• Thereareclearlinesofresponsibilityformanagingtheprogramme.• Costsandresultsareregularlymonitored.• Resultsareevaluatedoncompletion.
DFID’sSmartGuideonvalueformoneyandits4Eframeworks:
• Economy:gettinginputsoftheappropriatequalityattherightcost• Efficiency:turninginputsintooutputsascost-effectivelyaspossible• Effectiveness:theextenttowhichtheoutputsareleadingtothedesiredoutcomes• Equitythedegreetowhichtheresultsareequitablydistributed.
www.britishcouncil.org
19
BritishCouncil’sValueforMoneytoolkit:http://intranet.britishcouncil.org/uk/sites/ES/ADPages/ValueForMoneyToolkit.aspx
www.britishcouncil.org
20
Data sharing – Return on Investment
IfyouaremeasuringReturnonInvestmentasoneofyourevaluationoutcomes,haveyoumadecontactandhavetheagreementofyourlocalDepartmentforInternationalTradeofficefortheirsupportandsharingofdatabetweenthetwoorganisations?HaveyougotinplaceawrittenpartnershipagreementwithDepartmentforInternationalTradeforpermissions,datasharing,withnamedcontacts,timeframesandwhatdataistobeshared?
• BritishCouncilDataSharingCodeofPracticehttp://intranet.britishcouncil.org/Site/IGRM/DataProtection/Documents/Partner_and_Supplier_Data_Protection_Code.pdf
• Furtheradviceondataprotectionisonlineherehttp://intranet.britishcouncil.org/Site/IGRM/DataProtection/Pages/default.aspx
• ArtsCouncilEnglandcommissionedthisguidanceforthebenefitofthesector,bothfundedandnon-funded,sothatorganisationswhoeitherhavetoorwanttosharedatacandosoincompliancewithregulations.https://www.audiencedatasharing.org/
Some points to consider when thinking about your evaluation plan
• Ensurethattheevaluationispragmaticandproportionatetoyourneeds,generatingusefulevidencethatyoucanusetoinformdecisionmakingandusestheappropriateevaluationtoolsandmethodstohelpfindoutwhatyouwanttoknow.Whatworks,forwhom,inwhatrespects,towhatextent,inwhatcontexts,andhow?
• Itisimportanttounderstandyourstartingpoint/baselineinordertosaysomethinghaschanged;youneedtoknowwherethingsstoodbeforeyourproject.
• Itisimportanttoconsiderethicalimplicationsofevaluationactivities,includingprocedurestoprotectthewellbeingandprivacyofparticipants.TheSocialandEconomicResearchCouncilsetsoutsixresearchprincipleshttp://www.esrc.ac.uk/funding/guidance-for-applicants/research-ethics/our-core-principles/andmightbeworthbuildingintoyourevaluationbrief.
• Theviewsofstakeholdersshouldinformevaluationdesign.Consultationwithcommissioners,funders,artsorganisations,staff,projectmanagers,artists,andserviceuserswillidentifyresourcesandsupportsharedunderstandingandagreementaboutevaluationaims,prioritiesandmethods.Itcanhelptoensurethatallstakeholdershaverealisticexpectationsofwhatkindofdatawillbeneededandwhattheevaluationcanachieve.
• Thinkaboutthelongtermimpact/outcomesofyourprojects/programmesaftertheactivitystagehasfinished.Explorefollowingupwithparticipants,stakeholderspostyourprojecttoexplorethelongertermimpactofyourworkandwhathaschangedforthemaftertakingpartinyouractivity?
www.britishcouncil.org
21
• Connectyourcommunicationsplanandyourevaluationplanseamlessly.Gooddata
visualisationwillbringyourresultsalive.Distinguishingevaluationfromadvocacy:evaluationfindingscanbeusedtosupportadvocacybutadvocacyshouldnotdriveevaluation.
• Howwillyourespondtoyourevaluation,thelearningandthelessons,howwillitfeedbackintofutureplanningandfuturedelivery,willyoupublishawrittenresponsetoyourevaluation?
Howshouldcasestudiesbeusedinevaluation? Thereispowerinagoodstory.Youmayhaveusedcasestudystoriesformanyreasons,includingwhenadvocatingforaproject.Personaltestimonialfromparticipantswhohavehadapositiveexperiencecanbeverypersuasive.However,thisisnotwhatismeantby‘casestudy’inevaluation.Foranevaluator,thequalityofacasestudyhingesonthemethodsusedtocollectandanalysethedataandthewritingorpresentationofthestory.Theevaluatorwillselectcasestudiesthatarerelevanttotheoutcomesorissuestheevaluationseekstoexplore.Youcanhaveacasestudyofasingleindividual,groupsofindividuals,anobject,evenaplaceorasituation.Arangeofdifferentkindsofdatamaybepiecedtogetherinacasestudytotellastoryrelevanttothequestionsthatinformtheevaluation.
Itisimportanttorememberthatacasestudystoryisparticulartoanindividualcase;becauseoneparticipantexperiencedsomethingduringaproject,thisdoesnotnecessarilymeanthatallparticipantsdid,orthatanyfutureparticipantswill.Thisiswhyitisvitaltoselectyourcasescarefully,sothattheycangenuinelyhelptoilluminatetheparticularissuesyouareinvestigating.
Whataboutcreativeorarts-basedapproaches?Anevaluatordemonstrates‘creativity’everytimetheyviewanevaluationproblemfromafreshperspective,ordevisesanevaluationapproachtailoredtoaparticularcontext.Whilecreativityisverydefinitelynotjusttheprovinceofartistsorthearts,thereisagrowinginterestintheuseofevaluationmethodsthatusefilmandvisualarts,poetryandcreativewriting,music,dramaandperformingarts.
Arts-basedmethodscanbeparticularlypowerfulinuncoveringhiddenperspectivesandinempoweringparticipants.Theymayalsobelessintrusivethanmoreclinically-basedevaluationtoolsastheycanbeinspiredbyandmodelledontheinterventionitself.Theyalsoinvolveanumberofchallenges.Forexample,theresultsyougetfromthem(pictures,performancesorpoemsforexample)arebynaturedifficulttointerpretandyoumayrequiretechnicalskillsthatarenotapartofstandardevaluation.Itisparticularlyimportanttomakesurethatanycreativeevaluationapproachyouuse‘fits’theprojectandthepeoplewhoaretakingpartintheevaluation.Creativeandarts-basedevaluationmethodshttp://creativeandcredible.co.uk/wp-content/uploads/2015/07/CreativeCredible_Creative-and-arts-based-evaluation-methods.pdf
Isinternalorexternalevaluationbest? Externalevaluationcanappearmorecredible,avoidingissuesaroundareaswherebiasmaycreepin.
www.britishcouncil.org
22
Anexternalevaluatorprovidesspecialistskillsandknowledgeandmaybeabletodisseminatefindingsmorewidely.However,theevaluationis‘oneoff’andthatlearningmightnotbeembeddedwithintheprojectteam.Inreality,manysuccessfulprojectsdevelopbyusinganiterativeprocessinvolvingbothinternalandexternalevaluation.Hybridmethods(internalandexternalteammembers)canbeemployed,e.g.,self-evaluationbyinternalstaffintegratedintoanexternalevaluation;externalfacilitatorforaninternalevaluation).
Team meetings / training evaluation resources
• ArtsEvaluationPowerPointpresentationhttps://www.dropbox.com/s/2tllqny1yf91wdg/evaluation%20presentation.pptx?dl=0
• ExampleArtsEvaluationPlanhttps://www.dropbox.com/s/6rogbfoxn4yv3h7/example%20evaluation%20plan.docx?dl=0
• SouthAsiaArtsEvaluationWebinar(1.5hours)inWRFfileformatcanbedownloadedinfull(14mb)here:-http://bit.ly/2aVHZSS
• YoucandownloadtheWebExPlayerforWindowsforfreeheretoplaytheWebinarhttps://welcome.webex.com/client/wbxclient/atrecply.msi
• Podcast(5minutes)ontheAHRCCulturalValueProjectwithProfessorGeoffCrossickisonlinehere:-https://www.dropbox.com/s/1j2vqjgxxpxgd2k/Cultural-Value-podcast.mp3?dl=0
• Podcast(50minutes)ProfessorGeoffCrossickUnderstandingtheValueofArtsandCulturehttps://soundcloud.com/tlrhub/understanding-the-value-of-arts-and-culture?in=tlrhub/sets/trinity-long-room-hub-top-ten
• ArtsImpactTemplatePresentationhttps://www.dropbox.com/s/sz93jg8txepefjr/ArtsImpactTemplate_Presentation.pptx?dl=0
Some Evaluation Terminology
Aims(oftenlinkedtoobjectives)
Whyyouaredoingit,whatyourprojectisfor.Ausefulanalogyistothinkofyourprojectaimsasthedestinationofyourprojectjourney(eg.Wewantourbuildingtobeaccessible).
Criticalsuccessfactor Whyyourproject,ordifferentpartsofit,havesucceeded.
Evidence Evidencesimplymeanstangiblethingsthatyoucanusetoproveordemonstratethatyou’vemadeprogress.Evidenceisaboutprovingyou’vemadeprogress,havehadanimpact.
Thekindsofevidenceyouwanttocollectwilldependonyouraimsandobjectives,andalsotheindicatorsyouthinkwillbestshowyou’vemade
www.britishcouncil.org
23
progress.
Qualitativeevidence Qualitativeevidencetendstorelateto,andreflect,differentpeople’sthoughtsandfeelings.Forexample,shiftsinparticipants’confidencelevels,ortheirviewsonimprovementsyou’vemade.Itisoftenviewedas‘anecdotal’ratherthan‘factual’evidence,becauseitisbasedonindividuals’feelingsandjudgementsaboutwhathashappenedasaresultofyourproject.
Quantitativeevidence Oftenreferredtoas‘hard’evidence,becauseittendstobenumbers-related(eg.numberofparticipants,increaseinincome)andisviewedasbeingmorefactual.Itincludesnumbersofpeoplewhothinkacertainthing(eg.79%ofparticipantsviewedourtrainingasexcellent).
(KeyPerformance)Indicators(sometimesknownasKPIsormeasuresofsuccess)
Attheirsimplest,indicatorsarethingsthatshowyou’vemadeprogress.Theyarehowyouwillknowyou’remakingprogress,makingadifferent.Forexample,ifyourprojectaimistoimproveaccesstoaheritagebuilding,yourindicatorsmightinclude‘morepeoplevisiting’ora‘morediverserangeofvisitors’.
Limitingfactor Alimitingfactorissimplysomethingthathaslimitedyourprogressorimpact.Itmightalsobecalleda‘challenge’.Forexample,yourplanstoimproveaccesstoyourbuildingmightbehamperedbyyounotreallyhavingunderstoodtheaccessneedsofawideenoughrangeofvisitors.
Methodology Methodologysimplymeans‘howwewilldoit’.So,inevaluationterms,itmeanshowyouwillevaluateyourproject:howandwhenyouwillcollecttheevidenceandhowyouwilldoit,alongwithhowyouwillthenreviewthatevidenceandcreateandshareyourstoryaboutyourproject.
Objective Objectivesarespecificthingsthatyouwanttoachievethroughyourproject.Forexample,ifyouraimistoimproveaccesstoyourbuilding,yourobjectivesmightincludemakingthebuildingmorephysicallyaccessibleoradaptingyourwebsite.
Youmightsometimeshearpeoplereferto‘SMART’objectives.Thisisanacronym;itmeansobjectivesthatare:
Specific,Measurable,Achievable,Realistic&Timetabled
Outcome(sometimesknownasan‘indicator)
Anoutcomeisanotherwordforwhatwillbedifferent(foreveryoneinvolved,includingyou)ifyousucceed.It’sthe‘shift’orchangeyouwillseeifyouhavesucceeded.
Unanticipatedoutcome Thisissimplyachangethathashappened,thatyoudidn’tpredictor
www.britishcouncil.org
24
expect.
Legacyoutcome Thisissimplyachangethathappensfurtherintothefuture,asopposedtoduringthelifespanofyourproject.
Output Outputscommonlyrelatetonumbers.Anoutputisalsoatangiblethingwhichhappenedduringyourproject.Forexample,ifyourprojectisaboutmakingyourbuildingmoreaccessible,thenoutputsmightincludethephysicalchangesyouhavemade,andthenumberoftrainingsessionsyouhaveprovidedforstaff.
Stakeholder Stakeholdersarethepeopleororganisationswhoareinvolvedinyourproject,orwhohavea‘stake’init.Thesemightinclude:yourteam,volunteersortrustees,yourvisitorsorusersandyourfunders.