npc, measuring impact in the voluntary sector 2016
TRANSCRIPT
![Page 1: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/1.jpg)
vMEASURING SOCIAL IMPACTCecilie Hestbaek, NPCThe Big Connect, Tuesday 7th June 2015
![Page 2: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/2.jpg)
NPC: TRANSFORMING THE CHARITY SECTOR
2
NPC works at the nexus between charities and
funders
Charity
SectorFunder
Increasing the impact of charitieseg, impact-focused theories of change
Strengthening the partnership
Eg, collaboration towards shared
goals
Increasing the impact of funders
eg, effective commissioning
Consultancy & Think tank
![Page 3: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/3.jpg)
3
SOCIAL IMPACT
‘The effect of an activity on the social fabric of the community and well-being of the individuals and families.’
http://www.businessdictionary.com
![Page 4: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/4.jpg)
a) articulate how you think a service or intervention will have an impact;
b) and test your thinking against the evidence you have available.
4
THE ESSENCE OF IMPACT MEASUREMENT:
Exercise:Take 10 minutes to discuss with your neighbour why it’s important to
measure social impact and what the challenges are. Write key words on different colour post-its.
![Page 5: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/5.jpg)
5
WHY IS MEASURING IMPACT IMPORTANT?
Saves staff time
Influences the debate on
“what works”Improved services
Raises profileMotivates
staff / volunteers
Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater Impact. New Philanthropy Capital.
Helps secure funding
![Page 6: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/6.jpg)
6
CHALLENGES IN MEASUREMENT
Causality, consistency and predictability.
• What worked before might not work again because the external environment has changed
Time frame • Change can happen quickly; or it might take two decades to come about
Contribution and attribution
• A variety of external factors are beyond your organisation’s control and you will often have no
counterfactuals
Data collection• If you are targeting top-level decision makers, it can be
difficult to get an answer from them about why they did (or did not) change their mind on a particular issue.
…and ensuring that you measure outcomes, not outputs.
![Page 7: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/7.jpg)
7
THE FOUR PILLARS APPROACH TO MEASURING IMPACT
Map your theory of change
Prioritise what you measure
Choose your level of
evidence
Select your sources and
tools
Effective measurement framework developed
Strategic visionLeadership
Case for impact measurement
![Page 8: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/8.jpg)
DEVELOPING AND USING A THEORY OF CHANGE
![Page 9: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/9.jpg)
9
WHAT IS A THEORY OF CHANGE?
Links activities è intermediate outcomes è final outcomes
A description of how activities lead to outcomes
- Clarifies what the activities aim to achieve and how
- Provides the case for why achieving intermediate outcomes is important
- Provides a structure for identifying what can be measured
![Page 10: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/10.jpg)
10
HOW TO REPRESENT A THEORY OF CHANGE
Planning Triangle Logic Model Outcomes Chain
However you represent your theory of change, it should be supported by a written description.
![Page 11: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/11.jpg)
11
Children’s emotional resourcefulness
improves
Counselling
Clients’ ability to support their children's healthy development improves
Clients’ emotional or psychological
difficulties decrease
Mother / Child interaction improves
Clients’ capacity for self care increases
EXAMPLE (SIMPLIFIED) THEORY OF CHANGEMOTHERS’ COUNSELLING SERVICE
Activities Intermediate Outcomes Final outcome
![Page 12: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/12.jpg)
12
EXERCISE
• Take 2 minutes to think about 1-2 social outcomes that are central to what you do.
• Write them down on a piece of paper.
• As we go through the rest of the presentation, try to reflect on the measurement of that/those outcomes. How would you apply the theory and measure those?
Categories of outcomes include: ‘soft’ outcomes, such as attitudes, knowledge, skills, behaviour; and ‘hard’ outcomes, such as employment,
educational attainment, reduced reoffending etc.
![Page 13: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/13.jpg)
PRIORITISING OUTCOMES
![Page 14: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/14.jpg)
14
PRIORITISE OUTCOMES THAT:
• are directly influenced (rather than indirectly supported)
• are important / material to the mission
• are not too costly to measure
• will produce credible data
![Page 15: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/15.jpg)
CHOOSING THE RIGHT LEVEL OF EVIDENCE
![Page 16: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/16.jpg)
16
COUNTERFACTUAL
• Comparing the world with your organisation in it with what the world would be like without it.
• Control group
• Attribution
![Page 17: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/17.jpg)
17
LEVELS OF EVIDENCE
Randomised control trial
Anecdotes / quotes
Before and after survey
Self-reported change
Case studies
Control groups
Credibility
Basic Advanced
![Page 18: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/18.jpg)
SELECTING DATA SOURCES AND TOOLS
![Page 19: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/19.jpg)
• Quantitative data (numbers)• Statistical estimates• Prevalence of views,
attitudes and experiences• Admin data or questionnaires
(paper, web, etc.)
• Qualitative data (words)• Detailed understanding • In-depth interviewing
(telephone or face-to-face)• Observation• Stakeholders’ views
Proportion of beneficiaries whose outcomes have improved, and by how
much.
What did beneficiaries think, did it make a
difference to them? How? How could it have been
better?
DIFFERENT TYPES OF EVIDENCE
![Page 20: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/20.jpg)
20
EVALUATION TRAPS
Collect data that matters, and work together
Don’t force squares into circles & don’t
collect arbitrary data
![Page 21: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/21.jpg)
21
INSPIRING IMPACT: MEASURING UP! AND THE IMPACT HUBMeasuring Up!
• online, step-by-step self-assessment tool
• looks at the way you plan, evidence, communicate and learn from the difference your work makes
The Inspiring Impact Hub
• pulls together resources relevant to improving impact practice
• enables users to search and filter results according to their needs
inspiringimpact.org
![Page 22: NPC, Measuring Impact in the Voluntary Sector 2016](https://reader035.vdocuments.site/reader035/viewer/2022062900/58d1be1f1a28ab98278b6591/html5/thumbnails/22.jpg)
22
QUESTIONS AND COMMENTS