how to use and produce evidence to inform policy
TRANSCRIPT
What Works Centre for Local
Economic Growth
10th April 2015
The Partners
What Works Centre work programme
• Identify key policy interventions with sufficient evidence to properly assess effectiveness
– Employment training– Business advice and mentoring– Sports & cultural events & facilities– Access to Finance– Estate Renewal– Broadband– R&D (collaboration; grants; tax incentives)– Transport (by mode)– Apprenticeships– Enterprise Zones
• Undertake systematic ranking reviews (with initial focus on robust quantitative evidence)• Assure the findings are accessible to our users and disseminate them through our networks
Work programme
Our remit
Methodology
How to use this evidence
How to evaluate
Using and producing evidence to inform policy
Katrina Hann, Head of ResearchRichard Waggott, Principal Researcher
10th April 2015
About New Economy
• Research, strategy and policy for Greater Manchester
• Core Cities research and development hub
• Partnership with What Works Centre for Local
Economic Growth
Agenda
• What do we mean by evidence-based policy?
• Uses of evidence in the development of policy
• Good practice in developing evidence led policy
• Discussion and questions
‘This Government expects more of policy makers. More new ideas, more willingness to question inherited ways of doing things, better use of evidence and research in policy making and better focus on policies that will deliver long term goals’
Modernising Government, 1999
‘in the absence of perfect evidence, it is still essential that policy makers gather the most useful information available in the short term, as they—and we—move toward an evidence-driven future’
US National Economic Council
‘As we continue to improve and reform public services at a time when finances are tight, it is more important than ever that taxpayers’ money is spent in the most effective way, helping to ensure that policy decisions and professional practice are based upon robust evidence of what works to deliver value for money and better outcomes for all’
What Works Network Year 1 review
Evidence based policy making
Evidence base
What works
Copy, purchase, commission, learn from
What might work
Test, pilot, adapt, borrow, rethink
What doesn’t work
Avoid, stop, challenge,
decommission
Why we should engage with evidence
Current practice
based on…
Habit Politics Procurement
Vested interestsExpediency Cost
Why we don’t engage with evidence
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Case study: Public expenditure analysis
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Case Study: Complex Dependency
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Contextual conditions
Key policy conditions
Rationale Inputs Activities
Intended Outcomes
Outputs
Intended Impacts
Programme Objectives
Project:
Contextual conditionsOpportunity to secure the presence of the life sciences sector around the Alderley Park site and across the wider area.
Key policy conditionsJoint objective of Greater Manchester and Cheshire and Warrington LEPs to secure the long term sustainability of the site
RationaleSmall life sciences firms need incubation funding and follow-on venture capital to maximise growth
InputsLGF fundingPrivate sector match funding
ActivitiesIncubation fund establishedVenture Capital fund establishedRedevelopment of Alderley Park
Intended Outcomes• Jobs
created/safeguarded at Alderley Park
• GVA created• Staff employed by
recipients of fundOutputsNumber. of:Incubation fund investmentsSpin-outs accommodatedVC fund investmentsLoans repaid
Intended Impacts• Growth in local life
sciences sector and supply chain
• Increased prosperity of the GM and Cheshire area
Programme Objectives• Ensure the long term future of the life sciences sector on the
Alderley Park site• Creation and safeguarding of businesses and jobs
Project: Life Sciences Fund
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Why adopt a CBA approach?
• Localities can use Cost Benefit Analysis to deliver:
• Informed decision-making
• Cost-effective decision-making
• Equitable decision-making
It also hardwires the use and development of robust-evidence into decision-making
• Fiscal• Social• Economic
Costs
Benefits
All additional costs needed to deliver project
Fiscal Economic Social
Case Study: Cost Benefit Analysis
LocalAuthority NHS
DWP (AME)Police
WorkProgramme
PrimesRSLs
CJS (exclPolice)
Proactive Costs
Reactive Cost Savings
£-
£100,000
£200,000
£300,000
£400,000
£500,000
£600,000
£700,000
£800,000
£900,000
£1,000,000
£
Case Study: Family Intervention Project
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Evidence Based Policy Development Process
Define high level problem, outcomes and spend
Define the cohort
Design New Delivery Model
CBA using modelled assumptions
Test at a scale commensurate with the risk
Evaluation to collect actual data
Track actuals to inform reinvestment
Good Practice in Developing Evidence Led Policy1. Create demand before creating evidence
2. Evidence should be used to drive strategic priorities rather than solely to comply with externally imposed requirements
3. Development of evidence is a key tool for stakeholder engagement
4. Internal and external challenge is central to establishing if evidence is ‘good enough’
5. Care should be taken with the presentation of evidence to avoid misinterpretation
6. There should be a willingness to take on board negative findings to prevent evidence production becoming a box ticking exercise
7. Evidence needs to be mobilised and acted upon not only produced
8. Using evidence is a process not a one-off
Next steps
• Linking What Works outputs with CBA model
• Greater Manchester case study launched in June
• More events across country to disseminate findings
