evening seminar transforming the lives of troubled families · evening seminar transforming the...
TRANSCRIPT
Evening Seminar
Transforming the lives of troubled familiesManchester’s Evaluation Approach
Paul HolmeResearch Advisor, Manchester City Council
6 October, 2014 Corporate Research & Intelligence Chief Executives
Why Evaluate?Ability to make evidence based decisions
Better understand need – directing resources
Maximising investment potential
Avoiding costly mistakes
Providing confidence to partners
Delivering things that work
Building the evidence base
Evaluating Turnaround Slide 2
Epidemiology?“the branch of medicine which deals with the incidence, distribution, and possible control of diseases and other factors relating to health”
Or in the troubled families context… Incidence - What leads to the demand?
Distribution - What are the common factors?
People, Places, Services
Control - What are the right responses?
Evaluating Turnaround Slide 3
More simply
Slide 4
What data are we collecting?
Troubled Families / Complex dependency
Early Years / Improving School Readiness
LLLB / Integrated Health & Social Care
Transforming Rehabilitation
Work & Skills / Working Well
Evaluating Turnaround Slide 5
Troubled Families Evaluation
Randomised Control Trial
480 families randomised into NDM and BAU Testing impact of integration, sequencing & prioritisation
Evaluation Outcome Framework Developed with partners Tested in RCT Applied to city-wide roll-out
Cost Benefit Analysis Testing the business case
Challenge from Partners Regular review by MIB and LIT
Evaluating Turnaround Slide 6
Testing Integration
Evaluating Turnaround Slide 7
Business as UsualBusiness as Usual New Delivery ModelNew Delivery Model
Evaluation needs to: Track from the point of referralMeasure the situation before interventionMaintain model fidelity
Testing Sequencing
Evaluating Turnaround Slide 8
Business as UsualBusiness as Usual New Delivery ModelNew Delivery Model
Evaluation needs to: Track & report the progress of cases Record actions / responses / learningMaintain model fidelity
Testing Prioritisation
Evaluating Turnaround Slide 9
New Delivery ModelNew Delivery ModelBusiness as UsualBusiness as Usual
Evaluation needs to:Monitor waiting lists – measure non‐service Track additional support & costsMaintain model fidelity
Case for
Investment
Case for
Investment
How we test?
Evaluating Turnaround Slide 10
Evaluating Turnaround Slide 11
Objective Data Sources Tracking Individuals
What do we know?A lot about families!
The common presenting needs
The best services for these families
How long we have to work with a family
The average cost of working with that family
The likely outcomes to be achieved compared to Business as Usual
Evaluating Turnaround Slide 12
Example 1 – the RCT
Evaluating Turnaround Slide 13
13
Impact on Families
Debt
‐
reduced by 19%
Safeguarding
down by
c.17%
ASB
down by 7%
Crime
down by 28%
Absence
down 20%
Exclusions
down 22%
Mental health down 14%
Worklessness
down 2%
Example 2 – the stock & flow
Evaluating Turnaround Slide 14
In FlowIn Flow
The StockThe StockOut FlowOut Flow
Feedback learning to improve identification, targeting & outcomes
Feedback learning to improve identification, targeting & outcomes
Example 3 – the network
Evaluating Turnaround Slide 15
The Nucleus?The Nucleus?
FAMILY A
FAMILY B
TFU
TFU
What do we think we know?Everyone measures success differently!
Evaluating Turnaround Slide 16
Biggest calls on police time come from ‘Concern for Safety’ incidents
Biggest calls on police time come from ‘Concern for Safety’ incidents
Is a reduction in offending on its own an investable option for GMP?
Is a reduction in offending on its own an investable option for GMP?
Outcomes need to work for all potential investment partners
Outcomes need to work for all potential investment partners
What don’t we know?Longer term impact & outcomes!
Statistical accuracy of RCT will improve now that target cohort has been reached
Need to investigate cause and effect
Need to attribute benefits to interventions
Need to track sustainability of outcomes
How priorities may change – ensuring that the evaluation is able to adapt when required
Evaluating Turnaround Slide 17
Thank you
More information is available at:http://www.manchesterpartnership.org.uk/info/40/complex_families
Paul [email protected]
Evaluating Turnaround Slide 18
Questions ?