the views expressed are those of the authors and should not be attributed to the urban institute,...

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Social Impact Bonds / Pay for Success: Pros, Cons, Promises and Risks October 7, 2014 1:00PM ET Kelly Walsh, PhD Justice Policy Center Urban Institute John K. Roman Justice Policy Center Urban Institute Kyle McKay Texas Legislative Budget Board

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Leads a national movement State-based juvenile justice coalitions and organizations Laws, policies and practices that are fair, equitable and developmentally appropriate for all children, youth and families Photo: MorizaMoriza

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Pay for Success: Funding the Infrastructure for Evidence-Based Change Kelly Walsh, PhD Washington, DC

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER New financial instrument to support discretionary social programming o Brings evidence-based programs to the necessary scale Can support programs that o Involve significant start-up costs, but provide long-term savings o Serve large populations o Involve political or programmatic risks PFS addresses these programs with private capital o Offers an investment return if the program meets performance goals o Government only pays for successful outcomes Sometimes called social impact bonds o PFS does not operate like a bond What is Pay for Success? 4

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER An intermediary selects an evidence-based program The program is supported with private capital solicited by the intermediary The intermediary o oversees program implementation and o ensures model fidelity An independent evaluator determines if the program has met performance targets Government pays the intermediary for successful outcome How Does Pay for Success Work? 5

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Intermediary 6 The Structure of Pay for Success

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Drawbacks Requires significant expertise from multiple fields Complicates execution Could limit nonprofit innovation by focusing on programs with a proven track record May reallocate existing impact and philanthropic capital rather than drawing new capital Advantages Transfers risk away from the government Develops or scales local service networks Addresses the wrong pockets problem Allows agencies to pool resources and knowledge Provides significant flexibility to deploy programs across jurisdictions or regions Advantages and Disadvantages 7

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS projects should target justice system inefficiencies (e.g., prison population drivers) Once drivers are identified, PFS is developed in 5 steps 1234 5 Strategic Planning Price the product Make the deal Develop infrastructure Deliver service and TTA Evaluate the program Building PFS in 5 Steps 8

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS in the US 9 The US PFS market: almost $100 million in total investment Youth and the CJS: $ 9.6 million invested in New York City Rikers Island (2012) Adolescent Behavioral Learning Experience (ABLE) Program $27 million invested in Massachusettss Roca, Inc. (2014)

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Pay for Success: The Promise of Scale Finance John K. Roman, PhD Washington, DC

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Strategic Planning 11 123 45 123 4 Identify population and cost drivers Identify target population Find evidence- based solutions Assess PFS Suitability Price the Product Develop Infrastructure Evaluate the Program Make the Deal Deliver Service and TTA Strategic Planning Product Development

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Strategic Planning 12 123 45 123 4 Identify population and cost drivers Identify target population Find evidence- based solutions Assess PFS Suitability Price the Product Develop Infrastructure Evaluate the Program Make the Deal Deliver Service and TTA Strategic Planning Product Development

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS relies on good strategic planning to maximize cost effectiveness Before starting a program, governments should do the following: o Identify population and cost drivers o Identify the problems that generate these drivers o Find evidence-based solutions to the problems o Determine if the evidence-based solutions are PFS-compatible Governments should identify a research partner to assist with strategic planning Justice program databases can inform implementation: o Office of Justice Programs Crime Solutions o Washington State Institute for Public Policy (WSIPP) o Vanderbilt University Peabody Research Institute o Blueprints for Violence Prevention o The Urban Institutes Meta cost-benefit analyses for District of Columbia Crime Policy Institute (DCPI) o National Reentry Resource Centers What Works in Reentry Clearinghouse Strategic Planning 13

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Juvenile justice o Family-based intervention and prevention programs can prevent delinquent conduct and reduce recidivism Implementing these programs requires start-up investment to train local staff Adolescent diversion o Adolescent diversion from secure confinement has been found to have a large cost-benefit ratio Training and project implementation impose significant upfront costs PFS Opportunities in Juvenile Justice 14

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Scale of the Problem 15

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Drivers of costs and populations: states and counties spend $5.7 billion every year taking legal and physical custody of more than 60,000 juveniles adjudicated delinquent Early-intervention programs are under-utilized: Could reduce juvenile incarcerations by 70% At a cost that is 80% less than confinement Fewer than 5% of eligible families benefit from these programs. Potential for enormous cost savings: Family-Based therapies cost less than $15,000 per family Compared to an average out-of-home placement cost of $88,000 per youth per year. Strategic Planning Likely Results From Steven H. Goldberg, Caffeinated Capital 16

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Florida Redirection was managed by Evidence- Based Associates (EBA). From 2004-2013: Florida paid EBA a total of $65.5 million to manage the delivery of three intensive family therapy programs: Multisystemic Systemic Therapy (MST), Functional Family Therapy (FFT), and Brief Strategic Family Therapy (BSFT) More than 10,000 families in 18 of the states 20 judicial districts we served. Source: Florida Redirection as a National Model for Scaled Implementation 17

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER How Cost-Effective are MST Family- Based Therapies? 18 Source: Washington State Institute for Public Policy

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS Opportunities in Juvenile Justice 19 Source: National Center for Justice Planning

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS Opportunities in Juvenile Justice: from Caffeinated Capital 20 Source: Social Finance, UK

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS Opportunities in Juvenile Justice 21 Source: National Center for Justice Planning

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER PFS efficacy depends on maintaining a research emphasis Broad adoption of PFS will be a learning process o Governments, for-profits, and nonprofits learn how to collaborate o PFS offers a new instrument for sharing resources and benefits Future of PFS 22

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Comments? Questions? Email: [email protected] Twitter: @JohnKRoman

Social Impact Bonds: The cost of complexity Kyle McKay Presentation to National Juvenile Justice Network, October 7, 2014 All opinions expressed below are solely the authors and should not be attributed to any of the individuals or organizations with which McKay is associated.

How complex are social impact bonds? The idea of paying for success sounds simple. However, even basic performance-based contracts without investors can become unwieldy. Social impact bonds (SIBs) are particularly complex. There is uncertainty arising from defining outcomes, estimating the potential impact of interventions, measuring and attributing change, valuing benefits, demonstrating a fiscal return and getting interventions to scale. The weakness of the chain arises not from its weakest link but from the sum total of the weakness of every link. Fox and Albertson, Journal of Criminology and Criminal Justice

How complex are social impact bonds? Complexity in some instances meant that the actual transfer of risk is not clear RAND Europe

How complex are social impact bonds? Source: RAND Europe In Peterborough, key staff who deliver services to cohort members include: 6 full-time caseworkers provided by St. Giles Trust Fifty lay volunteers recruited, trained and managed by Sova. Up to six St Giles Trust volunteers (individuals who may be currently on probation or have personal experience of the criminal justice system). Two, part-time family specialist practitioners from Ormiston Children and Families Trust. One part-time recovery worker from Mind. One trainer from John Laing Training who delivers a construction skills course.

Can SIB borrowing grow the pie? Commercial investors likely to seek secured source of income State law and fiscal policy can create non-appropriation risks Appropriations for payments were made in advance of any potential payments in: Massachusetts Utah Peterborough In NYC, Bloomberg Philanthropies assumed the liability of the government by providing $7.2 million to be held by MDRC in a guarantee fund to back the loan

Can SIB borrowing grow the pie? Legislative budget offices: California Legislative Office analysis of Assembly Bill 1920 found Unknown, major future cost pressure potentially in the millions of dollars to fund [SIB] contracts Maryland Department of Legislative Services fiscal note on House Bill 517: It is assumed that the state will at least initially have to provide upfront funding for some or all of the SIB contract costs and general fund expenditures may increase. NJ fiscal note on Assembly 3289 If there are no savings realized by the departments and no repayments made to the lenders, the authority may be responsible for the full $15,000,000

Can SIB program outcomes grow the pie? In Peterborough, payment amounts were set on the basis that the SIB was innovative. RAND Europe The Peterborough SIB is too small to deliver substantial cashable savings (monetised benefits) RAND Europe There is no guarantee that a reduction in re-offending will reduce demand for prison places. Fox and Albertson, Journal of Criminology and Criminal Justice This tool is a more expensive way to scale programs than if government simply contracted directly with a service provider. McKinsey & Company

What kind of innovation can we expect from investor selected program? Profit-seeking investors will be most interested in social programs or models that are proven. MDRC Staff Investors appear to be sticking to models that have already been extensively evaluated. CLASP Surveys of impact investors: Lack of successful investments was the most commonly cited challenge to impact investment 2011 JP Morgan Survey 62% of respondents would sacrifice financial returns for greater impact, but 60% thought this was not necessary 2011 JP Morgan Survey 65 percent principally target market rate returns. There was also a clear client interest in risk-mitigating features. 2013 JP Morgan Survey

How confident can we be in the results? Interventions have to be delivered within systems... When a program achieves positive results, the success may be attributable to a wide range of factors, and it is often difficult to identify exactly why the program worked. MDRC staff The political imperative to demonstrate the success of this new financing scheme can create incentives for weaker evaluation designs that are more likely to show positive results, but that are spurious. MDRC staff

How confident can we be in the results? Overall, the results suggested that those released from HMP Peterborough were of lower risk of reoffending than those released from other prisons. QinetiQ, University of Leicester It is debatable whether the use of PSM in the current context violates the CIA assumption. This is because data held on the Police National Computer (PNC) are not very useful in controlling for selection of being released from HMP Peterborough. QinetiQ, University of Leicester The Peterborough SIB does not have a defined intervention model or theory of change. While this was perceived to facilitate individualised services, it poses some challenges for evaluation. RAND Europe Evaluations always have caveats. Value of research accrues over time with additional evidence.

Summary There is currently no conclusive evidence substantiating the success of this model. Austin/Travis County Reentry Roundtable, Feb 2014 Extensive evidence in history of performance-based contracting suggests a number of significant challenges and risks. Governments and stakeholders should carefully evaluate not only the intended benefits but the risks and opportunity cost of the model before starting a pilot. Comparison should be between direct financing and operations via public agencies or contracts, rather than no intervention. SIBs are a means to implementing a program and should be evaluated compared to other means of implementation. Important to remember that no program is a policy unto itself.

Resources Social Impact Bonds: Overview and Considerations, Elizabeth Lower-Basch, CLASPCLASP Massachusetts Contracts & SIB RFIs maintained by the Harvard Social Impact Bond LabHarvard Social Impact Bond Lab Evaluating Social Impact Bonds as a New Reentry Financing Mechanism: A Case Study on Reentry Programming in Maryland, Maryland Department of Legislative Services Maryland Department of Legislative Services Payment by results and social impact bonds in the criminal justice sector: New challenges for the concept of evidence-based policy? Chris Fox and Kevin Albertson, Journal of Criminology and Criminal JusticeJournal of Criminology and Criminal Justice

The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. JUSTICE POLICY CENTER Kelly Walsh, PhD [email protected] John K. Roman [email protected] Kyle McKay [email protected] Keshia Cheeks [email protected] Photo: MorizaMoriza

the views expressed are those of the authors and should not be attributed to the urban institute,...

Documents