predictive models for health and social care: a feasability study

© Nuffield Trust

Predictive Models for Health and Social Care: A Feasibility Study

Authors: Bardsley M, Billings J, Dixon J, Georghiou T, Lewis GH, Steventon A (2011) ‘Predicting who will use intensive social care: case finding tools based on linked health and social care data’, Age and Ageing 40(2): 265-270

February 2011

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Background

Key points

Evidence that admission to a nursing home can be delayed or avoided by means of preventative ‘upstream’ interventionsIncreasingly, public services will have to become more proactive in identifying and managing risk in older people, in order to mitigate them as much as possibleIt will be important to identify and support not just those at highest risk of these costs, but also those currently at lower risk who might become higher risk in futureCurrent methods of assessing risk largely rely on face to face assessments. In health care this approach has been shown to be less accurate at predicting predicting future events (hospitalisation) compared to statistical models.


Predictive modelling

• BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher-quality healthcare than the NHS at a lower cost. Kaiser identify high risk people in their population and manage them intensively to avoid admissions

• Modelling aims to identify people at risk of high costs in future• Use pseudonymous, person-level data• In health sector a number of predictive models are available

e.g. PARR++ and the combined model.*Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143


Predictive Models Identify who will be where on next year’s Kaiser Pyramid


Patterns in routine data to identify high-risk people next year


Patterns in routine data to identify high-risk people


Distribution of Future Utilisation is Exponential


Can we predict costly social care events in the same way?

• Do the data exist in local systems?• Can we extract individual, person-level data?• Can we link a wide range of health and social care data at the

person level?• Are the data accurate and complete enough to use?• Can we build a valid statistical model?


Feasibility Project

• Funded by Department of Health Care Services Efficiency Delivery (CSED) programme over 18 months

• Worked with 5 sites to extract data sets• Extracted person-level health & social care data sets • Linked data from GP records (2 sites); GP register (all sites);

Hospital (all sites); Social Care (all sites)• Undertook exploratory analyses and ran a range of models• Tested the impacts of different data sets on the models


Using health and social care data to predict health and social care use


Using routine data

• Less labour intensive so they can stratify the population systematically and repeatedly

• Avoid “non-response bias”• Can identify people with lower, emerging, risk• Important issues of confidentiality and consent to consider • Linking data sources at individual level across health and

social care is problematic where there is no NHS number in social care

• The tools are never 100% accurate• Data may be missing from routine databases on certain

groups


Predictive factors – examples


Information flows


Data collected

• From four ‘sites’ (~ PCT areas)• Total seven organisations: 3 PCTs, 1 Care trust, 3 LAs • Total 1.4M pop (range 100,000-700,000)


Data linkage - approach

First instance: NHS number (encrypted) from LA

In absence of NHS number:• Central ‘batch tracing’?• Shared PCT/LA databases?

Ultimately: •construction of ‘alternative IDs’

97% of individuals in one site (population ~400,000) were found to have unique ‘alternative ID’.


Data linkage

Groups of people in social care data – how many are we able to match to GP register list (of ages 75+)?

Varies, but better for those with > service use


Data linkage Social & secondary care interface


Predictive value, sensitivity and specificity of the model incorporating a £5,000 threshold

Predicting social care costs: feasibility study


Information on Social Care Needs


Simplifying and sorting

Accommodation Adaptation Adult Placement Advice Given Carer Support Counselling DA1 Day Care Direct Payments Domiciliary Employment/Training Equipment Fostering Group Activity Holiday Payment Housing Advice Given ILS Independent Living Individual Informal Projects Information & Advice Lifeline 400 Lifeline 400 + Smoke Alarm Mobility Training (DAY) Mobility Training (NIGHT) Nursing Nursing (Block Contract) Nursing Block Beds Old Codes OT Advice Given OT Rehab Sessions Other Other Carer Support Services Outreach Permanent Placements with Parents Professional Support R.O. Allowance Referral to Carers Contact Line Referral to Carers Information Service Referral to Crossroads Registration Rehabilitation Residential Residential Block Beds Respite Respite (Carer) Secure Accommodation Sheltered Social Development Special Sheltered Specialist Assessment and/or Treatment Specialist Communication Equipment Specific Approval Support Supported Living Talking Book Machine Time Allocation Vouchers Training Transport Washer


Individual histories


Transitions in care (75+ in one site)


Individual health and social care event timeline over a three-year period


Predictive Modelling

Attempting to predict: • For over 75s

a) Admission to care home (or receipt of high intensity home care) or £5,000 increase in social care costs in one year

b) Where person had no ‘significant’ costs in prior two years


Original models predicting change at £5k threshold


Diagram


Distillation flask


Trade off between PPV (blue line) and sensitivity (red line) according to different risk cut-offs


Which variables are important in pooled £1k model?


Models using lower £1k thresholds


Impact of adding new datasets


Iteration with Sites - Application

• Several sites said they would like to run the Combined Model and the social care model “side-by-side”

• Theographs thought to be very useful• Differences between sites: some sites preferred small numbers of

clients others wanted large numbers for mail-shots• Regarded concerns over invasion of privacy as “a non-issue if the

wording is right”• Keen to use predictive models for existing or planned re-ablement

services and multidisciplinary teams


Personal observations on social care data

We could access user level social care data but...• Different systems • Issues of standardisation /coding etc and the absence of standard,

structured coding schemes• Some very detailed information collected but can be difficult to use

in models• We found it harder to obtain information on descriptions of needs • Local concerns about data quality• Note the history of comparative benchmarking


Headlines

• We have created linked health/social care data sets in five sites (overall populations about 2 million people aged 75+)

• We modelled intensive social care:• Models had satisfactory PPV but low sensitivity at a risk score

threshold of 50 (although can be traded-off against each other)• Mainly social care driven• Though accuracy worse than PARR model, accuracy was

comparable with SPARRA and the combined model• Discussion with sites – they were less concerned with model

performance than we were• We tried dozens of ways to improve the basic model (5K)• We modelled change in social care costs with lower thresholds:

1K models are much better but usefulness may be diminished


And the data have so much more to offer.....

• Cost modelling • Iso-resource classification (individual budgets)• Trajectories and transitions of care• Evaluation (re-ablement)• Information for

- Commissioners and managers- Professionals- Service users and carers

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility StudyFebruary 2011

www.nuffieldtrust.org.uk

Sign-up for our newsletterwww.nuffieldtrust.org.uk/newsletter

Follow us on Twitter(http://twitter.com/NuffieldTrust)

© Nuffield Trust

predictive models for health and social care: a feasability study

Health & Medicine

care trust

sites social care

social care usefrom

intensive social care

number of predictive

health sector

range of models

data accurate