The Importance of measuring outcomes, including Patient Reported Outcome Measures (PROMS)BAOT Lifelong Learning Event

10 November 2010

Dr Alison Laver-FawcettHead of Programme, BHSC(Hons) Occupational TherapyYork St John University

The drive towards outcomes is nothing new...

Department of Health (DoH, 1998) stated that the modernisation of care “moves the focus away from who provides the care, and places it firmly on the quality of services experienced by, and the outcomes achieved for, individuals and their carers and families” (para. 1.7).

Commissioning for outcomes

On the 1st February 2010 I joined six colleagues for a one day meeting to contribute to the development of a ‘Commissioning for better outcomes in Dementia Services’ workbook. The workbook is being developed for commissioners to assist the implementation of the National Dementia Strategy

Why measure outcomes?

Back in 2000 Dr. Carolyn Unsworth stated: ‘Current pressures to document outcomes

and demonstrate the efficacy of occupational therapy intervention arise from fiscal restraints as much as from the humanitarian desire to provide the best quality health care to consumers.’

‘However, measuring outcomes is important in facilitating mutual goal setting, increasing the focus of therapy on the client, monitoring client progress, as well as demonstrating that therapy is valuable’(p. 147)

10 years on this still applies

• Disseminate results to stakeholders • (client, carer, commissioner, manager)

Occupational Therapy Process (COT, 2003)

1. referral or reason for contact

2. information gathering3. initial assessment4. reason for intervention /

needs identification / problem formulation (defining desired outcome)

5. goal setting (negotiating pre-determined outcome, baseline measures)

6. action planning7. action8. ongoing assessment

and revision of action9. outcome and outcome

measurement (establishing actual outcome)

10. end of intervention or discharge

11. review (long-term outcome)

Commissioning Outcomes

Service predetermined outcome(s): these may be articulated in a service specification, clinical guidelines, protocols or standards. In a service commissioning agreement PROMS may be mandated

Individual client / staff outcome goals: these are negotiated within the framework of the above for each service user

Sources of outcome data

Self-report: questionnaires, rating

scales or interviews given to the

service user to complete, including

PROMS

Therapist observation: standardised

outcome measures or observations

related to SMART goals

Sources of outcome data

Proxy-report: questionnaires or interviews given to the informal care-giver (family member, friend) to complete (may include PROMs)

Proxy-report: outcome data provided by another member of the MDT and / or other staff involved in the care package / assessment / management

Service data: discharge destination, length of stay, waiting time, delayed transfers of care, cost of packages of care, number, length and cost of interventions

What are Patient Reported Outcome Measures?

Patient Reported Outcome Measures (PROMS) employ short, self-completed questionnaires which measure the patient’s health status or health-related quality of life at a single point in time.

PROMs are used to derive measures of the outcomes of specific interventions.

Changes in health status measured by PROMs at two different points in time (e.g. before and after an operation) can be used to derive measure of the impact of health care interventions.

(Department of Health, DH, 2009)

Patient-Reported Outcome Measures (PROMs)

‘Patient-reported outcome measures (PROMs) provide a means of gaining an insight into the way patients perceive their health and the impact that treatments or adjustments to lifestyle have on their quality of life. These instruments can be completed by a patient or individual about themselves, or by others on their behalf.’ (PROM Group, UHCE, University of Oxford)

Requirement to collect PROMs The new Standard NHS Contract for

Acute Services, introduced in April 2008, included a requirement in Schedule 5 for providers to report from April 2009 on PROMs.

This was for patients undergoing Primary Unilateral Hip or Knee replacements, Groin Hernia surgery or surgery for Varicose Veins.

Context – what are the drivers for PROMs? Lord Darzi’s Interim Report on the future of the NHS

recommended that patient-reported outcome measures (PROMs) should have a greater role in the NHS (Darzi, 2007).

Lord Darzi’s report ‘High Quality Care for All’ (2008) outlines policy regarding payments to hospitals based on quality measures as well as volume. These measures include PROMs as a reflection of patients’ experiences and views.

Guidance has now been issued regarding the routine collection of PROMs for the selected elective procedures (DH, 2009).

(Jenkinson, Gibbons and Fitzpatrick, 2009, p3)

The Patient Reported Outcome Measurement Group in Oxford

Patient-reported outcome measures (PROMs) offer enormous potential to improve the quality and results of health services.

They provide validated evidence of health from the point of view of the user or patient.

They may be used to assess levels of health and need in populations, and in users of services

...over time they can provide evidence of the outcomes of services for the purposes of audit, quality assurance and comparative performance evaluation.

They may also improve the quality of interactions between health professionals and individual service users.

• (Jenkinson, Gibbons and Fitzpatrick, 2009, p3)

Generic and Condition Specific PROMS

A range of PROMS are available: Generic PROMS are applicable to

and comparable across a range of treatments or conditions,

Condition-specific PROMS are considered to ‘often be more sensitive to changes in health status’ (DH, 2009)

Example of a Generic PROM

Euro-Qol EQ-5D This was identified as a generic PROM by Department of Health (2009)

and also as applicable for use with people with stroke (Jenkinson, Gibbons and Fitzpatrick, 2009)

EQ-5D is a standardised instrument for use as a measure of health outcome

Applicable to a wide range of health conditions and treatments, it provides a simple descriptive profile and a single index value for health status.

EQ-5D is designed for self-completion by respondents and is ideally suited for use in postal surveys, in clinics and face-to-face interviews. It is cognitively simple, taking only a few minutes to complete. Instructions to respondents are included in the questionnaire.

For more information see: http://www.euroqol.org/

Examples of Condition Specific PROMs Oxford Hip Score (Available from:

http://phi.uhce.ox.ac.uk/pdf/OxfordScores/Oxhipscore.pdf )

Oxford Knee Score (Available from: http://phi.uhce.ox.ac.uk/pdf/OxfordScores/Oxkneescore.pdf )

Examples of Condition Specific PROMsStroke Impact Scale (SIS) The instrument content was derived from input

from stroke patients, caregivers and health professionals with experience in the field of stroke.

It contains 59 items across eight domains (strength, hand function, ADL/IADL, Mobility, Emotion, Memory, Communication and Social Participation).

Jenkinson, Gibbons and Fitzpatrick (2009) report that the SIS is an appropriate PROM for people who have had a stroke, p24)

Some existing rehabilitation measures are being identified in reports related to PROMSNottingham Extended Activities of Daily Living (ADL)

Scale ‘The Nottingham Extended ADL Scale was developed

and evaluated as a questionnaire for postal use (Nouri and Lincoln, 1987). It assesses the ability to carry out functional tasks, such as using public transport, housework, social life and hobbies. Scores in four areas: mobility, kitchen tasks, domestic activities and leisure activities can be added to give a summary score out of 22. Respondents are asked whether they do the activity rather than if they can do it, in order to assess level of activity rather than capability.’

(Jenkinson, Gibbons and Fitzpatrick, 2009, p25)

Bibliographic database comprising over 16000 records relating to patient-reported outcome measures, with a specially designed keyword search facility.

Part of the NHS Information Centre for health & social care. The most recent bibliographic update is current to April 2008.

Systematic reviews of PROMs relevant to specific disease and population (demographic) groups.

General information on patient-reported health outcomes and instrument selection, including guidance regarding the selection of appropriate instruments for use in clinical trials, clinical practice, and population surveys.

http://phi.uhce.ox.ac.uk/home.php [accessed 1 March 2010]

Links to related websites including instruments, resources, organisations, research groups, and journals.

A page about the Oxford Orthopaedic scores including PDFs of the questionnaires and guides to their usage.

http://phi.uhce.ox.ac.uk/home.php [accessed 1 March 2010]

Patient Reported Outcomes Measurement Group

Four new reports on PROMs for specific conditions have been published in 2010:

Stroke Heart Failure Epilepsy Asthma

http://phi.uhce.ox.ac.uk/newpubs.php

Where to look for outcome measures

Asher IE (2007) Occupational Therapy Assessment Tools: An Annotated Index (3rd ed)

Clarke C Sealey-Lapes C Kotsch L (2001) Outcome measures Information Pack for Occupational Therapy

COT (2009) SNOMED project list of assessments

Link to SNOMED list of OT assessments

http://www.cot.co.uk/homepage/search_results/?SearchIT=3F67BEEA1FDF9FCFEA7C4779A336E6CB (accessed 2 February 2010)

This is a list of published assessments – not all listed assessments have psychometric data that supports their use as measures of outcome

We are now working to create a list of outcome terms. Outcome measures have been identified from the list of assessments and are now being mapped to the ICF.

For mental health measures

National Institute for Mental Health in

England (2008) Outcomes Compendium.

Helping you to select the right tool for

best mental health care practice in your

field. [on-line] Available from:

http://www.dh.gov.uk/prod_consum_dh/

groups/dh_digitalassets/documents/

digitalasset/dh_093677.pdf [Accessed

28.10.2010]

Where to look

Some text books include test critiques of measures, for example:

Law, Baum and Dunn (2001) Measuring occupational performance: supporting best practice in occupational therapy.

Literature search:Use your hospital librarian to help youOn-line: Pub-Med and Google Scholar

Commissioning for outcomes

If commissioners make evidence of outcomes a requirement of a service specification and contract – are you ready to deliver evidence of the effectiveness of your service?

Take home message:

If we don’t define and measure outcomes we don’t have evidence of the worth of occupational therapy services

In a system where the driver is for ‘Commissioning for outcomes’ if we can’t evidence our outcomes there is a real risk that occupational therapy will not be commissioned!

Web resources

National Centre for Health Outcomes Development http://nchod.uhce.ox.ac.uk/index.html [accessed 1 March 2010]

National Institute for Mental Health in England (2008) Outcome Compendium. Helping you to select the right tool for best mental health care practice in your field. [on-line] Available from:http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_093677.pdf [Accessed 28.10.2010]

Reference:

Laver Fawcett AJ2007Principles of Assessment and Outcome measurement for Occupational Therapists and Physiotherapists.Chichester:John Wiley & Sons

References

Asher IE (2007) Occupational Therapy Assessment Tools: An Annotated Index (3rd ed) Bethesda: AOTA Press

Clarke C Sealey-Lapes C Kotsch L (2001) Outcome measures Information Pack for Occupational Therapy. London: College of Occupational Therapists

College of Occupational Therapists(2003) Occupational Therapy defined as a complex intervention. London: College of Occupational Therapists. Available from: http://www.cot.org.uk/MainWebSite/Resources/Document/OTDefined.pdf [accessed 26th January 2010]

Corr S, Siddons L (2005) An Introduction to the Selection of Outcome Measures. British Journal of Occupational Therapy, 68(5), 202-206

References

Department of Health (DH, 2009) Guidance on the Routine Collection of Patient Reported Outcome Measures (PROMS) [on-line] Available from: http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_081179.pdf [accessed 27.10.2010]

Heaton J, Bamford C (2001) Assessing the Outcomes of Equipment and Adaptations: Issues and Approaches. British Journal of Occupational Therapy, 64(7), p. 346-356

References

Jenkinson C, Gibbons E, Fitzpatrick R (2009) A Structured review of Patient Reported Outcome Measures related to Stroke. Oxford, Patient-reported Outcome Measurement Group, Department of Public Health, University of Oxford

Law M, Baum C, Dunn W (2001) Measuring occupational

performance: supporting best practice in occupational

therapy. Thorofare: Slack Nouri FM, Lincoln NB (1987) An extended activities of daily living

index for stroke patients. Clinical Rehabilitation; 1: 301-305.

Unsworth C (2000) Measuring the outcome of occupational

therapy: tools and resources. Australian Occupational Therapy

Journal 47: 147-158

Further reading

Doble S E, Santha JC (2008). Occupational well-being: Rethinking occupational therapy outcomes. Canadian Journal of Occupational Therapy, 75(3), 184-190.

James S, Corr S (2004) The Morriston Occupational Therapy Outcome Measure (MOTOM): Measuring What Matters. British Journal of Occupational Therapy, 67(5): 210-216.

Unsworth C (2005) Measuring Outcomes using the Australian Therapy Outcome Measures for Occupational Therapy (AusTOMs - OT): Data Description and Tool Sensitivity . British Journal of Occupational Therapy, 68(8), 354-366

Contact details

Alison J. Laver-Fawcett, PhD, O.T.(C), DipCOTHead of Programme, Occupational Therapy

Faculty of Health and Life Sciences

York St John University

Lord Mayor’s Walk, York

YO31 7EX

01904-624624

a.laverfawcett@yorksj.ac.uk

www.yorksj.ac.uk  

Research Centre for Occupation & Mental Health (RCOMH)

Developing world class research in occupation and mental health to influence best practice

www.yorksj.ac.uk/rcomh