bridget hopwood & amanda attwood

Effectively Measuring Healthcare ExperiencesMRS Healthcare Research Conference 2015

Bridget Hopwood, Director of Health Experiences

Amanda Attwood, Senior Project Manager

Picker Institute Europe


A not-for-profit healthcare research organisation:

Experts in gaining patient feedback:

Preferred partner of the Care Quality Commission (CQC) for developing questionnaires and methodologies

NHS patient and staff survey coordination centres for CQC and Department of Health

Full spectrum service

Quantitative / Qualitative / Analysis packages

Improvement Services and Support

Picker Institute Europe 3

Health Experiences Team - vulnerable and lesser-heard

groups

e.g. children & young people; elderly; learning disabilities

Examples of who we work with


Overview

How to effectively measure healthcare experiences

Patient experience

What is it and why should we measure?

Survey DesignExamining the principles of good design of patient reported experience measures (PREMs)

Survey ImplementationConsiderations when implementing a patient survey –choosing the appropriate methodology for the patient group

Case study – Central London Community Healthcare trust


“...what actually happens to patients as people in the process of receiving care & treatment “

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

Patient experience

…gives the patient a ‘voice’

Core to an organisation’s reputation &

productivity

A risk management issue

A key indicator of quality and safety

An opportunity!

Achieving Patient-Centred Healthcare: Indicators of Progress and Success

Measuring patient experience – why?

How to measure the patient voice effectively


Good questions?

What you ask and how you ask it reflects the output

e.g

How ‘good’ was your doctor today?

Very good / Good / Not very good

What does this tell us? How do we improve?


Quality questions

Quality data Quality improvement

4 Principles of PREM development

Patient involvement throughout

Patient centred care requires patient centred

questions

Experience based questions not satisfaction

Thoroughly test with the target group

Choose most appropriate method of data

collection


One size doesn’t fit all!

Survey Design Process


•Understanding of existing measures and research

Literature Review

•To gauge an understanding of the issues and priorities of the target group

Qualitative Stage

•Prioritise areas of importance to target groups as identified from qual stage

Questionnaire Design

•Test for:

•Comprehension

•Qu wording and length

•Recall

•Response codes

Questionnaire Cognitive Testing

•Adopt appropriate methodology for target group

Implementation (and pilot)



“There wasn’t much [to do] except eat and watch people

die”

[Male, aged 15yrs]

Different experiences, different priorities


Survey Design



Literature Review


Qualitative Stage



•Test for:

•Comprehension


•Recall

•Response codes




Questionnaire design

Focus on recent personal experience

Use as few questions as compatible with data needs

Speak their languageAsk in a way they can understand

Take into account age and cognitive ability in particular

Experience questions Actionable data


Example QuestionsHow good was the information you received?

Q Did you receive enough information?

1 Yes, enough information

2 Some, but not enough information

3 No information at all

Q Were you given information in a way you could understand?

1 Yes, definitely

2 Yes, to some extent

3 No

Q Was this information useful?

1 Yes, definitely

2 Yes, to some extent

3 No


Survey Design



Literature Review


Qualitative Stage



•Test for:

•Comprehension


•Recall

•Response codes




Key Stakeholders involved at all

stages – providing feedback and

expertiseHCPs, experts and patients and

carers


• Question wording and length

• Response options

• Questionnaire length

• Relevance

• Comprehension and recall

• Questionnaire Design: Colours and

illustrations

Appealing to children and appropriate to their ability

Maintain interest and engagement

Maximise child involvement and

completion

“there’s quite a lot of little

questions in the big

question... and you can’t

remember everything”

Cognitive Testing

Examples of our feedback tools for children and young people


Children’s FFT Paediatric inpatient

and day case survey

Acute hospital care

Inpatient & day case (now national CQC)

Outpatients (DH Innovation in outcomes competition winner)

Emergency Department

Neonatal (parents’ experiences)

Transition

CYP Friends and Family Test

Children’s community – SaLT; OT; physio

Chronic Conditions

Allergies

Sickle cell disease

ImplementationAmanda Attwood


Patients vs Experience

For most healthcare ‘consumer’ – types, there are many different ‘experiences’

Type of patientOld vs young

Male vs female

Ethnicity

Chronic vs Acute conditions

Cognitive Ability


Type of ExperienceInpatient

Outpatient

Urgent Care

Community Care – eg GP / walk-in centre/ home delivered care

Residential care/care homes

Different experiences; different priorities; different

expectations

Which Methodology should you use?


Overview Advantages Limitations

- Retrospective and cross-sectional

- Patients sampled from organisation

records and sent questionnaires at

home

- Relatively cost effective

- Highly scalable

- Long fieldwork period

- Time ‘lag’, reliance on recall

- Targeted sample called at home

(sometimes with a pre-approach letter)

- Allows for complex routing and filtering

- Rapid collection

- High quality data

- High response rates

- Greater unit costs than postal

- Some may consider intrusive

- Consent process

- May be conducted at people’s homes,

in care settings or other public

locations.

- Costs and approaches vary

depending on the purpose and

recruitment options.

- Similar to telephone surveys but allowing for more

complex questioning

- Increased unit cost

- Prone to interviewer effects and

cognitive bias

Contemporary approach for collecting

feedback in ‘near-real-time’: the use of

electronic devices to collect feedback

- Obviates need for data entry –

- Useful for collecting feedback in real time and

providing a rapid overview of current events. Useful

for service improvement in particular units

- Prone to interviewer effects and

cognitive bias

- Reliant on engaged volunteers to

administer

- Paper surveys are handed out at

bedsides or in clinics. Respondent then

hands back to volunteer, or envelope to

return survey.

- Works well in Outpatients clinics. Respondent can

complete whilst waiting or take home and return at

leisure

- Reliant on engaged staff volunteers to

ensure handed out

- Difficult to monitor response rates

- Internet based surveys

- Sent to people via email or link via

post or left open for self selection

samples

- Extremely low cost - Direct data entry

- Allows for complex routing

- Inherently limited coverage (eg only

50% of people aged 65+ use internet

Postal

Telephone

Face-to-Face

Electronic Devices

Hand out

On-line

Considerations

Will you reach all of your audience with one method?

How will you obtain sample/ can it be drawn from database?

Will it be possible without introducing bias?

Will it produce actionable data?

Will it be achievable within budget?

Will it be achievable within the timeframe?


When one size won’t fit all?

Working with large NHS Community trusts it is not possible to take a ‘one size fits all’ approach.

Central London Community Healthcare NHS TrustIn four London boroughs

Across 64 different service delivery units

Seeing more than 150,000 people each year

In over 160 sites


Reaching our target audience(s)?

NHS Trust patient sample/ access usually facilitated by the trust.

Contacts from record system for telephone/ mail out

Clinic for hand-out/ face-2-face/tablet interviews

QR Codes in clinics/ on letters

Web links on website/ letters

Kiosks on site


Service vs Methodology?

eg:

District nursing = home based care delivered by the nurse, mainly elderly housebound population, often with poor recall

Mail not suitable – poor vision, comprehension without assistance could be poor, poor motor skills and timescales mean recall (without assistance) limited

Telephone – most appropriate, eliminates bias, prompted recall


Services PatientsData

systemsClient Method

How does it work?

Around 2,000 interviews conducted each month using telephone, tablet, kiosk, paper (hand-out) methods & comment cards.

Data mapped and sent to the client formated ready to load straight into their own system (QlikView).

Monthly reports.


What does it mean to the client?

“Its important to us that we are able to provide a robust system with a third

party provider to assure our patients that their feedback is anonymous, rigorously

collated, and analysed to a high standard to show themes and trends”


Carol Dale

Head of Patient Experience

Central London Community Healthcare

The power of the patient voice!



“No decision about me without me!”

Harvey Picker


Buxton Court

3 West Way

Oxford OX2 0JB

Tel: + 44 (0) 1865 208100

Fax: + 44 (0) 1865 208101

[email protected]

www.pickereurope.org

Charity registered in England and Wales: 1081688

Charity registered in Scotland: SC045048

Company limited by guarantee registered in England and Wales

bridget hopwood & amanda attwood

Documents