an introduction patient reported outcome measures (proms)
DESCRIPTION
An introduction to the key concepts of patient Reported Outcome Measures, including reliability and validity, generic versus disease specific,selection criteria and their adaptation for different cultural groups.TRANSCRIPT
Copyright DHP Research & Consultancy Ltd 2010
An introduction to Patient Reported Outcome Measures
(PROM’s)
Dr Keith Meadows
DHP Research & Consultancy Ltd
July 2010
Copyright DHP Research & Consultancy Ltd 2010
Overview
• PROM’s What are they and what are they used for?
• Generic and condition specific - strengths and weaknesses
• What should we look for when choosing a PROM
• Adapting PROM’s for cultural and ethnic groups
• Interpreting PROM data
Copyright DHP Research & Consultancy Ltd 2010
Definitions
• Patient reported outcomes (PROM’s) are outcomes known only to the patient
• Patient reported outcome measures (PROM’s) are tools we use to measure patient outcomes
Copyright DHP Research & Consultancy Ltd 2010
Variations on a theme
• Health status
• Health-related quality of life (HRQoL)
• Well-being
• Health outcomes
• Quality of life
• Satisfaction
Copyright DHP Research & Consultancy Ltd 2010
But all are based on…
Self-assessment
Copyright DHP Research & Consultancy Ltd 2010
Common content e.g. SF-36
• Physical functioning
• Role limitations due to physical health
• Bodily pain
• General health perceptions
• Vitality
• Social functioning
• Role limitations due to emotional problems
• Mental health
Copyright DHP Research & Consultancy Ltd 2010
The underlying principle
To measure across the complete continuum of a given construct
No pain Severe pain
Copyright DHP Research & Consultancy Ltd 2010
Something for everyone
• Patient choice• Audit• Quality improvement (Clinician & Provider)• Research• Training
• After Black N. 2008
Copyright DHP Research & Consultancy Ltd 2010
General applications
• Measurement of the patient’s health status or health-related quality of life (HRQoL) at a single point in time
• PROMs are used to derive measures of the outcomes of specific interventions.
• Changes in health status at two different points in time (e.g. before and after an operation) can be used to derive a measure of the impact of health care interventions.
• Certain PROMs suitable for purposes of economic evaluation (e.g., estimation of quality-adjusted life years – QALYs)
Copyright DHP Research & Consultancy Ltd 2010
Some specific applications
• Personalised care planning
• Self-assessment
• Annual review
• Informed decision making • Population health
Copyright DHP Research & Consultancy Ltd 2010
Why consider the patient’s perspective?
If quality is to be at the heart of everything we do , it must be understood from the perspective of the patient’
‘Just as important (as clinical measures) is the effectiveness of care from the patient’s own perspective which will be measured through patient-reported outcome measures’
Next Stage Review 2008
Copyright DHP Research & Consultancy Ltd 2010
And also…
• Patients know better – ‘We know little of the clinical outcomes of NHS services from the patient’s perspective. PROM’s fill this gap’ DH 2007
• Clinical outcomes not always related to how the patient feels
• Patients like to be asked
Copyright DHP Research & Consultancy Ltd 2010
What can PROM’s tell us?
• Which is the best treatment for the condition?
• Is one subgroup of the population sicker than the others?
• Is an individual patient’s condition getting better?
After Coulter A 2008
Copyright DHP Research & Consultancy Ltd 2010
What should we look for when choosing a PROM?
…evidence that it is a measure of what it is supposed to measure
Copyright DHP Research & Consultancy Ltd 2010
….across the complete continuum of a given construct
No limitations Mobility Severe limitations
Copyright DHP Research & Consultancy Ltd 2010
….and locating individuals by providing meaningful scores on
that continuum
Very anxious anxiety No anxiety
Copyright DHP Research & Consultancy Ltd 2010
Therefore…
• It is important we choose the right PROM for our clinical practice, patients, study, trial
• Check the information we get and don’t get from a PROM
Copyright DHP Research & Consultancy Ltd 2010
The need for a conceptual model
• A diagram of proposed causal linkages among a set of concepts believed to be related to a particular disease (Earp & Enmett 1991).
• A taxonomy of patient outcomes according to the underlying health concepts they represent and proposes specific and causal relationships between different health concepts (Wilson & Cleary 1995)
Copyright DHP Research & Consultancy Ltd 2010
Diabetes Health Profile Conceptual Model© DHP Research & Consultancy Limited 2010© DHP Research & Consultancy Limited 2010
Impact
Management Treatment Symptoms Diet Complications
Diabetes
Emotional Behavioural
Anxiety MoodLimitations in
Social/workfunctioning
Eating behaviour
DHP
Copyright DHP Research & Consultancy Ltd 2010
Development based on….
• Literature review
• In-depth interviews with patients/clinicians family etc.
• Thematic analysis of qualitative data
Copyright DHP Research & Consultancy Ltd 2010
Conceptual framework
A representation of the expected relationship of items within a domain within a PROM concept
Item AItem B Domain score 1Item C Item CItem E Domain score 2Item F
Copyright DHP Research & Consultancy Ltd 2010
We therefore must look for evidence of…
• Validity • Reliability• Responsiveness• Precision
Copyright DHP Research & Consultancy Ltd 2010
Validity
• Does it measure what it is meant to?
- Content validity - does the content reflect the concept/construct measured, is the content representative?
- Face validity - do the individual items look as if they are measuring what they should?
- Criterion validity - can the construct be measured accurately?
- Construct validity - Is there a conceptual model or theoretical underpinning?
Copyright DHP Research & Consultancy Ltd 2010
Reliability
Are the results stable over time when applied to the same people at different time periods? (Test-retest reliability)
Copyright DHP Research & Consultancy Ltd 2010
Precision
Does the measure discriminate between different patient groups, health states, treatments etc?
Copyright DHP Research & Consultancy Ltd 2010
Responsiveness
Is the measure responsive to change when change is present?
Copyright DHP Research & Consultancy Ltd 2010
Anything else?
- Acceptability – will people fill it in?- Response rates- Item completion rates- Missing cases
Copyright DHP Research & Consultancy Ltd 2010
Feasibility – how easy will it be to use?
- Cost- Time- Ease of scoring- Interpreting scores- Supporting documentation (Manual, norm-
reference scoring etc.)
Copyright DHP Research & Consultancy Ltd 2010
Does the name of the PROM reflect what is being
measured?
• Not all PROM’s have a conceptual model/theoretical underpinning
• Some PROM’s are named by the items that are grouped together
Copyright DHP Research & Consultancy Ltd 2010
Implications of choosing the wrong PROM
• Fail to identify significant outcomes
• Mislead clinical practice
• Misrepresent treatment and disease impact
• After Cano S. 2008
Copyright DHP Research & Consultancy Ltd 2010
Generic and condition specific - strengths and weaknesses
Generic
• Suitable for the general population
• Comparisons with other conditions/disease groups
• Content may be redundant for certain condition/illnesses
• Not sensitive to detecting disease-specific issues
Condition-specific
• Specific to disease group
• Sensitive to detecting clinically significant changes
• Content relevant to target group
• Cannot compare with general population
Copyright DHP Research & Consultancy Ltd 2010
Disease-specific or generic?
‘Go for a combined approach?’
Copyright DHP Research & Consultancy Ltd 2010
Index v Multidimensional
• Overall score (but can be graded)
• Less information
• ?Easier to score
• Appropriate for cost-benefit analysis
• Provides a profile
• Reflects the important/different components of the illness
• Provides more information
• Can be long
Copyright DHP Research & Consultancy Ltd 2010
Approaches to interpreting PROM data
• Minimally Important Difference (MID) to change
• Known groups
• Response to treatment
• Normative and reference groups
• Statistical significance
• Effect size
Copyright DHP Research & Consultancy Ltd 2010
Effect size
• Effect size is a simple way of quantifying the difference between two groups that has many advantages over the use of tests of statistical significance alone. Effect size emphasises the size of the difference rather than confounding this with sample size.
Small 0.2 - 0.4
Moderate 0.5 – 0.7
Large ≥ 0.8
Copyright DHP Research & Consultancy Ltd 2010
Using PROM’s for cultural and ethnic groups
• Technical problems
• Conceptual problems
• Linguistic problems
Copyright DHP Research & Consultancy Ltd 2010
What is our aim?
To achieve cross-cultural equivalence:
• Semantic equivalence – equivalence in meaning of words
• Conceptual equivalence – validity of the concepts in the target language
• Idiomatic equivalence – equivalent idioms/expressions in target language
• Experiential equivalence – situations should fit target language
Copyright DHP Research & Consultancy Ltd 2010
Overview
• PROM’s must be fit for purpose
• Are they valid and reliable with a clear conceptual/theoretical underpinning
• Don’t choose PROM’s on basis of popularity alone
• Take an evidence-base approach in selecting the PROM
• Be sure what you want to measure
• Don’t assume the PROM is accurate in telling you what it measures
• Translation alone does not result in cultural equivalence
Copyright DHP Research & Consultancy Ltd 2010
Overview cont’d
• Statistical significance does not mean clinical significance
• p values can be misleading (large samples can result in high p values)
Copyright DHP Research & Consultancy Ltd 2010
This presentation is a selection of slides taken from our training course ’Patient Reported Outcome Measures (PROMs) in Clinical Trials and Health Care – An introduction.
If you would like more information on our training courses and or the Diabetes Health Profile email: [email protected]
Visit our website www.dhpresearch.com