t10 outcome assessment why, what and how? dr. frederike van wijck & john dennis

T10 OUTCOME ASSESSMENTWhy, what and how?

Dr. Frederike van Wijck & John Dennis

Learning OutcomePlan a safe, effective and appropriate

intervention, i.e.: • Design and adapt appropriate progressive

physical activity programme(s) after stroke using findings from the physical/ exercise assessments, etc…

Demonstrate competency in relevant assessment procedures:

• Monitor clients’ progress against agreed goals

Content1. Measuring outcomes: why?

2. Measurement: general principles

3. Measuring outcomes: what?– General framework: the ICF– Specific suggestions for the exercise-after-

stroke setting

4. Using outcome measures in an exercise after stroke setting: how?

Outcome measure – a definition:

“ a test or scale administered by therapists that has been shown to measure accurately a particular attribute of interest to patients and therapists and is expected to be influenced by the intervention”

(Mayo, 1995)

1. Measuring outcomes: why?

1. Screening: testing eligibility for exercise

2. Baseline assessment: establishing starting point for exercise programme

3. Follow-up assessment: charting change following exercise

4. Monitoring: to chart adherence and identify adverse effects

This session: baseline and follow-up assessment using outcome measures

1. Measuring outcomes: why not?

Common reasons for not using outcome measures:• It takes time away from the actual exercise• It is a burden for participants• It’s complex and a hassle for the instructors• You need training – we don’t have time for

that• What do these measures tell you anyway – I

know if something works!

Do you??!

SCIENCE VS. COMMON SENSE

Science:

“knowledge, ascertained by observation and experiment, critically tested, systematised and brought under general principles”

Cambridge English Dictionary

Common sense:

“normal understanding, good practical sense in every day affairs, general feeling (of mankind or community)”

Oxford English Dictionary

• Common sense is not good enough for exercise instructors/ health care professionals;

• Exercise/ rehabilitation/ health care needs to be based on science!

Content1. Measuring outcomes: why?

2. Measurement: general principles

3. Measuring outcomes: what?– General framework: the ICF– Specific suggestions for the exercise-after-

stroke setting

4. Using outcome measures in an exercise after stroke setting: how?

2. Measurement: general principles

Characteristics of good outcome measures:

1. Relevant

2. Valid

3. Reliable

4. Sensitive to change

5. Practicable

6. Results can be easily communicated(Wade, 1992)

Characteristics of good outcome measures

Relevance:

the pertinence of the information

Consider:Is this information useful – what does it tell

me?What am I going to do with the information?

Characteristics of good outcome measures

Validity:

the measure does what it is purported to do

Consider:• Which idea/ construct does this measure

address?

Characteristics of good outcome measures

Reliability:

the measure gives the same result each time the same quantity is measured.

Consider:– Intra-rater variation– Inter-rater variation-> Importance of protocols! (tutorial)

Characteristics of good outcome measures

Sensitivity to change:

the measure can detect changes that are relevant

Consider:• On what scale is/ are the item(s) scored?

E.g.:– 0/ 1 or Yes/ No– 0-10 (Visual Analogue Scale)

• Floor and ceiling effects

Characteristics of good outcome measures

Practicability:

the measure is quick and easy to use

Consider:• Amount of information required• Duration of the process• Complexity of the process• Burden on client (and you!)

Characteristics of good outcome measures

Communicability:

The results can easily be reported and understood

Consider:• Amount• Format• Standardisation of the information

Summary - general principles of measurement:

Characteristics of good outcome measures:

1. Relevant

2. Valid

3. Reliable

4. Sensitive to change

5. Practicable

6. Results can be easily communicated(Wade, 1992)

Content1. Measuring outcomes: why?

2. Measurement: general principles

3. Measuring outcomes: what?– General framework: the ICF– Specific suggestions for the exercise-after-

stroke setting

4. Using outcome measures in an exercise after stroke setting: how?

3. Measuring outcomes: what?

A general framework for outcome measurement in clinical practice: the ICF

International Classification of Functioning, Disability and Health

“Aim of the ICF classification is to provide a standard language and framework for the description of health and health-related

states.”

http://www.who.int/classification/icf/intros/ICF-Eng-Intro.pdf

ICF definitions

Impairments are problems in body function or structure such as a significant deviation or loss.

Activity limitations are difficulties an individual may have in executing activities.

Participation restrictions are problems an individual may experience in involvement in life situations.

ICF model

http://www.who.int/classification/icf/intros/ICF-Eng-Intro.pdf

The ICF: A general framework for outcome measurement in rehabilitation

Activity Limitations

Impairments Participation Restrictions

Health Condition

Outcome measures and the ICF

Which outcome measures do you use in your work ?

Where do they fit within the ICF?

Can you think of one outcome measure in each of the ICF domains for a person who has had a stroke?

Outcome measures for exercise after stroke

Activity LimitationsExample?

ImpairmentsExample?

Participation Restrictions

Example?

Person with stroke

3. Measuring outcomes: what/ how?

Suggested outcome measures for exercise after stroke specifically :

• 6 min. walk/ 10 m. walk

• Timed up and Go

• Visual Analogue Scale (VAS)

• Stroke Impact Scale

+ Register: for monitoring adherence

Onto: Measuring Outcomes: how?

• 6 min. walk/ 10 m walk

• VAS

• Timed up and Go

• Stroke Impact Scale

http://figuredrawings.com/Animation.html

6-minute/ 10 m. walk test

Construct: ?

Evaluation?

6-minute walk testConstruct: maximum walking distance in 6 minutes• Relevance: functional test for exercise endurance,

O2 uptake• Validity: good• Reliability: high• Sensitivity: ?• Practicability: good• Reporting: easy (distance (m))• Normative data for healthy people aged 60-89 yrs:

345-623 m(Steffen et al., 2002)

Timed Up and Go

Construct: ?

Evaluation?

Timed Up and Go

Construct: time to stand up from arm chair, walk 3 m, turn, walk back, sit down

• Relevance: functional test for basic mobility for frail elderly in community

• Validity: acceptable• Reliability: moderate - high• Sensitivity: ?• Practicability: good• Reporting: easy (time (s))• Normative data for healthy people aged 60-89 yrs:

7-12 s(Steffen et al., 2002)

VAS

Construct: ?

Evaluation?

VAS

Construct: person’s perception of a particular construct

• Relevance: depending on what is assessed. Can be used to assess individual goal attainment

• Validity: generally good• Reliability: generally high• Sensitivity: high • Practicability: caution with stroke, esp. higher

cortical problems and neglect (Price et al., 1999)• Reporting: easy

Correct use of VAS in stroke

Price et al. (1999), p. 1359.

Correct use of VAS in stroke

Price et al. (1999), p. 1360.

Stroke Impact Scale

Construct: ?

Evaluation?

Stroke Impact ScaleConstruct: the person’s perceived impact of stroke across

range of domains (incl. movement, ADL, cognition, communication, emotion, participation)

• Relevance: high• Validity: good (devised with target population)• Reliability: moderate – very high• Sensitivity: each item on 5-point scale + one VAS item• Practicability: mixed• Reporting: time-consuming but can be done by mail• Normative data: not applicable• Interpretation: change between 10-15 points clinically

meaningful(Duncan et al., 2003)

Pitfalls of measurement – and how to fix them

Problem

• Error:– Systematic– Random

• Wrong signals:– False +– False -

Solution?

Pitfalls of measurement – and how to fix them

Problem

• Error:– Systematic– Random

• Wrong signals:– False +– False -

Solutions:

• Errors:– Calibrate your instrument– Use standardised protocol

• Wrong signals:– Check sensitivity– Verify with other information

Pitfalls of measurement – and how to fix them

Problem

• Error:– Systematic– Random

• Wrong signals:– False +– False -

Solutions:

• Errors:– Calibrate your instrument– Use standardised protocol

• Wrong signals:– Check sensitivity– Verify with other information

Errors are inherent in any form of measurement!

Always be aware and try to reduce.

Summary

Outcome measures are necessary to:

• Establish baseline for exercise

• Evaluate change following exercise

-> science underpinning your work

EVIDENCE BASED PRACTICE

(meten is weten!)

Choosing your Outcome Measure how to go about it

Safe?

NO

YES

Relevant?

NO

YES

Science Robust?

NO

YES

Practicable?

NO

YES

GO

T

H

I

N

K

A

G

A

I

N

Safe?

NO

YES

Relevant?

NO

YES

Science Robust?

NO

YES

Practicable?

NO

YES

GO

T

References

• DUNCAN, P. W., LAI, S. M., BODE, R. K., PERERA, S. & DEROSA, J. (2003b) Stroke Impact Scale-16: A brief assessment of physical function. Neurology, 60, 291-6.

• DUNCAN, P. W. Stroke Impact Scale (SIS). Rehabilitation Outcomes Research Centre, US Department of Veteran Affairs. Available from: http://www1.va.gov/rorc/stroke_impact.cfm (last accessed 05/12/05).

• WADE, D. T. (1992) Measurement in Neurological Rehabilitation, Oxford, Oxford University Press.

• WORLD HEALTH ORGANISATION (2001). International Classification of Functioning, Disability and Health. Available from http://www.who.int/classifications/icf/en/