A Comprehensive Approach to the Measurement of Health Outcomes

Ron D. Hays, Ph.DUCLA Division of General Internal Medicine &

Health Services Research

K30 Module 2November 16, 2010 (9:00-10:30 am)

1st floor Conference Room 1357, UCLA

Patient-Reported Outcomes Measurement Information System (PROMIS)

• A nine-year $70 million commitment of NIH to improve and standardize measurement of patient-reported outcomes (PROs)– Self-reported health

• An answer to the PRO “Tower of Babel”

The Tower of Babel (Brueghel, 1563)

3

PROMIS-1 Network: 2004-2009

UNC –Chapel Hill ●

● Duke University*

● Stanford

●● University of

Pittsburgh

● University of Washington

Northwestern ♥ ●

NIH

♥Coordinating Center

StoneyBrook

Psycho-metricTesting

Item Bank(IRT-calibrated items)

0.0

0.5

1.0

1.5

2.0

2.5

-3 -2 -1 0 1 2 3

Theta

Info

rmat

ion

0.0

0.2

0.4

0.6

0.8

1.0

-3 -2 -1 0 1 2 3

Theta

Prob

abili

ty o

f R

espo

nse

Short FormInstruments

CAT

Literature Review

Item Pool

Patient Focus Groups

Expert Input and Consensus

Existing Items

Questionnaireadministered to largerepresentative sample

SecondaryData Analysis

CognitiveTestingTranslationExpert

Review

Newly Written

Items

Literature review

Focus groups

Archival data

analysis Expert review/

consensus

Binning and

winnowing

The Life Story of a PROMIS Item

Literacy level

analysis

Expert item revision

Cognitive interviews

Translation review

Large-scale testing

Validation studies

Calibration decisions

Intellectual property

Short formCAT

Statistical analysis

Domain Framework

Physical Functioning Item Bank

Item1

Item2

Item3

Item4

Item5

Item6

Item7

Item8

Item9

Itemn

50

• Are you able to get in and out of bed?• Are you able to stand without losing your balance for 1 minute?• Are you able to walk from one room to another?• Are you able to walk a block on flat ground?• Are you able to run or jog for two miles?• Are you able to run five miles?

Low High

Person Fatigue Score

Interpretation

Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Likely“I get tiredwhen I run

a marathon”

Unlikely“I get tiredwhen I getout of a chair” Item Location

Q Q QQ Q Q Q Q Q

Q Q Q Q Q Q Q Q QQ Q Q

Q Q QQ Q QQ Q Q

Q Q

Q Q

Q Q

Low High

PRO Bank Person Score

Interpretation Aids

30 40 50 60 70

M = 50, SD = 10

T = (z * 10) + 50

Low High

Example of high fatigue

Fatigue Score=60

30 40 50 60 70

This patient’s fatigue score is 60, significantly worse than average (50). People who score 60 on fatigue tend to answer questions as follows:

…”I have been too tired to climb one flight of stairs: VERY MUCH

…”I have had enough energy to go out with my family: A LITTLE BIT

Low High

Example of low fatigue

Fatigue Score=40

30 40 50 60 70

This patient’s fatigue score is 40, significantly better than average (50). People who

score 40 on fatigue tend to answer questions as follows:

…”I have been too tired to climb one flight of stairs: SOMEWHAT…”I have had enough energy to go out with my family: VERY MUCH

Computerized Adaptive Testing (CAT)• Select questions based on a person’s response

to previously administered questions.

• Iteratively estimate a person’s standing on a domain (e.g., fatigue, depressive symptoms)

• Administer most informative items

• Desired level of precision can be obtained using the minimal possible number of questions.

Beginning of CAT

01

23

45

Item 15

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=56

20 40 60 80

T-Score

Pos

terio

r Dis

tribu

tion

20 30 40 50 60 70 80

T-Score = 50 SE = 10

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15

T-Score: 50 SEM: 10

Best Item-I felt depressed

0.0

0.2

0.4

0.6

0.8

1.0

Item 15

T-Score

Pro

babi

lity

20 30 40 50 60 70 80

12345

20 40 60 80

T-Score

Pos

terio

r Dis

tribu

tion

20 30 40 50 60 70 80

I felt depressed1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 52 SE = 4

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15

T-Score: 52 SEM: 4 01

23

45

Item 10

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=57

Next Best Item-I felt like a failure

0.0

0.2

0.4

0.6

0.8

1.0

Item 10

T-Score

Pro

babi

lity

20 30 40 50 60 70 80

12345

I felt like a failure1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 53 SE = 3

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15,10

T-Score: 53 SEM: 3 01

23

45

Item 1

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=59

Next Best Item-I felt worthless

0.0

0.2

0.4

0.6

0.8

1.0

Item 1

T-Score

Pro

babi

lity

20 30 40 50 60 70 80

12345

20 40 60 80

T-Score

Pos

terio

r Dis

tribu

tion

20 30 40 50 60 70 80

I felt worthless1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 55 SE = 2

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15,10,1

T-Score: 55 SEM: 2 01

23

45

Item 3

T-ScoreIn

form

atio

n

20 30 40 50 60 70 80

|

Max at T-Score=58

Next Best Item-I felt helpless

0.0

0.2

0.4

0.6

0.8

1.0

Item 3

T-Score

Pro

babi

lity

20 30 40 50 60 70 80

12345

20 40 60 80

T-Score

Pos

terio

r Dis

tribu

tion

20 30 40 50 60 70 80

I felt helpless1. Never2. Rarely3. Sometimes4. Often5. Always

T-Score = 55

SE = 2

Items Administered

T-S

core

0 1 2 3 4 5 6 7 8

20

30

40

50

60

70

80 Items: 15,10,1,3,21,2,5

T-Score: 55 SEM: 2

20 40 60 80

T-Score

Pos

terio

r Dis

tribu

tion

20 30 40 50 60 70 80

CAT assessments can achieve higher precision than fixed forms

Rose et al, J Clin Epidemiol 2007 (accepted)

SE = 3.2rel = 0.90

SE = 2.2rel = 0.95

0

0.1

0.2

0.3

0.4

0.5

0.6

0 10 20 30 40 50 60 70 80

SF-36 items

CAT 10 itemsFull Item Bank

measurement precision (standard error)

normed theta values

HAQ items

SF-12 items

representativesample

rheumatoid arthritis patients

US-Representative Sample

4

5

3

2

1

10 20 30 40 50 60 70

Self-Reported

Health

Social Health

Mental Health

Physical Health

Physical Health

Function

Symptoms

Behavior

Interference

Quality

Intensity

Physical Function

Upper Extremity

GI Symptoms

Pain

Fatigue

Asthma Impact

Sexual Function

Sleep Function

Physical Activity

Mobility

Sleep Disturbance

Sleep-related Impairment

Satisfaction

Interest

LubricationVaginal

DiscomfortErectile FunctionOrgasm

Interfering Factors

Therapeutic Aids

Sexual Activities

Anal Discomfort

Self-Reported

Health

Social Health

Mental Health

Physical Health

Mental Health

Behavior

Cognition

Affect

Anxiety

Depression

AngerExperience of

Stress

Substance Abuse

Applied Cognition –

General Concerns

Self-efficacy

Applied Cognition –

Abilities

Positive

Negative

Illness Impact - Neg

Illness Impact - Pos

Subjective Well-being

Alcohol Use

Self-Reported

Health

Social Health

Mental Health

Physical Health

Social Health

Function

Relationships

Ability to Participate

Family Belongingness

Social Isolation

Quality Social Support

Peer Relationships

Satisfaction w Participation

Social RolesDiscretionary

Activit.

Self-Reported

Health

Social Health

Mental Health

Physical Health

Domains# Items

Adult Bank

# Items Adult ShortForms

# Items Peds Bank

# Items Peds Short

FormEmotional Distress –

Anger29 8 6

Emotional Distress – Anxiety

29 4, 6, 7, 8 15 8

Emotional Distress – Depression

28 4, 6, 8a, 8b 14 8

Fatigue 95 4, 6, 7, 8 23 10

Pain – Behavior 39 7

Pain – Interference 41 4, 6a, 6b, 8 13 8

Physical Function 124 4, 6, 8, 10, 20

-- Mobility 23 8

-- Upper Extremity 29 8

Asthma Impact 17 8

PROMIS Domains in AC, 2010

Domains# Items

Adult Bank

# Items Adult ShortForms

# Items Peds Bank

# Items Peds Short

FormsSatisfaction with

Discretionary Social Activities

12 7

Satisfaction with Social Roles

14 4, 6, 7, 8

Peer Relationships 15 8

Sleep Disturbance 27 4, 6, 8a, 8b

Sleep-Related Impairment

16 8

Global Health 10

PROMIS Domains in AC, 2010

Domains PROMIS-29 PROMIS-43 PROMIS-57Emotional Distress –

Anxiety4 6 8

Emotional Distress – Depression

4 6 8

Fatigue 4 6 8

Pain – Interference 4 6 8

Pain – Intensity 1 1 1

Satisfaction with Social Role

4 6 8

Sleep Disturbance 4 6 8

Physical Function 4 6 8

2010 PROMIS Profile Instruments

PROMIS Profiles

Anxiety

Depression

Fatigue

Pain InterferenceSleep DisturbancePhysical Function

Social Role

46

8

Mental

Physical

Social

Thank You

www.nihpromis.org

Reliability and SEM

• For z-scores (mean = 0 and SD = 1):– Reliability = 1 – SEM2

= 0.91 (when SEM = 0.30)= 0.90 (when SEM = 0.32)

• With 0.90 reliability– 95% Confidence Interval

• z-score: - 0.62 0.62• T-score = (z-score * 10) + 50

• T-score: 44 56

35