“Creating Composite Measures Using Factor Analysis:

The Total Illness Burden Index

Sherrie H. Kaplan, PhD, MPH

Professor of Medicine

UC Irvine School of Medicine

Academy Health ARM

June 8-10, 2008

Some Background…

Role of Purpose of Measurement

• Changes content of aggregate measure

• Changes tolerance of error

• Changes psychometric requirements of aggregate

• Changes ‘level of confidence’, dissemination strategy

How to create composites:Lessons from psychometrics…

• Choose measures that Choose measures that broadly broadly represent represent underlying (latent) construct (sampling underlying (latent) construct (sampling from domain of observables); each item from domain of observables); each item adds unique informationadds unique information

• Hypothesize structure of items in Hypothesize structure of items in composites before analysis (what composites before analysis (what measures what?)measures what?)

How to create composites:Lessons from psychometrics…

• Conduct Conduct confirmatory confirmatory cluster, latent cluster, latent variable analyses (construct validity)variable analyses (construct validity)

• Decide on scoring methods (simple Decide on scoring methods (simple algebraic sum, weighting, conjunctive or algebraic sum, weighting, conjunctive or compensatory); compensatory); testtest scoring methods scoring methods

• Test reliability, predictive validity of Test reliability, predictive validity of derived compositederived composite

Models for Composite Scoring

• Conjunctive scoring (‘ands’): highest, lowest levels achieved define score– Rheumatoid arthritis trials: patient responded if:

• at least a 20% improvement in tender joint count and • 20% improvement in swollen joint count and • at least 20% improvement in 3 out of 5 of the

following: pain assessment, global assessment, physician assessment, etc.

• Compensatory scoring (‘ors’): high scores on one component make up for low scores on another

Models for weighting• Expert defined

– Conditioned by ‘expert’ representation

• Regression-based– Conditioned by database (provider,

patient sample, sample size)

• Factor analysis-based– Conditioned by variables included in

factor analysis

• Reliability-based– Conditioned by database (sample size)

Classic Measurement Theory: Using Factor Analysis to Create Composites

• Each factor represents ‘latent’ construct

• Correlations of items with factors (factor ‘loadings’) represent statistical structure of set of variables

• Factor analysis does not require items have difficulty structure

                                                                                            ,

Cronbach’s alpha

• Measure of internal consistency reliability

• Given by formula:

– Where:• N = number of tests

• σYi2 = variance of item i

• σx2 = total test variance

                                                                                            ,

Cronbach’s alpha

• Alpha is unbiased reliability estimator if items have equal covariances (means and item variances may differ); i.e. have common factor in factor analysis

Total Illness Burden: The Latent Construct

• Patient-reported composite measure of severity of multiple diseases

• Taken together represent increasing risk for substantial declines in health and increased risk for mortality (1-5 years post initial observation)

Purposes of Measurement

• Post hoc case-mix adjustment

• A priori risk stratification of clinical trials

• Improve clinical decision making for ‘tailoring’ treatment

Subdimensions…

• Pulmonary disease• Heart disease• Stroke and neurologic disease• Gastrointestinal conditions• Other cancers (excluding prostate)• Arthritis• Foot and leg conditions

Subdimensions (cont’)

• Eye and vision conditions• Hearing problems• Hypertension • Diabetes

Sample Questions: COPD

1. During the past 6 months, how often did you have wheezing?

a. Never

b. Once or twice

c. About once a week

d. Several times a week

e. Several times a day

Sample Questions: COPD

4. During the past 6 months, did you use extra pillows in order to sleep at night because of problems with your breathing?

a. No

b. Yes, 1 pillow

c. Yes, 2 pillows

d. Yes, 3 or more pillows

Steps in Constructing Subdimensions

• Transformed variables to uniform metric by clinical definition of severity

• Tested reliability of clinically defined scale (Cronbach’s alpha > .70)

• Created composite of each subdimension using simple algebraic sum, mean

• Items in each subdimension varied

• Validated each subdimension as scale using SF-36, etc.

Steps in Constructing Composite

• Conducted principal components analysis, higher order factor analysis using scales as entries

• First factor explained 67% of variance

• Other factors had Eigen values, scree indicating single factor solution

• Factor loadings ranged from .40 - .70

• Used factor loadings to create composite

• Validated derived composite

Understanding and Reducing Disparities in Diabetes Care:Coached Care for Diabetes

Sherrie H. Kaplan, PhD, MPH

Sheldon Greenfield, MD

NovoNordisk

Lund, Sweden

May 28-20, 2008

Characteristics of Patient SampleCharacteristics Registry

(n=3,894)Survey Sample

(n=1001)

Mean age 58.9 60.1

% Male 43.2 39.3

% White 33.1 25.8

% Hispanic 50.5 48.9

% Asian 16.4 25.3

% Medicare 21.4 20.9

% Medicaid 50.7 54.1

% Commercial 19.2 16.2

% Uninsured 8.7 8.7

Principal Components Analysis: First Factor

TIBI Scale Sample 1 Sample 2GI disease .604 .628

Atherosclerotic heart dis .650 .621

Neurologic problems .433 .360

Hearing problems .452 .388

Hypertension .328 .448

Cardiopulmonary .712 .704

Feet problems .613 .587

Arthritis .449 .618

Vision problems .475 .367

Cronbach’s alpha (.799)

TIBI Scale Scale Mean α if item deleted

GI disease .58 .767Atherosclerotic heart dis .79 .783Neurologic problems .19 .799Hearing problems .36 .798Hypertension .74 .799Cardiopulmonary .81 .755Feet problems .67 .787Arthritis .57 .790Vision problems .51 .798

Correlation of TIBI with Patient-Reported Health Status Measures by Ethnic Group

Health Status Measures

Whites Mexican-American

Vietnamese

SF-36 PFI10 -.63 -.38 -.55CESD .50 .50 .55Diabetes Burden .37 .33 .37

Other TIBI Validation Studies…

Preventing Cardiovascular Disease: Identification of Co-Morbidity Subgroups who may not Benefit from Aggressive Diabetes Management

Greenfield S, Nicolucci A, Pellegrini F, Kaplan SH

QuED Study

• Prospective cohort study of consecutively enrolled patients with diabetes who completed TIBI at enrollment in Italian Quality of Care and Outcomes in Type 2 Diabetes Study Group (n=2,613)

QuED Study

Patient Characteristics

Mean age 62.7 [10.3]

% Female 45

% < 5 yrs education 52

% BMI > 30 28

HbA1c value 7.2

QuED: Total 5-yr CV Events by TIBI

TIBI Group HR 95% HR CI P-value

0-3 1

3-6 0.95 0.61-1.47 .81

6-9 1.11 0.74-1.67 .61

9-12 1.46 1.02-2.1 .04

>12 1.57 1.16-2.12 .003

QuED: % 5-yr Survival by TIBI

TIBI Group % HR (95% CI) P-value

0-3 85.9 1

3-6 85.0 1.11(.75-1.65) .59

6-9 80.2 1.42(1.00-2.02) .05

9-12 80.0 1.41(0.96-2.08) .08

>12 75.6 1.63(1.25-2.13) <.000

“Complex diabetes patients, those with the greatest burden from competing co-morbidities (highest TIBI scores) may benefit less from aggressive glycemic control due to their increased risk of mortality from other causes before those benefits could be realized.”

Conclusions

• Using factor analysis, possible to derive latent construct that reflects patients’ “total illness burden”

• Potentially useful in case-mix adjustment, clinical trials design, clinical decision making

• Future research aimed at improving sensitivity, specificity, particularly at ‘intermediate ranges’