1

Variable Selection for Tailoring Treatment

S.A. Murphy, L. Gunter & J. Zhu

May 29, 2008

2

Outline

• Motivation

• Need for Variable Selection

• Characteristics of a Tailoring Variable

• A New Technique for Finding Tailoring Variables

• Comparisons

• Discussion

3

Motivating ExampleSTAR*D "Sequenced Treatment to Relieve Depression"

Preference Treatment Intermediate Preference Treatment Intermediate Treatment Two Outcome Three Outcome Four

Follow-up Follow-up

CIT + BUS Remission L2-Tx +THY Remission

Augment R Augment RTCP

CIT + BUP L2-Tx +LI

CIT Non- Non- Rremission remission

BUP MIRTMIRT + VEN

Switch R Switch RVEN

SER NTP

30+ baseline variables, 10+ variables at each treatment level, both categorical and continuous

4

Simple Example

Nefazodone - CBASP Trial

Randomization

Nefazodone

Nefazodone + Cognitive Behavioral Analysis System of Psychotherapy (CBASP)

50+ baseline covariates, both categorical and continuous

5

Simple Example

Nefazodone - CBASP Trial

Which variables in X are important for choosing the optimal treatment?

Xpatient’s medical history, severity of depression, current symptoms, etc.

A Nefazodone OR Nefazodone + CBASP

R depression symptoms post treatment

6

Need for Variable Selection

• In clinical trials many pretreatment variables are collected to improve understanding and inform future treatment

• Yet in clinical practice, only the most informative variables for tailoring treatment can be collected.

• A combination of theory, clinical experience and statistical variable selection methods can be used to determine which variables are important in tailoring.

7

Current Statistical Variable Selection Methods

• Current statistical variable selection methods focus only on finding good predictors of the response

• Also need variables to help determine which treatment is best for individual patients, e.g. tailoring variables

• Experts typically have knowledge on which variables are good predictors, but intuition about tailoring variables is often lacking

8

What is a Tailoring Variable?

• Tailoring variables help us determine which treatment is best

• Tailoring variables qualitatively interact with the treatment; different values of the tailoring variable result in different best treatments.

No Interaction Non-qualitative Interaction Qualitative interaction

0.0 0.4 0.8

0.0

0.4

0.8

X1

R

A=1

A=0

0.0 0.4 0.8

0.0

0.4

0.8

X2

R

A=1

A=0

0.0 0.4 0.8

0.0

0.4

0.8

X3R

A=0

A=1

9

Qualitative Interactions

• We focus on two important factors– The magnitude of the interaction between the

tailoring variable and the treatment indicator– The proportion of patients for whom the best choice

of treatment changes given knowledge of the variable

big interaction small interaction big interaction

big proportion big proportion small proportion

0.0 0.4 0.8

0.0

0.4

0.8

X4

R

A=0

A=1

0.0 0.4 0.8

0.0

0.4

0.8

X5

R

A=0

A=1

0.0 0.4 0.8

0.0

0.4

0.8

X6R

A=0

A=1

10

0.0 0.4 0.8

0.0

0.4

0.8

Xj

R

A=0

A=1=a*

Magnitude of the Interaction

• We estimate magnitude factor by: Dj = change in the effect of the best treatment a*=1 over the range of variable Xj

maximum effect oftreatment a* on R

Dj = max effect – min effect

minimum effect of treatmenta* on R

11

Proportion

• We estimate the proportion factor by: Pj = percentage of patients in the sample whose

best treatment changes when variable Xj is considered

Treatment A=0 is best for 2 out of 7 subjects even though treatment A=1 is best overall

0.0 0.4 0.8

0.0

0.4

0.8

Xj

R

A=0

A=1=a*

2

7jP

12

Ranking Score U

• We combine D and P to make a score U for each X pretreatment variable.

• Variables are ranked by their score, U; higher U’s correspond to higher evidence of a qualitative interaction by the X variable.

• We use this ranking in a variable selection algorithm to select important tailoring variables.

13

Variable Selection Algorithm

1. Select important predictors of R from X using a predictive variable selection method (reducing noise in R)

2. Rank interactions between X and A using score U, select all with nonzero U.

3. Construct a combined ranking of variables selected in steps 1 and 2

4. Choose between variable subsets using a criterion that trades off number of variables and estimated maximal response due to tailoring.

14

Simulations

• Data simulated under wide variety of realistic decision making scenarios (with and without qualitative interactions)

• Compared:– Ranking method, U, using variable selection algorithm

– Standard technique: Lasso on (X, A, XA)

• 1000 simulated data sets: recorded percentage of time each variable’s interaction with treatment was selected for each method

15

Simulation Results

× Binary Qualitative Interaction Non-qualitative Interaction Spurious Interaction

× Continuous Qualitative Interaction Non-qualitative Interaction Spurious Interaction

0 20 40 60

02

06

0

Standard Method

variable number

% o

f tim

e c

ho

sen

0 20 40 60

02

06

01

00

New Method

variable number

% o

f tim

e c

ho

sen

0 20 40 60

02

04

06

08

0

Standard Method

variable number

% o

f tim

e c

ho

sen

0 20 40 60

02

04

06

0

New Method

variable number

% o

f tim

e c

ho

sen

16

Simulation Results

Generative

Model

Ave(# of Spurious Interactions Selected)

Standard

Method

New

Method

One Binary Qualitative Interaction

Four Non-qualitative Interactions5.59 0.92

One Continuous Qualitative Interaction

Four Non-qualitative Interactions6.44 0.01

17

Nefazodone - CBASP Trial

Aim of the Nefazodone CBASP trial – to compare efficacy of three alternate treatments for major depressive disorder (MDD):1. Nefazodone, 2. Cognitive behavioral-analysis system of

psychotherapy (CBASP) 3. Nefazodone + CBASP

Which variables might help tailor the depression treatment to each patient?

18

Nefazodone - CBASP Trial

• For our analysis we used data from 440 patients with

X64 baseline variables

A Nefazodone vs. Nefazodone + CBASP

R

Hamilton’s Rating Scale for Depression score, post treatment

19

Nefazodone - CBASP Trial

• Used bootstrap samples to produce a selection percentage for each variable.

• Permutated the rows of the X*A matrix to produce thresholds. The highest ranked spurious interaction is less than the 80% threshold in 80% of repeated permutations.

20

Nefazodone - CBASP Trial

0 20 40 60

02

04

06

0

Standard Method

variable number

% o

f tim

e c

ho

sen

0 20 40 600

10

30

New Method

variable number

% o

f tim

e c

ho

sen

21

Discussion• This method provides a list of potential

tailoring variables while reducing the number of false leads.

• Replication is required to confirm the usefulness of a tailoring variable.

• Our long term goal is to generalize this method so that it can be used with data from Sequential, Multiple Assignment, Randomized Trials as illustrated by STAR*D.

22

• Email Susan Murphy at samurphy@umich.edu for more information!

• This seminar can be found at http://www.stat.lsa.umich.edu/~samurphy/seminars/SPR0508.ppt

• Support: NIDA P50 DA10075, NIMH R01 MH080015 and NSF DMS 0505432

• Thanks for technical and data support go to– A. John Rush, MD, Betty Jo Hay Chair in Mental Health at the

University of Texas Southwestern Medical Center, Dallas– Martin Keller and the investigators who conducted the trial `A

Comparison of Nefazodone, the Cognitive Behavioral-analysis System of Psychotherapy, and Their Combination for Treatment of Chronic Depression’

23

Interacion Plot

Alcohol Dependence

Fitte

d R

25

30

35

0 1

Txt=Combo

Txt=Nef

Interaction Plot

24

Interacion Plot

Obsessive Compulsive Disorder

Fitte

d R

10

15

20

25

30

0 1

Txt=Combo

Txt=Nef

Interaction Plot