adaptive interventions and smart methods: applications in...

Adaptive Interventions and SMART Methods:Applications in Child and Adolescent Mental

Health

Daniel Almirall1,2 Xi Lu (Lucy)1,2,3

Inbal (Billie) Nahum-Shani1,2

Susan A. Murphy1,2,3

1Institute for Social Research, University of Michigan2The Methodology Center, Penn State University3Department of Statistics, University of Michigan

Rush University - April-21-2015

Almirall, Xu, Nahum-Shani, Murphy Getting SMART 1 / 57

Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)

SMART Design PrinciplesTake Home Points

SMART Case Studies

Outline: 37 slides + discussion (hopefully lots!)

Adaptive InterventionsWhat? Why?

Sequential Multiple Assignment Randomized Trial (SMART)What are SMARTs?

SMART Design PrinciplesKeep it SimpleChoosing Primary and Secondary Hypotheses

Take Home Points

SMART Case Studies




SMART Case Studies

What? Why?

Adaptive Interventions




SMART Case Studies

What? Why?

Definition: A Adaptive Intervention is

I a sequence of individually tailored decision rulesI that specify whether, how, or whenI and based on which measuresI to alter the dosage (duration, frequency or amount), type,

or deliveryI at critical decision points in the course of care.

Adaptive Interventions (AIs) help guide the type of sequentialtreatment decision making that is typical (and often needed!) ofclinical practice.




SMART Case Studies

What? Why?

Concrete Example of an Adaptive InterventionChild ADHD in Schools, Ages 6-12

I Responder status measured by school-teacher.I Goal is to min. symptoms / max. school performance.




SMART Case Studies

What? Why?

Concrete Example of an Adaptive InterventionChild ADHD in Schools, Ages 6-12

I What does it look like from researcher’s/clinician’s POV?I What does it look like from the child’s/parent’s POV?




SMART Case Studies

What? Why?

What are the parts of an Adaptive Intervention?1. Critical decision points: based on time or other measures2. Treatment options at each stage3. Tailoring variables: to decide how to adapt treatment4. Decision rules: inputs tailoring variable, outputs treatments

aka: dynamic treatment regimens, adaptive txt strategies, txt algorithms,medication algorithms, stepped care, txt policies, multi-stage strategies...




SMART Case Studies

What? Why?

Why Adaptive Interventions?Necessary...

I Nature of chronic disorders/phenomena (substance use,mental health, autism, diabetes, cancer, HIV/AIDS)

I Waxing and waning course (multiple relapse, recurrence)I Life events, comorbidities, non-adherence may arise

I Disorders for which there is no widely effective treatment.

I Disorders for which there are widely effective treatments,but they are costly or burdensome.

I Bottom line: High heterogeneity in response to treatmentI Within person (over time) and between person

All require sequences of treatment decisions!




SMART Case Studies

What? Why?

Ok, so adaptive interventions are great, but......there are so many unanswered questions.

Next, we’ll talk research...but first...




SMART Case Studies

What are SMARTs?

SEQUENTIAL MULTIPLE ASSIGNMENTRANDOMIZED TRIALS (SMARTs)




SMART Case Studies

What are SMARTs?

What is a Sequential Multiple AssignmentRandomized Trial (SMART)?

I Multi-stage trials; same participants throughoutI Each stage corresponds to a critical decision pointI At each stage, subjects randomized to set of treatment

optionsI The goal of a SMART is to inform the development of

adaptive interventions.

I will give you an example SMART, but first...




SMART Case Studies

What are SMARTs?

Background for an Example SMARTADHD Treatment in Children Ages 6-12

I Both medication (MED) and behavioral modification(BMOD) have been shown to be efficacious

I However, there is much debate on whether first-lineintervention should be pharmacological of behavioral,especially in younger children

I Further, there is a need for a ”rescue treatment” if the firsttreatment does not go well because 20-50% of children donot substantially improve on BMOD or MED

I So important questions for clinical practice include“What treatment do we begin with: BMOD or MED?””Among non-responders, what second treatment is best?”


Concrete Example of a SMART: Child ADHDPI: William Pelham, PhD, Florida International University, IES-Funded GrantN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)

One of Four Adaptive Interventions Within the SMART

4 Embedded Adaptive Interventions in this SMART



SMART Case Studies

Keep it SimpleChoosing Primary and Secondary Hypotheses

SMART DESIGN PRINCIPLES




SMART Case Studies


SMART Design Principles

I KISS Principle: Keep It Simple, Straightforward

I Power for simple important primary hypotheses

I Take Appropriate steps to develop a moredeeply-individualized (optimized) Adaptive Intervention




SMART Case Studies


Keep It Simple, StraightforwardOverarching Principle

At each stage, or critical decision point,...I Restrict class of treatment options only by ethical,

feasibility, or strong scientific considerations

I If you do restrict randomizations, use low dimensionalsummary to restrict subsequent treatments

I Use binary responder statusI Should be easy to use in actual clinical practice

I Collect additional, auxiliary time-varying measuresI To develop a more deeply-tailored Adaptive InterventionI Think time-varying effect moderators




SMART Case Studies


SMART Design: Primary Aims

Choose a simple primary aim/question that aids developmentof an adaptive intervention.

Statistical methods used here aim to reduce uncertainty so theinvestigator can come away with a solid answer.

Sample size for the SMART chosen based on the hypothesistest associated with this aim (e.g., use standard α = 5%).


Primary Aim Example 1What is the effect of starting with BMOD vs MED on longitudinal outcomes?

PowerES N0.8 340.5 830.2 505ρ = 0.60α = 0.05β = 0.20



SMART Case Studies


SMART Design: Secondary Aims

Choose secondary aims/questions that further develop theAdaptive Intervention and take advantage of sequentialrandomization to eliminate confounding.

Statistical methods used here aim to generate hypotheses,e.g., generate good hypotheses about additional tailoringvariables or moderators.

Here, investigators will tolerate hypothesis tests with higherType-I error, e.g., α = 10%.


Secondary Aim Example 1Among non-responders, is it better to INTENSIFY vs AUGMENT?On various occassions, I have seen this be the Primary Aim.

Secondary Aim Example 2Is there a difference between two of the embedded adaptive interventions?This could also be a Primary Aim.

Sample size calculators exist for this; see Oetting, Levy, Weiss,and Murphy 2011. Zhiguo Li at Duke. Kelley Kidwell at UMich.

Secondary Aim Example 3Build a more deeply tailored adaptive intervention (go beyond the 4 embedded adaptiveinterventions). Rarely, would this be a Primary Aim.



SMART Case Studies

TAKE HOME POINTS




SMART Case Studies

Take Home the Following

I Adaptive Interventions individualize treatment up-front andthroughout; they are guides for clinical practice

I SMARTs are used to build better Adaptive InterventionsI Next study: RCT of SMART-optimized AI vs control

I SMARTs are not adaptive trial designs (confusing!)

I SMARTs do not have to be complicated

I SMARTs do not necessarily require larger sample sizes




SMART Case Studies

SMART CASE STUDIES


Autism SMART (N = 61)PI: Kasari (UCLA). (ages 5-8; planned N = 98 but recruitment difficult, despitemulti-site. Wk12 response rates much higher than anticipated.)

Longitudinal Analysis of the Autism SMARTYt = Socially communicative utterances over 36 weeks

AI Estimate 95% CI(AAC,AAC+) 51.4 [45.6, 57.3](JASP,AAC) 40.7 [34.5, 46.8]

(JASP,JASP+) 39.3 [32.6, 46.0]

Moderators Analysis of the Autism SMART

2040

6080

100

Baseline SCU < 40 (75%)

Time (weeks)

Soc

ially

Com

mun

icat

ive

Utte

ranc

es

0 12 24 36

(AAC, AAC+)(JASP, AAC)(JASP, JASP+)

2040

6080

100

Baseline SCU ≥ 40 (25%)

Time (weeks)

Soc

ially

Com

mun

icat

ive

Utte

ranc

es

0 12 24 36

(AAC, AAC+)(JASP, AAC)(JASP, JASP+)

Child ADHD SMARTPI: William Pelham, PhD, Florida International UniversityN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)

Longitudinal Analysis of the ADHD SMARTYt = Classroom performance over 8 months (school year)

Time (months)

Cla

ssro

om p

erfo

rman

ce

2.1

2.2

2.3

2.4

2.5

2.6

2.7

1 2 3 4 5 6 7 8

DTRBMOD.INTBMOD.MEDMED.BMODMED.INT

AI Color(MED, MED+) Purple

(MED, MED+BMD) Blue(BMD,BMD+MED) Red

(BMD,BMD+) Green

Interventions for Minimally Verbal Children with AutismPI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)

Interventions for Minimally Verbal Children with AutismPI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)

Non-Responders

(Parent training no

feasible) JASP (joint

attention and social play) Continue JASP

JASP + Parent Training R

DTT (discrete trials training)

Continue DTT

DTT + Parent Training

Responders

(Blended txt

unnecessary)

R

Non-Responders

(Parent training not

feasible)

Responders

(Blended txt

unnecessary)

R

JASP + DTT

Continue JASP

R

JASP + DTT

Continue DTT

R

Adaptive Implementation Intervention in Mental HealthPI: Kilbourne; Co-I: Almirall (Aim is to improve the uptake of a psychosocial interventionfor mood disorders)

BestFit Weight Loss StudyPI: Sherwood, University of Minnesota (N = 500, anticipated)

Treatment for Alcohol DependencePI: Oslin, University of Pennsylvania

Early Trigger for NR: 2+ HDD CBI

CBI + Naltrexone

R

Late Trigger for NR: 5+ HDD

CBI

CBI + Naltrexone

Non-Response R

Non-Response R

Naltrexone

TDM + Naltrexone

8 Week Response R

Naltrexone

TDM + Naltrexone

8 Week Response R



SMART Case Studies

Thank you! Questions?

Email me with questions about this presentation:I Daniel Almirall: [email protected]

Find papers on SMART:I http://www.lsa.stat.umich.edu/∼samurphy/ (Susan Murphy)

More papers and these slides on my website (Daniel Almirall):I http://www-personal.umich.edu/∼dalmiral/




SMART Case Studies

EXTRA SLIDES[beyond this point, slides may not be coherent]




SMART Case Studies

GENERATING HYPOTHESES vs BUILDING vs EVALUATING

ADAPTIVE INTERVENTIONS?




SMART Case Studies

3 Different Research Questions/Aims⇒ 3 Different Research Designs

I Aim Type 1: When generating hypotheses about anAdaptive Intervention

I Aim Type 2: When building an Adaptive Intervention

I Aim Type 3: When evaluating a particular AdaptiveIntervention



Some example questions:

I Aim Type 1 (Hypothesis Gen.) Example: Doesaugmenting txt for non-responders (as observed in aprevious trial) correlate with better outcomes?

I Aim Type 2 (Building) Example: What are the besttailoring variables or decision rules?

I Aim Type 3 (Evaluating) Example: Does an adaptiveintervention have a statistically and clinically signif. effectas compared to control intervention?


I Aim Type 1: When generating hypotheses about anAdaptive Intervention

I Aim Type 2: When building an Adaptive Intervention

I Aim Type 3: When evaluating a particular AdaptiveIntervention

Obs. Study Exp. Studye.g., analysis of e.g., e.g.,

Type Aim previous RCT SMART RCT1 Hypothesis Gen. YES ≈ ∼2 Building ≈ YES ≈3 Evaluating ∼ ≈ YES



SMART Case Studies

A (potentially not-so) SMART Alternative


Back to the idea of generating hypotheses vs buildingvs evaluating adaptive interventions...

Obs. Study Exp. Studye.g., analysis of e.g., e.g.,

Type Aim previous RCT SMART RCT1 Hypothesis Gen. YES ≈ ∼2 Building ≈ YES ≈3 Evaluating ∼ ≈ YES

Why not use multiple RCTs to build an adaptive intervention?

Concrete Example of a SMART: Child ADHDPI: William Pelham, PhD, Florida International University, IES-Funded GrantN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)



SMART Case Studies

Why Not Use Multiple RCTs to Construct an AI?Three Concerns about Using Multiple Trials as an Alternative to a SMART

1. Concern 1: Delayed Therapeutic Effects

2. Concern 2: Diagnostic Effects

3. Concern 3: Cohort (Sample Selection) Effects

All three concerns emanate from the basic idea thatconstructing an adaptive intervention based on a ”local”study-to-study point of view may not be optimal.




SMART Case Studies

Hypothesis-generating Observational StudiesPost-hoc Analyses Useful for Building Adaptive Interventions

I Give examples of different observational study questionsthey can examine using data from a previous 2-arm RCT

I Standard observational study caveats apply:I No manipulation usually means lack of heterogeneity in txt

options (beyond what is controlled by experimentation inoriginal RCT)

I Some RCTs use samples that are too homogeneousI Confounding by observed baseline and time-varying factorsI Unobserved, unknown, unmeasured confounding by

baseline and time-varying factors




SMART Case Studies

Hypothesis-generating Observational StudiesPost-hoc Analyses Useful for Building Adaptive Interventions

I There exists a literature for examining the impact oftime-varying treatments in observational studies

I Marginal Structural Models (Robins, 1999; Bray, Almirall, etal., 2006) to examine the marginal impact of observedtime-varying sequences of treatment

I Structural Nested Mean Models (Robins, 1994; Almirall, etal., 2010, 2011) to examine time-varying moderators ofobserved time-varying sequences of treatment

I Marginal Mean Models (Murphy, et al., 2001): to examinethe impact of observed adaptive interventions




SMART Case Studies

Early precursors to SMART

I CATIE (2001) Treatment of Psychosis in Patients withAlzheimer’s

I CATIE (2001) Treatment of Psychosis in Patients withSchizophrenia

I STAR*D (2003) Treatment of Depression




SMART Case Studies

Other Alternatives

I Piecing Together Results from Multiple TrialsI Choose best first-line treatment on the basis of a two-arm

RCT; then choose best second-line treatment on the basisof another separate, two-arm RCT

I Concerns: delayed therapeutic effects, and cohort effects

I Observational (Non-experimental) Comparisons of AIsI Using data from longitudinal randomized trialsI May yield results that inform a SMART proposalI Understand current treatment sequencing practicesI Typical problems associated with observational studies

I Expert Opinion




SMART Case Studies

Why Not Use Multiple Trials to Construct an AIThree Concerns about Using Multiple Trials as an Alternative to a SMART

1. Concern 1: Delayed Therapeutic Effects

2. Concern 2: Diagnostic Effects

3. Concern 3: Cohort Effects

All three concerns emanate from the basic idea thatconstructing an adaptive intervention based on a myopic, local,study-to-study point of view may not be optimal.




SMART Case Studies

Why Not Use Multiple Trials to Construct an AIConcern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions

Positive Synergy Btwn First- and Second-line Treatments

Tapering off medication after 12 weeks of use may not appearbest initially, but may have enhanced long term effectivenesswhen followed by a particular augmentation, switch, ormaintenance strategy.

Tapering off medication after 12 weeks may set the child up forbetter success with any one of the second-line treatments.




SMART Case Studies

Why Not Use Multiple Trials to Construct an AIConcern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions

Negative Synergy Btwn First- and Second-line Treatments

Keeping the child on medication an additional 12 weeks mayproduce a higher proportion of responders at first, but may alsoresult in side effects that reduce the variety of subsequenttreatments available if s/he relapses.

The burden associated with continuing medication an additional12 weeks may be so high that non-responders will not adhereto second-line treatments.




SMART Case Studies

Why Not Use Multiple Trials to Construct an AIConcern 2: Diagnostic Effects

Tapering off medication after 12 weeks initial use may notproduce a higher proportion of responders at first, but may elicitsymptoms that allow you to better match subsequent treatmentto the child.

The improved matching (personalizing) on subsequenttreatments may result in a better response overall as comparedto any sequence of treatments that offered an additional 12weeks of medication after the initial 12 weeks.




SMART Case Studies

Why Not Use Multiple Trials to Construct an AIConcern 3: Cohort Effects

I Children enrolled in the initial and secondary trials may bedifferent.

I Children who remain in the trial(s) may be different.I Characteristics of adherent children may differ from study

to study.I Children that know they are undergoing adaptive

interventions may have different adherence patterns.

Bottom line: The population of children we are makinginferences about may simply be different from study-to-study.


adaptive interventions and smart methods: applications in...

Documents