Characteristics Associated WithDecreased or Increased MortalityRisk From Glycemic TherapyAmong Patients With Type 2Diabetes and High CardiovascularRisk:Machine LearningAnalysis ofthe ACCORD Trialhttps://doi.org/10.2337/dc17-2252

OBJECTIVE

Identifying patients whomay experience decreased or increased mortality risk fromintensive glycemic therapy for type 2 diabetes remains an important clinical chal-lenge. We sought to identify characteristics of patients at high cardiovascular riskwith decreased or increasedmortality risk from glycemic therapy for type 2 diabetesusing newmethods to identify complex combinations of treatment effect modifiers.

RESEARCH DESIGN AND METHODS

The machine learning method of gradient forest analysis was applied to understandthe variation in all-causemortalitywithin theAction to Control Cardiovascular Risk inDiabetes (ACCORD) trial (N = 10,251), whose participants were 40–79 years old withtype 2 diabetes, hemoglobin A1c (HbA1c) ‡7.5% (58 mmol/mol), cardiovascular dis-ease (CVD) or multiple CVD risk factors, and randomized to target HbA1c <6.0%(42 mmol/mol; intensive) or 7.0–7.9% (53–63 mmol/mol; standard). Covariates in-cluded demographics, BMI, HbA1c glycation index (HGI; observed minus expectedHbA1c derived from prerandomization fasting plasma glucose), other biomarkers,history, and medications.

RESULTS

The analysis identified four groups defined by age, BMI, and HGI with varied risk formortality under intensive glycemic therapy. The lowest risk group (HGI <0.44,BMI <30 kg/m2, age <61 years) had an absolute mortality risk decrease of 2.3%attributable to intensive therapy (95% CI 0.2 to 4.5, P = 0.038; number needed totreat: 43), whereas the highest risk group (HGI ‡0.44) had an absolute mortality riskincrease of 3.7% attributable to intensive therapy (95% CI 1.5 to 6.0; P < 0.001;number needed to harm: 27).

CONCLUSIONS

Age, BMI, and HGI may help individualize prediction of the benefit and harm fromintensive glycemic therapy.

1Center for Primary Care and Outcomes Research,Center for Population Health Sciences, Depart-ments of Medicine and of Health Research andPolicy, Stanford University, Palo Alto, CA2Harvard Medical School, Boston, MA3Department of Veterans Affairs Eastern Colo-rado Healthcare System, Denver, CO4Division of General Internal Medicine, Depart-ment of Medicine, University of Colorado Schoolof Medicine, Denver, CO5Diabetes Unit, Massachusetts General Hospital,Boston, MA6Division of General Internal Medicine, Massa-chusetts General Hospital, Boston, MA

Corresponding author: Sanjay Basu, basus@stanford.edu.

Received 26 October 2017 and accepted 5December 2017.

This article contains Supplementary Data onlineat http://care.diabetesjournals.org/lookup/suppl/doi:10.2337/dc17-2252/-/DC1.

© 2017 by the American Diabetes Association.Readers may use this article as long as the workis properly cited, the use is educational and notfor profit, and the work is not altered. More infor-mation is available at http://www.diabetesjournals.org/content/license.

Sanjay Basu,1,2 Sridharan Raghavan,3,4

Deborah J. Wexler,2,5 and

Seth A. Berkowitz2,5,6

Diabetes Care 1

CARDIOVASCULA

RANDMETA

BOLIC

RISK

Diabetes Care Publish Ahead of Print, published online December 26, 2017

Individualizing the glycemic target forpatients with type 2 diabetes is now theguideline-recommended strategy (1), buthow best to individualize glycemic targetsremains unclear. A major reason for cau-tion regarding intensive glycemic targetsis the Action to Control CardiovascularRisk in Diabetes (ACCORD) trial (N =10,251, conducted 2001–2009) (2), whichwas halted due to increased all-causemortality in the intensive therapy arm.ACCORD targeted nearly normal glycemiclevels in the intensive glycemic therapyarm, achieving a median hemoglobin A1c(HbA1c) of 6.4% (46 mmol/mol), com-pared with an achieved HbA1c of 7.5%(58 mmol/mol) in the standard therapyarm.Meta-analyses of data fromACCORDand other trials find that microvascularevents are reduced with intensive glyce-mic control (3), but the lack of overallmortality benefit in trials as well as theincreased mortality observed in ACCORDrenders uncertain the risk-to-benefit cal-culation in any given patient.Although current guidelines donot rec-

ommend targets as low as those used inACCORD, real-world evidence suggestsmany patients are treated with multidrugregimens to levels achieved in the inten-sive therapy armof ACCORD (4–6). There-fore, understanding the heterogeneoustreatment effects (HTEs) of intensive gly-cemic therapy with regard to mortality isimportant. Specifically, which subgroupof patients in ACCORD was most likelyto experience increased mortality? Con-versely, because some patients do derivecardiovascular benefit from glycemictherapy (7), did any subgroups in ACCORDexperience benefit? Unfortunately, uni-variable subgroup analyses of the trialdata have been unable to explain the ma-jor variations inexcessmortality inACCORDfrom intensive therapy (8,9), despite ex-amining factors including hypoglyce-mia and hypoglycemia unawareness(which was actually less common amongthose who died in the intensive therapyarm) (10,11), age (12), cardiac autonomicdysfunction (13), weight gain (9), and rateof HbA1c reduction (14). Although sev-eral factors in combination are thoughtto account for mortality HTEs, univariablesubgroup analyses are not capable ofidentifying them and are subject tofalse-positive findings due to multipletesting (15,16).Recently, the advancement ofmachine

learning methodsdparticularly the

approach of gradient forest analysis(17)dhas aided the search for HTEs (Fig.1). Gradient forest analysis can partitiona trial population into subgroups char-acterized by multiple simultaneous char-acteristics (multivariable rather thanunivariable analysis), using cross-validationto reduce the likelihood of false-positiveresults (17). The gradient forest approachalso inherently accounts for interactionsamong multiple variables (e.g., be-tween age and HbA1c) and is unbiased in

predicting the difference in treatment ef-fect between study arms, unlike oldermachine learning methods that can bebiased and focus on the absolute rate ofevents (e.g., risk of mortality) rather thanheterogeneous treatment effects (e.g.,how individual features affect the treat-ment’s ability to reduce the risk of mor-tality) (17).

The objective of this studywas to applygradient forest analysis to identify sub-groups of ACCORD participants with

Figure 1—Conceptualization of gradient forest analysis to detect heterogeneous treatment effectsfrom trial data. Our implementation of gradient forest analysis involved repeated random samplingfrom both arms of the trial data set to compute the treatment effectdthe difference in theprobability of the primary outcome between the intensive and standard glycemic therapyarmsdamong subgroups of trial participants. After selecting subsamples of the trial data, ourapproach selected combinations of explanatory variables (X1, X2, . . .Xj in the figure) from onesubsection of data to divide the study population subsets with lower vs. higher treatmenteffectsdin this case, all-causemortalitydwhen comparing intensive vs. standard therapy.We thenused another subsection of data to update the preliminary values of the explanatory variables usedto subdivide the population into final values that maximized between-group differences and min-imized within-group differences in treatment effects among each subgroup. By using multiplesubsections of data for the estimation of subgroups, the method produces unbiased estimates ofHTE that are robust to outliers (17). The overall process is then repeated thousands of times toidentify which variables and cut point values define consistent subgroups across thousands ofrandom samplings from the trial data. The final subgroups chosen at the end of the decision treeare referred to as “leaves” of the tree.

2 Mortality Risk From Glycemic Therapy Diabetes Care

decreased or increased risk of all-causemortality attributable to intensivetherapy.

RESEARCH DESIGN AND METHODS

Source of DataACCORD was a randomized, controlledtrial of intensive versus standard glycemiccontrol (open-label target of HbA1c

,6.0% [42 mmol/mol] vs. 7.0–7.9% [53–63 mmol/mol], respectively), with a mul-tifactorial design in which participantswere additionally randomized to inten-sive versus standard lipid treatment(double-blinded assignment to fibrateplus statin or placebo plus statin, respec-tively), or intensive versus standard bloodpressure treatment (open-label targetof systolic blood pressure ,120 mmHgor,140 mmHg, respectively) (2). The tri-al was conducted at 77 clinical sites inNorth America between January 2001and June 2009. Participants in both armsreceived glucose-lowering medications.The glycemic control component of thetrial was terminated early due to highermortality in the intensive therapy arm,with a median on-protocol follow-uptime of 3.7 years and a median on- plusoff-protocol follow-up time of 4.9 years.The full duration of available data wereused in this project. This analysis wasapproved by the Stanford University In-stitutional Review Board (e-Protocol#39321).

ParticipantsParticipants (Supplementary Table 1)were 40–79 years oldwith type 2 diabetes,HbA1c $7.5% (58 mmol/mol), and priorevidence of cardiovascular disease (CVD)or risk factors for CVD (e.g., dyslipidemia,hypertension, smoking, or obesity; thosewithout a prior cardiovascular event werebetween the ages of 55 and 79) (2,18,19).Exclusion criteria for ACCORD includedBMI .45 kg/m2, serum creatinine .1.5mg/dL, or serious illnesses that mightlimit trial participation or life expectancy.Data from all study arms were included,with variables identifying glycemic, bloodpressure, and lipid study arm to controlfor randomized therapy selection (15).

OutcomeThe primary outcome for the currentstudywas the difference in all-causemor-tality between therapy arms, assessedfrom the point of enrollment to thetime of study termination in June 2009.

Mortality assessment in ACCORD wasmasked to therapy arm. The secondaryoutcomewas the difference in compositemicrovascular events (including nephrop-athy, retinopathy, and neuropathy) be-tween study arms, defined in ACCORDas renal failure, end-stage renal dis-ease (dialysis), serum creatinine .3.3mg/dL, photocoagulation or vitrectomy, orMichigan Neuropathy Screening Instru-mentscore.2.0.Aswithmortality,assess-ment of microvascular events in ACCORDwas masked to therapy arm. The second-ary outcome was chosen to determinewhether subgroups of participants identi-fiedbased onHTEs formortality exhibitedsimilar HTEs in diabetes-relatedmicrovas-cular complications because the stron-gest support for intensive therapy hascome from studies of reduced microvas-cular events. To help find groups withhigh mortality risk and low microvascularbenefit, and vice versa, the decision treebased on the primary outcome wastested on the secondary outcome to de-termine whether the same features thatpredicted a higher or lower effect of in-tensive treatment on mortality wouldalso predict a higher or lower effect ofintensive treatment on microvascularevents.

PredictorsPotential predictor variables for HTEs(itemized in Supplementary Table 1) in-cluded the subset of characteristics pre-viously hypothesized to be related tocardiovascular or all-cause mortalityamong persons with type 2 diabetes: de-mographics (age, sex, race/ethnicity),study arm, type and number of glucose-lowering medications (including insulinuse and oral glucose-loweringmedicationby class, individually, and in combination),diabetes history (years since diabetesdiagnosis, hypoglycemia in prior 7 days),prior ulcer or amputation, history of eyedisease or surgery, loss of vibratory sen-sation or monofilament sensation, bio-markers (HbA1c, fasting blood glucose,hemoglobin glycosylation index [HGI][20] [defined as observed 2 predictedHbA1c [%], where predicted HbA1c =0.009 3 fasting plasma glucose [mg/dL] +6.8, using the single baseline fastingplasma glucose], lipid profile, serum cre-atinine, estimated glomerular filtrationrate by the Modification of Diet in RenalDisease Study equation, serumpotassium,urine microalbumin, urine creatinine, alanine

aminotransferase, creatinine phosphokinase,systolic and diastolic blood pressure, heartrate, and BMI), and CVD covariates (tobaccosmoking;atrialfibrillationorotherarrhythmiaby electrocardiogram; left ventricular hyper-trophy by electrocardiogram; prior myocar-dial infarction, stroke, angina, bypass surgery,percutaneouscoronary intervention,orothervascularprocedure;andbloodpressuremed-ications, cholesterol medications, and antico-agulant/antiplatelet medications). HGI wasincluded among the covariates because itwaspreviouslysuggestedasapotentiallyuse-ful indicator of diabetes severity as well as apredictorofHTEs inmortality amongpersonswith type 2 diabetes (20–22). Treatmentarm (intensive vs. standard) is inherentlypart of the gradient forest analysis becausetheoutcome isdefinedasdifference inmor-tality between the two arms.

All predictor variables were taken fromthe baseline (prerandomization) studyvisit because our goal was to identifyfactors clinicians could use before thedecision to set more, or less, intensiveglycemic targets. Therefore, time-varyingcovariateswere not incorporated into theanalysis.

Sample SizeA total of 10,251participantswere includedfrom the ACCORD trial, which includes thecomplete sample of participants enrolled.

Missing DataMissing data were not imputed be-cause,1% of data for any predictor vari-able were missing from the trial dataset.

Statistical Analysis MethodTo ensure transparency and reproducibil-ity of the analysis, statistical code is linkedat https://sdr.stanford.edu concurrentwith publication. Our implementation ofgradient forest analysis proceeded in foursteps (Fig. 1). First, ACCORD trial datawere divided in half randomly, with anequal number of intensive and standardglycemic control arm participants in eachof the twodata subsets. Second, variableswere chosen by randomly sampling sub-sets of potential predictors to construct adecision tree made of those predictorsthat could split the first of the two sub-samples of data into subgroups withhigher and lower treatment effect (seeFig. 1). Treatment effect was defined asthe absolute difference in the all-causemortality rate between the intensiveand standard therapy arms. Subgroups

care.diabetesjournals.org Basu and Associates 3

were required to be .5% of the overallstudy sample; we tested the consistencyof the approach to ensure the same resultif we used limits of .1% to .8%. Third,once the initial decision tree was con-structed from the first subsample ofdata, the values of each predictor thatwould define branches in the decisiontree were refined using the second sub-sample of data so that the final subgroupsat the bottom of the tree (“leaves” of thetree) had maximum between-group dif-ferences and minimum within-group dif-ferences in treatment effect. Refinementin the second data subset reduces the in-fluence of outliers and helps produce un-biased HTE estimates (17). The overallapproach was repeated 4,000 timesfrom the first step to produce a “forest”of trees by repeated random resamplingof the data (cross-validation). No changein estimated variable importance was ob-served beyond 4,000 trees. Variable im-portance was defined as the frequencywith which a given variable was incorpo-rated into a tree at the first, second, andfurther split points (i.e., a variable canchange positions between trees, but vari-able selection for each position is trackedto monitor its importance). After the for-est was constructed and cross-validated,the summary (average) decision tree wasselected that separated participants intothe subgroups that were most consistentacross all trees in the forest (23).To assess performance of the summary

decision tree, the absolute risk differencein mortality was calculated between theintensive and standard glycemic controlarms within each subgroup (leaf) of thetrial population and compared across thesubgroups (Q test for heterogeneityamong subgroups and stratified log-ranktest for trend in Kaplan-Meier all-causemortality rates across subgroups). Abso-lute risk difference is the guideline-recommended outcome variable becauseit provides a clinically meaningful abso-lute, as opposed to relative, measure ofeffect (24–26). In addition, we estimatedthe Cox proportional hazards model forthe outcome of mortality by treatmentarm within each leaf, the hazard ratio oftreatment, and the C statistic (area underthe receiver operating characteristiccurve) for discrimination of higher fromlower overall mortality by leaf.Heterogeneous treatment effects

models should not be confused for riskmodels (e.g., Cox models of the risk of

mortality). A heterogeneous treatmenteffect model seeks to determine charac-teristics that are associated with treat-ment effectiveness. Hence, it modelsthe difference in event rates betweentreatment arms (the treatment effect)and tries to find the covariates that areassociated with the treatment beingmore effective or less effective. A riskmodel, by contrast, finds correlates asso-ciated with a given outcome, such asidentifying characteristics associatedwith the risk for mortality. Hence, it mod-els the absolute event rate and tries tofind the covariates (e.g., such as sex,blood pressure, etc.) that make overallmortality higher or lower; treatmentmay ormaynot bea covariate. A standardrisk model does not specifically look forthose factors that modify the treatmenteffect (i.e., interaction terms betweenstudy arm and covariates), whereas ourgradient forest approach focuses exclu-sively on finding influential interactionterms, indicating those factors that mod-ify the treatment effect. Furthermore, se-lection of an interaction term betweentreatment and effect modifiers may bereduced in significance by the larger ef-fect on model fit and C statistic by thenoninteracted terms and reveal only mod-ification on a relative scale versus the abso-lute scale of the gradient forest approach.

Sensitivity AnalysesIn sensitivity analyses, the summary de-cision treewas testedwith the alternativeoutcome of difference in CVD mortalitybetween study arms, defined in ACCORDas mortality suspected to be attributableto myocardial infarction, other acute cor-onary event, cardiovascular procedure,congestive heart failure, arrhythmia, orstroke. The effect of intensive therapywas notably larger (more adverse) forCVDmortality than for all-cause mortalityin the ACCORD trial (2).

Analyses were performed in R 3.3.3software (The R Project for Statistical Com-puting, Vienna, Austria).

RESULTS

ParticipantsOf the 10,251 study participants includedin the analysis, 718 died during studyfollow-up from all causes, including 327participants (6.4%) in the standard therapyarm and 391 participants (7.6%)in the in-tensive therapy arm. CVD was attributedas the cause of death for 331 participants

(3.2% of participants, 46.1% of deaths), in-cluding 144 (2.8% of participants) in thestandard glycemic therapy arm and 187(3.6% of participants) in the intensive glyce-mic therapy arm. As in the original ACCORDpublication (2), the hazard ratio of treat-ment was 1.17 (95% CI 0.98, 1.40) for all-cause mortality in the intensive versusstandard glycemic group overall, after in-cluding all predictor covariates in a stan-dard Cox regressionmodel, and1.20 (95%CI1.04, 1.39)without predictor covariatesincluded.

Model SpecificationThe summary decision tree (Fig. 2) sepa-rated the ACCORDpopulation by variationin all-cause mortality rate differences be-tween the standard and intensive therapyarms. Thefirst split of the treewas definedby the HGI, which was selected as the keysplitting variable in 2,390 of 4,000 trees(59.8%). For participants with low HGI(,0.44, or 75% of the study sample),the next split was defined by BMI, whichwas selected as a subsequent splittingvariable in 2,322 of 4,000 trees (58.1%).The group with a low BMI (,30 kg/m2, aderived value rounded to the nearestkg/m2) was further split by age (,61years), which was selected in 1,814 of the4,000 trees (45.4%). The three variables de-fining the decision tree were available for9,801 of the 10,251 ACCORD trial partici-pants (95.6%).

Model PerformanceThe summary decision tree split the studysample into groups with significantly dif-ferent risk for all-cause mortality from in-tensive glycemic therapy, as reported inTable 1 (P , 0.001 by the Q test for het-erogeneity in absolute mortality risk dif-ference between intensive vs. standardtherapy among the four groups, andP , 0.001 by the stratified log-rank testfor a trend in absolutemortality differencefrom subgroup 1 through subgroup 4).

Subgroup (leaf) 1 had 877 participants(8.6% of the 10,251-participant totalsample) and was defined by HGI ,0.44,BMI ,30 kg/m2, and age ,61 years old.Subgroup 1 had an absolute mortalityrate reduction (benefit) of 2.3% from in-tensive glycemic therapy (95% CI 0.2 to,4.5 decrease; hazard ratio 0.41; 95% CI0.17, 0.98; P = 0.038 by the log-rank testadjusting for censoring). Participants insubgroup 1 had a number needed to treat(NNT) of 43 over 5 years to observe 1 less

4 Mortality Risk From Glycemic Therapy Diabetes Care

deathwith intensive rather than standardglycemic therapy.Subgroup (leaf) 2 had 1,717 partici-

pants (16.7% of sample) and was definedby HGI ,0.44, BMI ,30 kg/m2, andage $61 years old. Subgroup 2 had nosignificant absolute mortality rate reduc-tion or increase, with an absolute risk in-crease of 0.7% from intensive glycemictherapy (95% CI 1.6% decrease to 3.1%increase; hazard ratio 1.11, 95% CI 0.77,1.60; P = 0.560).Subgroup (leaf) 3 had 4,678 partici-

pants (45.6% of sample) and was definedby HGI ,0.44 and BMI $30 kg/m2. Sub-group 3 had no significant absolute mor-tality rate reduction or increase, with anabsolute risk increase of 0.9% from inten-sive glycemic therapy (95% CI 0.4% de-crease to 2.1% increase) and a hazardratio of 1.12 (95%CI 0.91, 1.50;P = 0.220).Subgroup (leaf) 4 had 2,529 partici-

pants (24.7% of sample) and was definedby HGI $0.44. Subgroup 4 had an abso-lute mortality rate increase of 3.7% fromintensive glycemic therapy (95% CI 1.5 to6.0 increase) and a hazard ratio of 1.57(95% CI 1.20, 2.04; P , 0.001). Partici-pants in subgroup 4 had a number neededto harm of 27 over 5 years associated with

1 additional death in the intensive thanstandard glycemic therapy arm.

Figure 3 illustrates the survival curvesamong the intensive and standard glyce-mic therapy arms of ACCORD, stratifiedby the subgroups. Supplementary Table 2lists the other clinical features among thesubgroups by arm, revealing that covari-ates were balanced across the therapyarms within each subgroup. Hence, im-balance in important covariates betweenarms did not result from the stratifica-tion into subgroups, meaning that thegradient forest analysis did not produceconfounding by measured covariates.Critically, Supplementary Table 2 also re-veals that because no single predictor var-iable could explain the subgroups, thedecision tree did not capture featuresthat would be otherwise obvious from aunivariable subgroup analysis; rather, themultivariate machine learning analysishad the power to reveal variations inmortality that would not be detectableto univariable subgroup analyses alongany of the measured variables in thestudy. Overall, the out-of-the-bag errorrate of the model, a measure of the pre-diction error during out-of-sample cross-validation, was low, with a value of 5.6%.

We evaluated whether the secondaryoutcome of composite microvascularevents varied among the subgroups(Supplementary Table 3 and Supplemen-tary Fig. 1). The average decrease in mi-crovascular outcomes was nonsignificantfor all four subgroups, consistent withthe overall results of the ACCORD trial(24). However, the average outcomeswere better for subgroup 1 (absoluterisk decrease of 4.2%, 95% CI 10.6 de-crease to 2.1 increase, P = 0.15) than forsubgroup 4 (absolute risk decrease of2.3%, 95% CI 6.1 decrease to 1.5 increase,P = 0.60).

In sensitivity analyses, we evaluatedthe summary decision tree with the out-come of CVD mortality (SupplementaryTable 4 and Supplementary Fig. 2). Aswith absolute risk differences in all-causemortality, absolute risk differences inCVD mortality between intensive andstandard glycemic therapy differed signif-icantly between the four subgroups (P,0.001 for heterogeneity and for trend).Subgroup 1 had an absolute cardiovascu-lar mortality risk decrease of 1.7% in theintensive therapy arm (95% CI 0.2 to 3.2decrease, P = 0.027), and subgroup 4 hadan absolute cardiovascular mortality risk

Figure 2—Summary risk stratification decision tree developed to identify the absolute change in risk of all-cause mortality among persons with type 2diabetes subject to intensive therapy, based on baseline characteristics of individual participants in the ACCORD trial (2001–2009, N = 10,251). Negativevalues indicate reduced absolutemortality (benefit from intensive glycemic control), whereas positive values indicate increased absolutemortality (harmfrom intensive glycemic control).

care.diabetesjournals.org Basu and Associates 5

increase of 2.3% in the intensive ther-apy arm (95% CI 0.6 to 3.9 increase,P = 0.004).

CONCLUSIONS

We sought to inform clinical decisions re-garding the safety of intensive glycemictherapy among patients with type 2 dia-betes and elevated CVD risk by identify-ing HTE in all-cause mortality within theACCORD trial. We found that by usingthe covariates of HGI, age, and BMI, wecould classify participants in the ACCORDtrial into subgroups with clinically mean-ingful differences in mortality attribut-able to intensive glycemic therapy. Themean all-cause mortality rate among in-dividuals with diabetes in the U.S. is;6%over 5 years, so an absolute risk increaseof 4% or an absolute risk reduction of 2%is clinically meaningful (25). Approxi-mately 25% (n = 2,529) of participantsbelonged to a subgroup experiencingincreasedmortality attributable to intensiveglycemic therapy, whereas 9% (n = 877)belonged to a subgroup that experiencedreduced mortality attributable to inten-sive glycemic therapy. We did not findthat hypoglycemia, medication classes,number of medications, combinations ofmedications, baseline diabetes complica-tions, or cardiovascular risk factors couldexplain the heterogeneous treatment ef-fects from intensive glycemic therapy.Wealso did not find a trade-off between mi-crovascular and mortality risk, becausethe patients with the highest mortalityrisk from intensive therapy also had theleast evidence of microvascular benefit,and vice versa.

Our findings support and extend priorstudies of glycemic control in diabetesmanagement. We found that despitethe average treatment effect of highermortality, there were some groups thatmay have benefited from, along withsome that were likely harmed by, inten-sive glycemic therapy; nearly two-thirds(n = 6,395) experienced neither benefitnor harm. Because the risk of benefitand of harm varies among individualswith type 2 diabetes, our results supportcurrent guidelines that advocate for indi-vidualized treatment decisions and alsohelp such guidelines to be made opera-tional in clinical practice (1). Clinically, thedecision tree we developed through adata-drivenmultivariate subgroup analysisuses readily available clinical data andmayassist clinician-patient discussions about

Tab

le1—

Chan

gein

abso

lute

risk

ofall-ca

use

mortalityfrom

intensive

versusstan

dardglyce

mic

control,am

ongsu

bgroupsiden

tified

bygradientforest

analysis

oftheACCORD

Trial

(2001–

2009,

N=10

,251)

Group

Intensive

therapy

Stan

dard

therap

y

Totald

eaths(N

=9,801of

10,251

withvariablesto

stratifyrisk)n(%)

Deathsam

ong

intensive

therapy

(N=4,900of

5,128

withvariablesto

stratifyrisk)n(%)

Deathsam

ong

stan

dardtherapy

(N=4,901of

5,123

withvariablesto

stratifyrisk)n(%)

Hazardratioof

intensive

vs.stan

dard

treatm

ent(95%

CI),C

statistic

Absolute

risk

difference

%(95%

CI)

Pvalue(lo

g-rank

test)of

difference

ineventrates

betw

eenarms,

withinleaf

Stratified

log-rank

test(difference

intreatm

enteffect

across

leaves)

Leaf

1(leftmostinFig.2)

(HGI,

0.44,

BMI,

30kg/m

2,age

,61

years)

424

453

25(2.9)

7(1.7)

18(4.0)

0.41

(0.17,0.98)0.64

(0.52,0.76)

22.3(2

4.5to

20.2)

0.04

,0.001

Leaf

2(HGI,

0.44,B

MI,

30kg/m

2,

age$61

years)

811

906

116(6.8)

58(7.2)

58(6.4)

1.11

(0.77,1.60)0.62

(0.56,0.67)

0.7(2

1.6to

3.1)

0.56

Leaf

3(HGI,

0.44,B

MI$

30kg/m

2)

2,37

52,30

325

0(5.3)

137(5.8)

113(4.9)

1.12

(0.91,1.50)0.64

(0.60,0.68)

0.9(2

0.4to

2.1)

0.22

Leaf4(rightmostinFig.2)(HGI$

0.44)

1,29

01,23

923

4(9.3)

143(11.1)

91(7.3)

1.57

(1.20,2.04)0.66

(0.62,0.71)

3.7(1.5to

6.0)

,0.001

SeeFig.2forvisualizationof

subgroup

s.Notethat

thehazard

ratioof

intensivevs.stand

ardtreatm

entandtheCstatistic(areaun

derthereceiver

operatingcharacteristiccurve)

fordiscriminationof

higher

from

lower

overallm

ortalityby

leaf

was

estimated

theCox

prop

ortion

alhazardsmod

elfortheou

tcom

eof

mortalityby

treatm

entarm

withineach

leaf.

6 Mortality Risk From Glycemic Therapy Diabetes Care

glycemic therapy. Although the ACCORDHbA1c target of ,6.0% (42 mmol/mol) isnot guideline recommended, many pa-tients are currently treated to ,6.5%(48 mmol/mol, the achieved mean inthe intensive therapy arm) with regimensother than metformin alone (5,19). Be-cause ;25% of ACCORD-eligible patientswere observed to have high risk of harmfrom intensive therapy, deescalation ofglycemic therapy may be warranted forsome patients. Our study also adds to a

growing body of literature, includinga prior study using ACCORD data, that ahighHGImay be an important indicator ofdiabetes severity as well as a predictor ofHTEs in mortality among persons withtype 2 diabetes (20–22). A higher HGImay indicate higher postprandial glucoselevels and increased glycemic variability.Notably, the HGI in this report does notrequire that mean glucose levels be de-termined by continuous glucose monitor-ing, as is common in studies of type 1

diabetes. Rather HGI was calculatedusing a single HbA1c and fasting plasmaglucose measurement, offering potentialconvenience for clinical use.

More broadly, these results point to-ward the application of innovative meth-ods for the detection of heterogeneoustreatment effects from clinical trial data.Our findings highlight the point that trialsummary statistics, which are averages,mayobscure clinically important heteroge-neities and that the rigorous application

Figure 3—Survival curves for all-cause mortality among subsets identified by each subgroup in the decision tree (see Fig. 2 for subgroups). P values arefrom the stratified log-rank test adjusted for censorship of Kaplan-Meier all-causemortality rates among the intensive vs. standard glycemic therapy arm.A: Leaf 1 (leftmost in Fig. 2) (HGI ,0.44, BMI ,30 kg/m2, and age ,61 years). B: Leaf 2 (HGI ,0.44, BMI ,30 kg/m2, and age $61 years). C: Leaf3 (HGI,0.44 and BMI$30 kg/m2). D: Leaf 4 (rightmost in Fig. 2) (HGI$0.44).

care.diabetesjournals.org Basu and Associates 7

of machine learning methods with con-servative cross-validation approachesmay aid in finding consistent subgroupsthat experience substantial differences intreatment effects. Extensive theoreticaland empirical research suggests that theability of conventional univariable sub-group analyses to detect clinically impor-tant heterogeneity in treatment effects isvery limited (26–28). Previous studies ofHTEs in ACCORD data have consideredsingle variables, finding that hypoglyce-mia and cardiac autonomic dysfunctiondid not explain the harmof intensive ther-apy (10,11,13). The machine learningmethod accounting for multiple simul-taneous covariates and interactionsbetween them was therefore able to ex-plain the variation in mortality betterthan previous univariable analyses. Wein fact found in our sensitivity analysesthat no single covariate would be ableto distinguish the subgroups, and there-fore, our multivariable machine learninganalysis had the power to explain var-iations that were not possible to findwith traditional univariable subgroupanalyses.A prior analysis of age as a source of

HTEs found that younger age was associ-ated with increased harm (12). Our find-ing that younger age, in combinationwithlower BMI and HGI, is instead associatedwith benefit may represent the interac-tion of factors not considered in univari-able analyses. In general, consideringseveral factors in combination may be re-quired to explain clinically important var-iations in benefit and harm seen in clinicaltrials. Consequently, multivariate HTEmodeling has been increasingly recom-mended (15,16,29). Our data-driven ap-proach also adjusts for type I error dueto multiple hypothesis testing, a majordisadvantage of traditional subgroupanalysis methods. We used rigorouscross-validation to reduce the chance offalse-positive findings.Our analysis nevertheless has im-

portant limitations. As a result of theACCORD trial being stopped early, wecould assess only shorter-term outcomes.Further, the ACCORD trial was conductedbefore the widespread availability ofsodium–glucose cotransporter 2 andglucagon-like peptide 1 agents, whichhave cardiovascular benefits that affectthe risk of mortality with glycemic ther-apy (30–33). In addition, because wewanted a clinical decision tree that was

useful in practice, we focused on pre-treatment characteristics rather thantime-varying covariates, which may bemore useful in predicting outcomes overtime but are also more complex for clini-cians to use. Next, although we usedmethods to minimize the risk of type Ierror and did not observe imbalance incovariates within subgroups, our study isnevertheless a post hoc analysis of a sin-gle trial. With machine learning methods,as with correlative statistical methods ingeneral, variable selection does not provecausality, and the variables selected mayonly be surrogates for more complexphysiological processes. HGI is a summarymeasure that may not have a definitivephysiological meaning and can be calcu-lated in alternative ways; here, it servesas a useful and readily calculable markerof complex physiological processes andwas found to separate the variation inmortality better than alternative covari-ates. HGI thus likely reflects a complexunderlying heterogeneity in treatment ef-fect. Explaining mechanistically the phys-iological relationships that underlie theheterogeneous treatment effects ob-served is not possible from the availabledata, although they are broadly consis-tent with clinical observation and pointto areas for further study (34). In addi-tion, the number of deaths among thestandard and intensive therapies in leaf1 were too small (7 and 18 subjects). Fi-nally, it is important to note that theseresults naturally apply to the populationthat met inclusion criteria for ACCORD,which includes people with type 2 diabe-tes with HbA1c of $7.5%, who were be-tween the ages of 40 and 79 years andhad CVD or were between the ages of55 and 79 years and had anatomical evi-dence of significant atherosclerosis, albu-minuria, left ventricular hypertrophy, orat least two additional risk factors forCVD, such as dyslipidemia, hypertension,tobacco smoking, or obesity.

Our study suggests several directionsfor futurework. Because only internal val-idation was done in this report, pro-spectively validating the decision tree onan independent trial data set and onpopulation-based observational datawould help assess the generalizability ofour findings. Ultimately, it will be impor-tant to evaluate the effect of using thedecision tree on clinical practice and pa-tient outcomes. More generally, hetero-geneous treatment effects are likely to be

the norm, rather than the exception, inmany areas of investigation. Therefore,it may be advantageous to design trialsthat can identify HTEs up front, ratherthan relying on post hoc analysis as wehave done here. A prior simulation studyrevealed that alternative trial designs,which randomize persons in a stepwisefashion to incrementally higher levels oftherapy intensification, could increasestatistical power to detect HTEs and pro-vide more granular estimates of treat-ment benefit or harm (28). Finally, theanalysis suggests that HGImay be a usefulclinical indicator of risk and advanced di-abetes, necessitating future prospectivestudy as a useful clinical biomarker.

Cliniciansmay use HGI, age, and BMI tohelp individualize decisions about glyce-mic control among people with type 2diabetes. This may lead to deescalationof therapy for many patients while alsoidentifying patients who do not face in-creased all-causemortality risk from theircurrent glycemic therapy. Further, themethods used in this study offer a princi-pled way to help inform individualizedcare using data from randomized trials.The application of similar methods mayenable us to learn more from the contri-bution that clinical trial participantsmake, bringing us closer to the goal ofpersonalized medicine.

Acknowledgments. This manuscript was pre-pared using ACCORD research materials ob-tained from the National Heart, Lung, andBlood Institute Biologic Specimen and Data Re-pository Information Coordinating Center anddoes not necessarily reflect the opinion or viewsof the ACCORD or the National Heart, Lung, andBlood Institute.Funding. Research reported in this publicationwassupportedby theNational InstituteonMinorityHealth and Health Disparities (DP2-MD-010478and U54-MD-010724 to S.B.) of the NationalInstitutes of Health, the American Heart Asso-ciation (17MCPRP33670728 to S.R.), and theNational Institute for Diabetes and Digestiveand Kidney Diseases (U01-DK-098246 and R18-DK-10273 to D.J.W., and K23-DK-109200 toS.A.B.) of the National Institutes of Health.The content is solely the responsibility of the

authors and does not necessarily represent theofficial views of the National Institutes of Healthor the American Heart Association.Duality of Interest. No potential conflicts of in-terest relevant to this article were reported.AuthorContributions. S.B. wrote the first draftof the manuscript. S.B., S.R., D.J.W., and S.A.B.revised the manuscript, reviewed the results,and contributed to thediscussion. S.B. andS.A.B.conducted analyses. S.A.B. conceived of the

8 Mortality Risk From Glycemic Therapy Diabetes Care

research idea. S.B. is the guarantor of this workand, as such, had full access to all the data in thestudy and takes responsibility for the integrity ofthe data and the accuracy of the data analysis.

References1. AmericanDiabetesAssociation. Glycemic targets.Sec. 6. In StandardsofMedicalCare inDiabetesd2017.Diabetes Care 2017;40(Suppl. 1):S48–S562. Action to Control Cardiovascular Risk in Diabe-tes StudyGroup; Gerstein HC,MillerME, ByingtonRP, et al. Effects of intensive glucose lowering intype2diabetes. N Engl JMed 2008;358:2545–25593. Zoungas S, Arima H, Gerstein HC, et al.; Collab-orators on Trials of Lowering Glucose (CONTROL)group. Effects of intensive glucose control on mi-crovascular outcomes in patients with type 2 di-abetes: a meta-analysis of individual participantdata from randomised controlled trials. Lancet Di-abetes Endocrinol 2017;5:431–4374. Sussman JB, Kerr EA, Saini SD, et al. Rates ofdeintensification of blood pressure and glycemicmedication treatment based on levels of controland life expectancy in older patientswithdiabetesmellitus. JAMA Intern Med 2015;175:1942–19495. Lipska KJ, Ross JS, Miao Y, Shah ND, Lee SJ,Steinman MA. Potential overtreatment of diabe-tes mellitus in older adults with tight glycemiccontrol. JAMA Intern Med 2015;175:356–3626. McCoy RG, Van Houten HK, Ross JS, MontoriVM, Shah ND. HbA1c overtesting and overtreat-ment among US adults with controlled type 2 di-abetes, 2001-13: observational population basedstudy. BMJ 2015;351:h6138.7. Holman RR, Paul SK, BethelMA,Matthews DR,Neil HA. 10-year follow-up of intensive glucosecontrol in type 2 diabetes. N Engl J Med 2008;359:1577–15898. Skyler JS, Bergenstal R, BonowRO,et al.; Amer-ican Diabetes Association; American College ofCardiology Foundation; American Heart Association.Intensive glycemic control and the preventionof cardiovascular events: implications of theACCORD, ADVANCE, and VA diabetes trials: a posi-tion statement of the American Diabetes Associa-tion and a scientific statement of the AmericanCollege of Cardiology Foundation and the AmericanHeartAssociation. DiabetesCare 2009;32:187–1929. Riddle MC. Counterpoint: intensive glucosecontrol and mortality in ACCORD–still lookingfor clues. Diabetes Care 2010;33:2722–272410. Bonds DE, Miller ME, Bergenstal RM, et al.The association between symptomatic, severe hy-poglycaemia and mortality in type 2 diabetes:

retrospective epidemiological analysis of theACCORD study. BMJ 2010;340:b490911. Seaquist ER, Miller ME, Bonds DE, et al.;ACCORD Investigators. The impact of frequent andunrecognized hypoglycemia on mortality in theACCORD study. Diabetes Care 2012;35:409–41412. Miller ME, Williamson JD, Gerstein HC, et al.;ACCORD Investigators. Effects of randomizationto intensive glucose control on adverse events,cardiovascular disease, andmortality in older ver-sus younger adults in the ACCORD trial. DiabetesCare 2014;37:634–64313. Pop-Busui R, Evans GW, Gerstein HC, et al.;Action to Control Cardiovascular Risk in DiabetesStudy Group. Effects of cardiac autonomic dys-function on mortality risk in the Action to ControlCardiovascular Risk in Diabetes (ACCORD) trial.Diabetes Care 2010;33:1578–158414. Riddle MC, Ambrosius WT, Brillon DJ, et al.;Action to Control Cardiovascular Risk in DiabetesInvestigators. Epidemiologic relationships betweenA1C and all-cause mortality during a median 3.4-year follow-up of glycemic treatment in theACCORD trial. Diabetes Care 2010;33:983–99015. Burke JF, Hayward RA, Nelson JP, Kent DM.Using internally developed risk models to assessheterogeneity in treatment effects in clinical trials.Circ Cardiovasc Qual Outcomes. 2014;7:163–169.16. Hayward RA, Kent DM, Vijan S, Hofer TP.Mul-tivariable risk prediction can greatly enhance thestatistical power of clinical trial subgroup analysis.BMC Med Res Methodol 2006;6:1817. Athey S, Imbens G. Recursive partitioning forheterogeneous causal effects. Proc Natl Acad SciU S A 2016;113:7353–736018. ACCORD Study Group; Buse JB, Bigger JT,Byington RP et al. Action to Control Cardiovascu-lar Risk in Diabetes (ACCORD) trial: design andmethods. Am J Cardiol 2007;99:21i–33i19. Gerstein HC, Riddle MC, Kendall DM, et al.;ACCORD Study Group. Glycemia treatment strat-egies in the Action to Control Cardiovascular Riskin Diabetes (ACCORD) trial. Am J Cardiol 2007;99:34i–43i20. Hempe JM, Liu S, Myers L, McCarter RJ, BuseJB, Fonseca V. The hemoglobin glycation indexidentifies subpopulations with harms or benefitsfrom intensive treatment in the ACCORD trial. Di-abetes Care 2015;38:1067–107421. van Steen SC, Schrieks IC, Hoekstra JB, et al.;AleCardio study group. The haemoglobin glyca-tion index as predictor of diabetes-related com-plications in the AleCardio trial. Eur J Prev Cardiol2017;24:858–866

22. McCarter RJ, Hempe JM, Gomez R, ChalewSA. Biological variation in HbA1c predicts risk ofretinopathy and nephropathy in type 1 diabetes.Diabetes Care 2004;27:1259–126423. Banerjee M, Ding Y, Noone AM. Identifyingrepresentative trees from ensembles. Stat Med2012;31:1601–161624. Ismail-Beigi F, Craven T, Banerji M, et al. Ef-fect of intensive treatment of hyperglycaemia onmicrovascular complications of type 2 diabetes inACCORD: a randomised trial. Lancet 2010;376:419–43025. National Center for Health Statistics. Health,United States, 2016: With Chartbook on Long-Term Trends in Health. Washington, D.C., Depart-ment of Health and Human Services, 201726. VanderWeele TJ, Knol MJ. Interpretation ofsubgroup analyses in randomized trials: heteroge-neity versus secondary interventions. Ann InternMed 2011;154:680–68327. Wallach JD, Sullivan PG, Trepanowski JF, SainaniKL, Steyerberg EW, Ioannidis JP. Evaluation of evi-denceof statistical support andcorroborationof sub-group claims in randomized clinical trials. JAMAIntern Med 2017;177:554–56028. Basu S, Sussman JB, Hayward RA. Detectingheterogeneous treatment effects to guide per-sonalized blood pressure treatment: a modelingstudy of randomized clinical trials. Ann InternMed 2017;166:354–36029. Kent DM, Rothwell PM, Ioannidis JP, AltmanDG, Hayward RA. Assessing and reporting hetero-geneity in treatment effects in clinical trials: a pro-posal. Trials 2010;11:8530. Marso SP, Bain SC, Consoli A, et al.; SUSTAIN-6 Investigators. Semaglutide and cardiovascularoutcomes in patients with type 2 diabetes. NEngl J Med 2016;375:1834–184431. Marso SP, Daniels GH, Brown-Frandsen K,et al.; LEADER Steering Committee; LEADER TrialInvestigators. Liraglutide and cardiovascular out-comes in type 2 diabetes. N Engl J Med 2016;375:311–32232. Zinman B,Wanner C, Lachin JM, et al.; EMPA-REG OUTCOME Investigators. Empagliflozin, car-diovascular outcomes, and mortality in type 2diabetes. N Engl J Med 2015;373:2117–222833. NealB, PerkovicV,MahaffeyKW, et al.; CANVASProgram Collaborative Group. Canagliflozin andcardiovascular and renal events in type 2 diabe-tes. N Engl J Med 2017;377:644–65734. American Diabetes Association. 8. Pharmaco-logic approaches to glycemic treatment. DiabetesCare 2017;40(Suppl. 1):S64–S74

care.diabetesjournals.org Basu and Associates 9