multiple problems
DESCRIPTION
Multiple problems. Keith J Barrington CHU Sainte Justine, Montréal. Twins and triplets in neonatal research. Neonatal research is unique in often enrolling large numbers of genetically identical or closely related subjects. - PowerPoint PPT PresentationTRANSCRIPT
MULTIPLE PROBLEMSKeith J BarringtonCHU Sainte Justine, Montréal
TWINS AND TRIPLETS IN NEONATAL RESEARCH Neonatal research is unique in often enrolling
large numbers of genetically identical or closely related subjects.
Particularly true when entry criteria are not very restrictive, such as by gestational age or birth weight
For some outcomes there is now evidence of a genetic influence, for other outcomes there may well be genetic effects
THERE ARE SITUATIONS IN OTHER FIELDS THAT ARE ANALOGOUS Enrolment in maternal treatment trials of mothers
carrying twins But they are always in the same group Could analyze all data independently, just select the
1st twins outcomes, randomly select 1, analyze ‘by pregnancy’
Enrolment of individuals with bilateral eye disease Such as retinopathy, very highly correlated, but for
local treatments, can be randomized separately, as a block or into different groups
THE 2 EYE PROBLEM Statistical techniques for addressing the
correlation within blocks of 2 have been developed for the eyes in ophthalmologic research
All eye pairs are ‘monozygotic’ All eye pairs experience near identical
environmental conditions.
ETROP STUDY Infants with bilateral eye disease (80%) One eye randomized to early treatment Other eye treated conventionally
Equivalent to randomizing twins to opposite groups
No correction for block
2 consents, one for enhanced screening and another for randomization to treatment 40 refusals and 401 randomized
WHAT OUTCOMES MAY BE GENETICALLY INFLUENCED? Lung disease
Hyaline Membrane Disease Nagourney et al 1996
Bronchopulmonary Dysplasia Bhutani et al. Lavoie et al.
Brain Injury IVH, less evidence
Bhutani et al not significant. Retinopathy
Bizzarro et al Patent Ductus Arteriosus
Lavoie et al
WHAT OUTCOMES MAY BE CORRELATED? Other outcomes may be correlated in twins
because of shared intra-uterine environment Chorioamnionitis Pre-Eclampsia Diabetes
Could all affect both twins (or all 3 triplets)
HOW DOES THIS AFFECT RESEARCH? If there is a correlation in an outcome
between twins, and the twins are in the same group, how does this affect trial design and analysis?
GATES AND BROCKLEHURST BJOG 2004 ‘Inclusion of non-independent data means
that the ‘‘effective sample size’’ of the trial is reduced: there are fewer independent outcomes in the trial than the number of babies that took part in it.
Analysing all babies as if they are independent will therefore overestimate the sample size and give confidence intervals that are too narrow’
FOR EXAMPLE: If you had a trial with 100 twins, who had
perfectly concordant results: You have in reality only 50 ‘subjects’ in the
trial Therefore much less power than you thought
But you have an ‘n’ of 100 for the analyses Therefore artificially narrowing the confidence
intervals
HOW IMPORTANT IS THIS IN REALITY? In reality: The magnitude of this effect will depend on: The proportion of twins, the degree of
concordance, the method of analysis, and the impact of the intervention being studied Even if the intervention has no overall impact, if
it changes the concordance between twins, may have a major effect on results
And depends on how many twins are in the same group, or opposite groups
DOES INDEPENDENT RANDOMIZATION REALLY DECREASE ENROLMENT? I have heard individual parents tell me that
they only want their babies in the trial if they can get the same treatment.
I have also heard parents say they want their babies in opposite arms, to be sure that at least one gets the better treatment
CONSORT DIAGRAMS IN NEONATAL RESEARCH Individually randomized versus block
randomized multiples Is there a difference in consent rate?
CONSENT RATE BY METHOD OF MULTIPLE RANDOMIZATION
Problem : often not clear from the publication how twins were enrolled.
Limitation: many other reasons for differential consent rates
Methods: calculated number of refusals of consent as a proportion of those approached for consent
LARGE RCTS EUNO: individual randomization.
610 refusals of 2227 approached NOCLD: block randomization
587 enrolled of 1555 eligible (?number of refusals)
SUPPORT: block randomization 748 refusals of 2064 approached
CAP: individual randomization 1628 refusals of 3634 approached
COIN: individual randomization 233 refusals of 906 approached
RANDOMIZATION MODELS If infants are randomized to the same group,
i.e. as a cluster, then the analysis should take account of that
If there is little correlation between twins (eg severe IVH) then a strict cluster approach, such as the multiple outputation technique will reduce power, and risk type 2 error
If there is strong correlation between twins (eg BPD) then not correcting for clusters will, in general, make confidence intervals more narrow, and risk type 1 error
ACTUAL EXAMPLE NOCLD trial Supported by NIH, plan to use a multiple
outputation technique to analyze because : Block randomization (first of multiples to be
enrolled was randomized, other(s) entered in the same arm)
Strong evidence of coherence in the primary outcome, BPD.
PUBLISHED DATA Survival without BPD 129/294 with iNO 105/288 infants with placebo
p = 0.03 RR= 1.26; 95% CI, 1.02 to 1.55 Attempted Cochrane meta-analysis
If all the data are analyzed as if they were independent
p=0.08 RR = 1.12
RANDOMIZATION MODELS If twins are randomized independently, then
those who are in different groups do not require correction
Adaptive models which control for the degree of correlation among twins in the same group should be used Such as GEE
ANOTHER EXAMPLE If twins are individually randomized, then
using stringent cluster techniques for all the pairs will decrease power
Schreiber et al iNO 51/105 infants death or BPD Placebo 65/102
(relative risk, 0.76; 95 percent confidence interval, 0.60 to 0.97; P=0.03)
THE INO TRIALS (MAPPINO)Distribution of multiples
by treatment group
Trial n % multiples iNO Placebo
Van Meurs 2005 22 4.9 10 (45.5%) 12 (54.5%)
Srisuparp 2002 2 5.9 0 (0.0%) 2 ( 100%)
Kinsella 2006 130 16.4 61 (46.9%) 69 (53.1%)
Hascoet 2005 14 9.7 10 (71.4%) 4 (28.6%)
Schreiber 2003 26 12.6 11 (42.3%) 15 (57.7%)
Kinsella 1999 2 2.4 1 (50.0%) 1 (50.0%)
Dani 2006 2 5.0 2 ( 100%) 0 (0.0%)
EUNO 2008 152 19.0 71 (46.7%) 81 (53.3%)
Ballard 2006 84 14.4 47 (56.0%) 37 (44.0%)
OVERALL 434 13.1 213 (49%) 221 (51%)
† Subhedar removed from the analysis as zero cell counts caused model instability.* χ2 test for heterogeneity p > 0.05
Estimates derived from N=1000 iterations of log-binomial model using multiple outputation method.
Death or CLD (Best available definition)
0.2 0.5 1 2 5
OVERALL*
iNO
40 / 48 (83%)
6 / 16 (38%)
43 / 105 (41%)
42 / 61 (69%)
54 / 64 (84%)
170 / 224 (76%)
292 / 398 (73%)
4 / 20 (20%)
165 / 294 (56%)
134 / 399 (34%)
954 / 1629 (59%)
RR (95% CI)Trial Placebo
27 / 32 (84%)
4 / 18 (22%)
56 / 102 (55%)
51 / 84 (61%)
56 / 62 (90%)
174 / 225 (77%)
294 / 395 (74%)
8 / 20 (40%)
184 / 288 (64%)
137 / 401 (34%)
992 / 1627 (61%)
Favours placeboFavours iNO
EUNO 2008
Ballard 2006
Dani 2006
Kinsella 2006
Van Meurs 2005INNOVO 2005
Hascoet 2005
Schreiber 2003
Srisuparp 2002
Kinsella 1999
0.96 (0.91, 1.01) p=0.095
1.01 (0.83, 1.23)
0.85 (0.74, 0.98)
0.53 (0.19, 1.46)
0.99 (0.91, 1.08)
0.98 (0.88, 1.09)
0.93 (0.82, 1.07)
1.11 (0.85, 1.43)
0.77 (0.57, 1.04)
1.59 (0.55, 4.62)
0.99 (0.81, 1.21)
PRIMARY ENDPOINT 1
IMPLICATIONS? Do not use multiple outputation for
independently enrolled infants The impact of analyzing multiples in the
same group as if they were independent are dependent on how highly correlated the outcomes are, and what proportion of twins.
GATES AND BROCKLEHURST Re-analysis of the antenatal TRH trial (20%
multiples) Monte Carlo simulation of trials with 33%
multiples
Method Antenatal TRH trialTRH group Placebo group OR (95% CI)
Assume independence between babies
34/136 (25.0) 43/139 (30.9) 0.74 [0.42, 1.31]
Analyse by pregnancy 29/112 (25.9) 34/113 (30.1) 0.81 [0.43, 1.51] Random selectionExpected average result 27/112 (24.1) 32.33/113 (28.6) 0.80 [0.55, 1.52]Mean of 1000 repetitions 0.83 {0.04} Cluster trial methods 34/136 (25.0) 43/139 (30.9) 0.74 [0.41, 1.34]
Method Trial BIntervention group Placebo group OR (95% CI)
Assume independence between babies
138/680 (20.3) 136/680 (20.0) 1.02 [0.78, 1.33]
Analyse by pregnancy 107/500 (21.4) 128/500 (25.6) 0.79 [0.59, 1.06] Random selectionExpected average result 83/500 (16.6) 100/500 (20.0) 0.80 [0.58, 1.10]Mean of 1000 repetitions 0.80 {0.05}
Extremes 107/500 (21.4) 75/500 (15.0) 1.54 [1.11, 2.14] 59/500 (11.8) 128/500 (25.6) 0.39 [0.28, 0.55]
Cluster trial methods 138/680 (20.3) 136/680 (20.0) 1.02 [0.78, 1.33]
ANALYSIS OF BINARY OUTCOMES FROM RANDOMISED TRIALS INCLUDING MULTIPLE BIRTHS: WHEN SHOULD CLUSTERING BE TAKEN INTO ACCOUNT?YELLAND ET AL 2011A= INTRAPAIR CORRELATION OF 0.1B=INTRAPAIR CORRELATION OF 0.5C=INTRAPAIR CORRELATION OF 0.9
Paediatric and Perinatal EpidemiologyVolume 25, Issue 3, pages 283-297, 6 APR 2011 DOI: 10.1111/j.1365-3016.2011.01196.xhttp://onlinelibrary.wiley.com/doi/10.1111/j.1365-3016.2011.01196.x/full#f1
ANALYSIS OF BINARY OUTCOMES FROM RANDOMISED TRIALS INCLUDING MULTIPLE BIRTHS: WHEN SHOULD CLUSTERING BE TAKEN INTO ACCOUNT?
Paediatric and Perinatal EpidemiologyVolume 25, Issue 3, pages 283-297, 6 APR 2011 DOI: 10.1111/j.1365-3016.2011.01196.xhttp://onlinelibrary.wiley.com/doi/10.1111/j.1365-3016.2011.01196.x/full#f2
EXAMPLE OF INDEPENDENTLY RANDOMIZED NEWBORNS SHAFFER ET AL 2009 BMC RESEARCH METHODOLOGYAnalysis of bronchopulmonary dysplasia for the IVIG trial
Birth weight stratum Analysis Estimated
correlation Odds ratio Standard error
95% Confidence
interval
501 to 1000 g Logistic N/A 0.912 0.122 (0.702,1.18
5)GLMM 0.438 0.898 0.127 (0.649,1.14
6)(n = 903 babies) GEE 0.361 0.901 0.122 (0.691,1.17
5)1001 - 1500 g Logistic N/A 0.944 0.155 (0.683,1.30
3)GLMM 0.537 0.923 0.138 (0.651,1.19
4)(n = 1509 babies) GEE 0.557 0.882 0.155 (0.626,1.24
4)
Randomization was done independently for twins. The multiple gestation rates were 15% and 16% for the 501 to 1000 g babies and 1001 to 1500 g babies, respectively.
WHAT SHOULD WE DO? Because different methods of analysis can
give different results it is important to specify the analysis before the data are available, to prevent ‘creative accounting’
Subjects randomized as a cluster should be analyzed as a cluster, some techniques seem more conservative and may inflate the risk of type 2 error
WHAT SHOULD WE DO? When subjects are randomized
independently, subjects that are in the same group should be analyzed by methods that can adapt to the degree of correlation Reasonable to plan from the start that if the
correlation is less than a certain limit, it will not be adjusted for. EG ≤ 0.2
PUBLICATION OF PROTOCOLS Protocols as designed at the start of a trial
may have different primary outcomes to those actually published.
A major reason seems to be that the primary outcome was not significant!
Chen et al CMAJ 2004
WHAT SHOULD WE DO Develop empirical data concerning the effects
of different randomization models on consent rates
Protocols should be published and, in addition to other details such as the primary outcome, should detail analysis methods Including the methods that will be used for twins
During enrolment it should be clearly identified in the data set if siblings are in the trial
Original data should be available, to enable SR and meta-analysis which replicates the original analysis
WHICH RANDOMIZATION METHOD IS BEST? In terms of the mathematical consequences
randomizing the first eligible infant, and entering the second in the alternate group Genetic and other risk factors balanced Maintains or even increases power Questionable acceptability
What about triplets? Multiple group trials? For some families randomizing 1st eligible infant
and entering siblings in the same group may be only acceptable method Don’t know how frequent this is, may ensure good
representation of multiples Effectively discounts the contribution of the multiples
to the analysis
WHICH RANDOMIZATION METHOD IS BEST? Independent randomization has less effect on
power Examples I showed have good twin
representation Doesn’t completely eliminate the need to
adjust the analysis for those multiples which are in the same group
Preserves the value of the contribution of the multiples
HOW DOES THIS AFFECT THE FAMILIES? Are there consequences of the method of
enrollment on the families in the long term? If one dies or does poorly? If there is an important treatment effect?
Little information of the long term effects of trial participation, particularly for multiples