probabilities bayesian predictive
TRANSCRIPT
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 1
Sequential Monitoring of ClinicalTrialsSession 4 - Bayesian Evaluation of Group Sequential Designs
Presented August 8-10, 2012
Daniel L. GillenDepartment of Statistics
University of California, Irvine
John M. KittelsonDepartment of Biostatistics & Informatics
University of Colorado Denver
c©2012 Daniel L. Gillen, PhD and John M. Kittelson, PhD
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 2
Bayesian paradigm
Bayesian Operating Characteristics
I Thus far, we have primarily focused on the evaluation of aclinical trial design with respect to frequentist operatingcharacteristics
I type I errorI statistical powerI sample size requirementsI estimates of treatment effect that correspond to early
terminationI precision of confidence intervals
I However, there has been much interest in the design andanalysis of clinical trials under a Bayesian paradigm
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 3
Bayesian paradigm
Bayesian Operating Characteristics
I In the Bayesian paradigm, we consider a joint distributionp(θ, X ) for the treatment effect parameter θ and the clinicaltrial data X .
I We most often specify the prior distribution pθ(θ) and thelikelihood function pX |θ(X |θ), rather than specifying thejoint distribution p(θ, X )
I The prior distribution pθ(θ) represents the information aboutθ prior to (in the absence of) any knowledge of the value ofX
I From a clinical trial, we observe data X = x and baseinference on the posterior distribution pθ|X (θ|X = x),where by Bayes rule
pθ|X (θ|X = x) =pX |θ(X |θ) pθ(θ)∫pX |θ(X |θ) pθ(θ) dθ
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 4
Bayesian paradigm
Bayesian Operating Characteristics
I As with frequentist inference, we are interested in:
I point and interval estimates of a treatment effect,
I a measure of strength of evidence for or against particularhypotheses, and perhaps
I a binary decision for or against some hypothesis.
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 5
Bayesian paradigm
Bayesian Operating Characteristics
I Bayesian inferential quantities include:
1. Point estimates of treatment effect that are summarymeasures of the posterior distribution such as:
1.1 the posterior mean : E(θ|X = x),
1.2 the posterior median : θ0.5 such that
Pr(θ ≤ θ0.5|X = x) ≥ 0.5 and Pr(θ ≥ θ0.5|X = x) ≥ 0.5,
1.3 the posterior mode : θm such that
pθ|X (θm|X = x) ≥ pθ|X (θ|X = x) for all θ
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 6
Bayesian paradigm
Bayesian Operating Characteristics
2. Interval estimates of treatment effect that are computed byfinding two values (θL, θU) such that
Pr(θL ≤ θ ≤ θU |X = x) = 100(1 − α)%
2.1 the central 100(1− α)% of the posterior distribution of θ isdefined by finding some ∆ such that θL = θ̂ −∆ andθU = θ̂ + ∆ provides the desired coverage probability, whereθ̂ is one of the Bayesian point estimates of θ
2.2 the interquantile interval is defined by defining θL = θα/2 andθU = θ1−α/2, where θp is the p-th quantile of the posteriordistribution, i.e., Pr(θ ≤ θp|X = x) = p
2.3 the highest posterior density (HPD) interval is defined byfinding some threshold cα such that the choices
θL = min{θ : pθ|X (θ|X = x) > cα} and
θU = max{θ : pθ|X (θ|X = x) > cα}
provide the desired coverage probability
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 7
Bayesian paradigm
Bayesian Operating Characteristics
3. Posterior probabilities or Bayes factors associated withspecific hypotheses that might be used to make a decisionfor or against a particular hypothesis.
3.1 For instance, in the sepsis trial example, we might beinterested in computing the posterior probability of the nullhypothesis Pr(θ ≥ 0|X = x) or the posterior probability of thedesign alternative Pr(θ ≤ −0.07|X = x).
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 8
Bayesian paradigm
Bayesian Operating Characteristics
I A note on Bayesian inference under group sequentialtesting
I The Bayesian inference presented on the previous slides isunaffected by the choice of stopping rule, so long as there isno need to consider the joint distribution of estimates acrossthe multiple analyses of the accruing data
I So long as one is content to regard inference at each analysismarginally, then the stopping rule used to collect the data isimmaterial
I However, the expected cost of a clinical trial does dependvery much on the stopping rule used, even when Bayesianinference is used as the basis for a decision
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 9
Bayesian paradigm
Bayesian posterior probability scale in RCTdesign
I In considering the Bayesian approach, we maintain thestance that the derivation of the stopping rule is relativelyunimportant.
I There is a 1:1 correspondence between stopping rulesdefined for frequentist statistics (on a variety of scales)and Bayesian statistics for a specified prior
I Specifically, in RCTdesign we consider a robust approachto Bayesian inference based on a coarsening of the databy using the asymptotic distribution of a nonparametricestimate of treatment effect
I For example, in the case of the sepsis trial, we use theapproximate normal distribution for the difference in 28 daymortality rates
θ̂ ≡ p̂1 − p̂0∼̇N(
θ,p1(1− p1) + p0(1− p0)
N
)
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 10
Bayesian paradigm
Bayesian posterior probability scale in RCTdesign
I Reasoning behind the “coarsened” Bayesian approach
1. Robustness of inference to misspecification of the likelihood
2. No true conflict between frequentist and Bayesian inference.Instead, they merely answer different questions.
2.1 Role of statistics is to help quantify the strength of evidenceused to convince the scientific community
2.2 Reasonable people might demand evidence:
(1) demonstrating results that would not typically be obtainedunder any other hypothesis (frequentist), or
(2) that overpowers his/her prior beliefs
2.3 To address these different standards of proof within the samesetting, it would seem most appropriate to use the sameprobability model in each approach
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 11
Bayesian paradigm
Bayesian posterior probability scale in RCTdesign
I Reliance on the asymptotic distribution of the estimatorimplies that a normal prior is conjugate andcomputationally convenient
θ ∼ N(ζ, τ2)
I Thus we can define a Bayesian posterior probabilitystatistic by computing the approximate posteriorprobability that the null hypothesis H0 : θ ≥ θ0 is false
Bj(ζ, τ2, θ0) = Pr(θ ≤ θ0 |Sj = sj)
= Φ
(θ0[Njτ
2 + V ]− τ2sj − V ζ√V τ√
Njτ2 + V
),
where V = p0(1− p0) + p1(1− p1) and Φ(z) is thecumulative distribution function for a standard normalrandom variable
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 12
Bayesian paradigm
Bayesian posterior probability scale in RCTdesign
I Reliance on the asymptotic distribution of the estimatorimplies that a normal prior is conjugate andcomputationally convenient
θ ∼ N(ζ, τ2)
I Thus we can define a Bayesian posterior probabilitystatistic by computing the approximate posteriorprobability that the null hypothesis H0 : θ ≥ θ0 is false
Bj(ζ, τ2, θ0) = Pr(θ ≤ θ0 |Sj = sj)
= Φ
(θ0[Njτ
2 + V ]− τ2sj − V ζ√V τ√
Njτ2 + V
),
where V = p0(1− p0) + p1(1− p1) and Φ(z) is thecumulative distribution function for a standard normalrandom variable
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 13
Case Study : Sepsis Trial
Boundaries on various design scales
I Bayesian posterior probabilities for observed estimates onthe Futility.8 boundary
I Computed using a optimistic (ζ = −0.09) dogmatic prior(τ2 = 0.0152) prior
> changeSeqScale(Futility.8,seqScale("B",priorTheta=-0.09, priorVariation=0.015^2))
STOPPING BOUNDARIES: Bayesian scale(Posterior probability of hypotheses based on priordistribution having median -0.09and variation parameter 0.000225)
a dTime 1 (N= 425) 1 0.7954Time 2 (N= 850) 1 0.8239Time 3 (N= 1275) 1 0.8361Time 4 (N= 1700) 1 0.8423
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 14
Case Study : Sepsis Trial
Boundaries on various design scales
I Example interpretation (1st analysis, efficacy bound (a) ):Under the optimistic dogmatic prior, after sampling 425subjects if the observed rate in the treatment arm were16.97% lower than the control arm, the posteriorprobability of the event θ < 0 is computed to be approx 1
I Example interpretation (1st analysis, futility bound (d) ):Under the optimistic dogmatic prior, after sampling 425subjects if the observed rate in the treatment arm were4.73% lower than the control arm, the posterior probabilityof the event θ < −0.0866 is computed to be 0.7954
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 15
Case Study : Sepsis Trial
Boundaries on various design scales
I Bayesian posterior probabilities for observed estimates onthe Futility.8 boundary
I Computed using a pessimistic (ζ = 0.02) dogmatic prior(τ2 = 0.0152) prior
> changeSeqScale(Futility.8,seqScale("B",priorTheta= 0.02, priorVariation=0.015^2))
STOPPING BOUNDARIES: Bayesian scale(Posterior probability of hypotheses based on priordistribution having median 0.02and variation parameter 0.000225)
a dTime 1 (N= 425) 0.5240 1Time 2 (N= 850) 0.5228 1Time 3 (N= 1275) 0.5218 1Time 4 (N= 1700) 0.5208 1
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 16
Case Study : Sepsis Trial
Boundaries on various design scales
I Bayesian posterior probabilities for observed estimates onthe Futility.8 boundary
I Computed using a consensus (ζ = −0.04) dogmatic prior(τ2 = 0.0152) prior
> changeSeqScale(Futility.8,seqScale("B",priorTheta=-0.04,priorVariation=0.015^2))
STOPPING BOUNDARIES: Bayesian scale(Posterior probability of hypotheses based on priordistribution having median -0.04and variation parameter 0.000225)
a dTime 1 (N= 425) 0.9999 1.0000Time 2 (N= 850) 0.9999 1.0000Time 3 (N= 1275) 0.9997 0.9999Time 4 (N= 1700) 0.9996 0.9999
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 17
Case Study : Sepsis Trial
Boundaries on various design scales
I Note 1: By default, seqScale() will compute theposterior probability of the hypothesis being tested foreach boundary
I seqDesign() defaults to a symmetric test so that thefutility boundary tests the (1−α) power point for with size α
I For the Futility.8 design, this is the 97.5% power pointcorresponding to θ = −0.0866
I To see this, one can view the Futility.8$hypothesis
> Futility.8$hypothesis
HYPOTHESES:Theta is difference in probabilities (Treatment - Comparison)One-sided hypothesis test of a lesser alternative:
Null hypothesis : Theta >= 0.00 (size = 0.0250)Alternative hypothesis : Theta <= -0.07 (power = 0.8888)
Boundary hypotheses:Boundary a rejects : Theta >= 0.0000 (lower size = 0.025)Boundary b rejects : Theta <= -0.0866 (lower power = 0.975)Boundary c rejects : Theta >= 0.0000 (upper power = 0.975)Boundary d rejects : Theta <= -0.0866 (upper size = 0.025)
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 18
Case Study : Sepsis Trial
Boundaries on various design scales
I Note 2: An asymmetric boundary can be obtained bymodifying the alpha option in seqDesign()
I As this changes the hypotheses being tested at eachanalysis, it will change the boundaries and operatingcharacteristics of the stopping rule
I Note 3: To simply compute a different posterior probability,you can use the threshold option in seqScale()
changeSeqScale(Futility.8,seqScale("B", threshold=-0.06,
priorTheta=-0.09,priorVariation=0.015^2))
STOPPING BOUNDARIES: Bayesian scale(Posterior probability that treatment effect exceeds -0.06based on prior distribution having median -0.09and variation parameter 0.000225)
a dTime 1 (N= 425) 0.0031 0.1461Time 2 (N= 850) 0.0155 0.1471Time 3 (N= 1275) 0.0509 0.1365Time 4 (N= 1700) 0.1225 0.1225
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 19
Case Study : Sepsis Trial
Boundaries on various design scales
I Bayesian posterior probabilities for observed estimates onthe Futility.8 boundary
I Computed using a non-informative prior (τ =∞)
> changeSeqScale(Futility.8,seqScale("B",priorTheta= 0.02,priorVariation=Inf))
STOPPING BOUNDARIES: Bayesian scale(Posterior probability of hypotheses based on priordistribution having median 0.02and variation parameter Inf)
a dTime 1 (N= 425) 1.0000 0.9991Time 2 (N= 850) 0.9975 0.9946Time 3 (N= 1275) 0.9891 0.9880Time 4 (N= 1700) 0.9766 0.9808
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 20
Case Study : Sepsis Trial
Sensitivity of posterior probabilities to prior
I Bayesian inference can depend heavily on the choice ofprior distribution pθ(θ) for the treatment effect parameter
I For that reason, Bayesian inferential procedures havesometimes been criticized because it is not clear how theprior distribution should be selected for any particularproblem
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 21
Case Study : Sepsis Trial
Sensitivity of posterior probabilities to prior
I The Bayesian evaluation of stopping rules proceeds muchthe same as for the evaluation of stopping rules withrespect to frequentist inference
I The major difference relates to the magnitude of theresults that need to be presented
I When evaluating a stopping rule with respect to frequentistinference, we present the estimate, confidence interval, andP value for clinical trial results corresponding to the stoppingboundaries
I For Bayesian inference, we must consider how thatinference is affected by the choice of prior
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 22
Case Study : Sepsis Trial
Sensitivity of posterior probabilities to prior
I In RCTdesign, the approach we take is to use a spectrumof normal prior distributions specified by their mean andstandard deviation
I Normal prior is specified entirely by two parameters, greatlyreducing the dimension of the space of prior distributions tobe considered
I Means and standard deviations are commonly used andunderstood by many researchers
I Normal distribution is known to maximize entropy over theclass of all priors having the same first two moments
I Potentially deleterious if an individual’s well-based prior ismore informative than the normal prior with the same meanand standard deviation
I In this case, an analysis with a normal prior may suggest thatthe data has overwhelmed an investigator’s initial belief, whenit in fact has not
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 23
Case Study : Sepsis Trial
Sensitivity of posterior probabilities to prior
Bayesian Evaluation of Group Sequential Designs - 05/01/2006, 6
Table 1: Posterior probabilities of hypotheses for trial results corresponding to stopping boundaries of Futility.8 stopping rulewith four equally spaced analyses after 425, 850, 1275, and 1700 subjects have been accrued to the study. Posterior probabilitiesare computed based on optimistic, the sponsor’s consensus, and pessimistic centering of the priors using three levels of assumedinformation in the prior. The variability of the likelihood of the data corresponds to the alternative hypothesis: event rates of
0.30 in the control group and 0.23 in the treatment group.
Efficacy (lower) Boundary Futility (upper) Boundary
Posterior Probability of Posterior Probability ofBeneficial Treatment Effect Insufficient Benefit
Pr(θ < 0|X) Pr(θ ≥ −0.087|X)
Crude Est Sponsor’s Crude Est Sponsor’sAnalysis of Trt Optimistic Consensus Pessimistic of Trt Optimistic Consensus Pessimistic
Time Effect ζ = −.09 ζ = −.04 ζ = .02 Effect ζ = −.09 ζ = −.04 ζ = .02
Dogmatic Prior: τ = 0.0151:N=425 -0.170 1.000 1.000 0.524 0.047 0.795 1.000 1.0002:N=850 -0.085 1.000 1.000 0.523 -0.010 0.824 1.000 1.0003:N=1275 -0.057 1.000 1.000 0.522 -0.031 0.836 1.000 1.0004:N=1700 -0.042 1.000 1.000 0.521 -0.042 0.842 1.000 1.000
Consensus Prior: τ = 0.0401:N= 425 -0.170 1.000 1.000 0.991 0.047 0.981 0.999 1.0002:N= 850 -0.085 1.000 0.998 0.974 -0.010 0.976 0.997 1.0003:N=1275 -0.057 0.999 0.993 0.955 -0.031 0.970 0.994 1.0004:N=1700 -0.042 0.998 0.987 0.936 -0.042 0.963 0.991 0.999
Noninformative Prior: τ = ∞1:N= 425 -0.170 1.000 1.000 1.000 0.047 0.999 0.999 0.9992:N= 850 -0.085 0.998 0.998 0.998 -0.010 0.995 0.995 0.9953:N=1275 -0.057 0.989 0.989 0.989 -0.031 0.988 0.988 0.9884:N=1700 -0.042 0.977 0.977 0.977 -0.042 0.981 0.981 0.981
interval estimates of a treatment effect, a measure of strength of evidence for or against particular
hypotheses, and perhaps a binary decision for or against some hypothesis. Commonly used Bayesian
inferential quantities include:
1. Point estimates of treatment effect that are summary measures of the posterior distri-
bution such as the posterior mean (E(θ|X = x)), the posterior median (θ0.5 such that
Pr(θ < θ0.5|X = x) ≥ 0.5 and Pr(θ ≥ θ0.5|X = x) ≥ 0.5), or the posterior mode (θm such
that pθ|X(θm|X = x) ≥ pθ|X(θ|X = x) for all θ).
2. Interval estimates of treatment effect that are computed by finding two values (θL, θU ) such
that Pr(θL < θ < θU |X = x) = 100(1 − α)%. Various criteria can be used to define such
“credible intervals”:
(a) the central 100(1 − α)% of the posterior distribution of θ is defined by finding some ∆
such that θL = θ̂ − ∆ and θU = θ̂ + ∆ provides the desired coverage probability, where
θ̂ is one of the Bayesian point estimates of θ.
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 24
Case Study : Sepsis Trial
Sensitivity of posterior probabilities to prior
I As is often the case, the optimistic and pessimistic priorspresented were chosen rather arbitrarily, and thus may notbe relevant to some of the intended audience for thepublished results of a clinical trial
I As such, it may be beneficial to present contour plots ofBayesian point estimates (posterior means), lower andupper bounds of 95% credible intervals, and posteriorprobabilities of the null and alternative hypotheses for aspectrum of prior distributions
I In RCTdesign, such plots can be produced withseqBayesContour()
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 25
Case Study : Sepsis Trial
Contour plot of posterior median conditional upon stopping atanalysis 2 for futility (observed statistic on boundary)
−0.10 −0.08 −0.06 −0.04 −0.02 0.00 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r S
D fo
r θ
0.990 0.900 0.500 0.200 0.025Power (lower) to detect θ
2.00
0.50
0.20
0.10
0.05
Prio
r In
form
atio
n A
bout
θ a
s P
ropo
rtio
n o
f Inf
orm
atio
n fr
om M
axim
al S
ampl
e S
ize
XO
Mdn(θ l M = 2, T = −0.01)
−0.082
−0.067−0.052
−0.036
−0.021−0.006
0.01
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 26
Case Study : Sepsis Trial
Contour plot of posterior median conditional upon stopping atanalysis 2 for futility (user-defined contours)
−0.10 −0.08 −0.06 −0.04 −0.02 0.00 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r S
D fo
r θ
0.990 0.900 0.500 0.200 0.025Power (lower) to detect θ
2.00
0.50
0.20
0.10
0.05
Prio
r In
form
atio
n A
bout
θ a
s P
ropo
rtio
n o
f Inf
orm
atio
n fr
om M
axim
al S
ampl
e S
ize
XO
Mdn(θ l M = 2, T = −0.01)
−0.09−0.08
−0.07−0.06
−0.05−0.04
−0.03
−0.02−0.01
0
0.01
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 27
Case Study : Sepsis Trial
Contour plot of credible interval bounds at analysis 2 futilitybound (user-defined contours)
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r S
D fo
r θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
XO
Q0.025(θ l M = 2, T = −0.01)
−0.1
−0.09
−0.08
−0.07
−0.06
−0.05
−0.04−0.03
−0.02−0.01
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
2.00
0.50
0.20
0.10
0.05
Prio
r In
form
atio
n A
bout
θ a
s P
ropo
rtio
n o
f Inf
orm
atio
n fr
om M
axim
al S
ampl
e S
ize
XO
Q0.975(θ l M = 2, T = −0.01)
−0.06−0.05
−0.04−0.03−0.02−0.01
00.01
0.02
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 28
Case Study : Sepsis Trial
Contour plot of posterior probabilities for a 2 analysis design
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r S
D fo
r θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
XO
Pr(θ > −0.0849) (prior)
0.9990.99
0.9750.95
0.90.5
0.1
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
XO
Pr(θ > −0.0849 l M = 1, T = −0.01)
0.999
0.99
0.9750.95
0.9
0.5
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
2.00
0.20
0.10
0.05
Prio
r In
form
atio
n A
bout
θ a
s P
ropo
rtio
n o
f Inf
orm
atio
n fr
om M
axim
al S
ampl
e S
ize
XO
Pr(θ > −0.0849 l M = 2, T = −0.042)
0.999
0.99
0.975
0.95
0.9
0.5
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r S
D fo
r θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
XO
Pr(θ < 0) (prior)
0.1
0.50.9
0.9750.99
0.999
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
XO
Pr(θ < 0 l M = 1, T = −0.084)
0.5
0.9
0.975
0.99
0.999
−0.10 −0.06 −0.02 0.02
Prior Median for θ
0.990 0.800 0.200 0.010Power (lower) to detect θ
2.00
0.20
0.10
0.05
Prio
r In
form
atio
n A
bout
θ a
s P
ropo
rtio
n o
f Inf
orm
atio
n fr
om M
axim
al S
ampl
e S
ize
XO
Pr(θ < 0 l M = 2, T = −0.042)
0.5
0.9
0.975
0.99
0.999
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 29
Case Study : Sepsis Trial
Contour plot of ASN for Futility.8 design
−0.10 −0.08 −0.06 −0.04 −0.02 0.00 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r SD
for θ
0.990 0.900 0.500 0.200 0.025Power (lower) to detect θ
2.00
0.50
0.20
0.10
0.05
Prio
r Inf
orm
atio
n Ab
out θ
as
Prop
ortio
n o
f Inf
orm
atio
n fro
m M
axim
al S
ampl
e Si
ze
XO
Expected Sample Size Under Prior
850900
950
9501000
10001050
10501100
11001150
1200
1250
1300
Contours of Sample Size Distribution
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 30
Case Study : Sepsis Trial
The seqBayesContour() function
I Important arguments to seqBayesContour() are:
dsn : the seqDesign() object used for computingBayesian operating characteristics
analysis.index : the analysis time to condition on; if missing,operating characteristics will be computedfor all analyses
observed : the observed statistic to condition on; ifmissing, boundary values are used
priorTheta : mean of prior distribution
priorVariation : variance of prior distribution
thetaThreshold : threshold for computing posterior probabilities;if missing, boundary hypotheses are used
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 31
Case Study : Sepsis Trial
The seqBayesContour() function
I Important arguments to seqBayesContour() are:
posteriorProbContours : specifies contour lines for whichposterior probabilities are to be drawn;defaults to c(0.001, 0.01, 0.025, 0.05,0.1, 0.5, 0.9, 0.975, 0.99, 0.999)
posteriorMeanContours : if missing, plot with posterior meancontours will be drawn; if NULL, noplot will be drawn
lowerCredibleBoundContours : if missing, plot with contoursrepresenting lower limits of specifiedcredible interval will be drawn; ifNULL, no plot will be drawn
upperCredibleBoundContours : same as above
PlotASN : ASN contours; defaults to TRUE
SampSizeQuantiles : quantiles of sample size distributionfor which contours are to be drawn;defaults to 0.75
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 32
Bayesian paradigm
Predictive probability scale in RCTdesign
I A Bayesian approach similar to the stochastic curtailmentprocedures would consider the Bayesian predictiveprobability that the test statistic would exceed somespecified threshold at the final analysis
I This statistic uses a prior distribution and the observeddata to compute a posterior distribution for the treatmenteffect parameter at the j-th analysis
I Using the sampling distribution for the as yet unobserveddata and integrating over the posterior distribution, thepredictive distribution of the test statistic at the finalanalysis can be computed
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 33
Bayesian paradigm
Predictive probability scale in RCTdesign
I Again relying on the asymptotic distribution of theestimator and utilizing a normal prior, θ ∼ N(ζ, τ2), we cancompute a predictive probability statistic analogous to aconditional power statistic as
Hj (c, ζ, τ2) =
ZPr(θ̂J < c |Sj = sj , θ) p(θ |Sj = sj ) dθ
= Φ
0B@ [Πjτ2 + σ2/NJ ][c − θ̂j ] + [1 − Πj ][θ̂j − ζ]σ2/NJq[1 − Πj ][τ2 + σ2/NJ ][Πjτ2 + σ2/Nj ]σ2/NJ
1CA .
I Note that taking the limit as τ2 → 0 (yielding a point-massprior at ζ), the predictive probability statistic converges tothe conditional power statistic discussed previously
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 34
Bayesian paradigm
Probability of observing an economically important estimates oftreatment effect
I Science vs. Marketing
I Just as we separate statistical significance from clinicalsignificance, marketing tends to have it’s own threshold forsignficance
I What will sell?
I In this case, Bayesian predictive probabilities may be ofuse in judging whether it is useful to continue a clinical trialin the hopes of obtaining an economically attractiveestimate of treatment effect
I For example, suppose that in the case of the sepsis trialan observed reduction of 6% (difference) in mortalitywould be deemed highly marketable...
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 35
Case Study : Sepsis Trial
Boundaries on various design scales
I Bayesian predictive probabilities for observed estimateson the Futility.8 boundary
I Computed using a consensus prior (ζ = −0.04,τ2 = 0.042)
> 1-changeSeqScale(Futility.8,seqScale("H",priorTheta=-0.04,priorVariation=.04^2,threshold=-.06))
STOPPING BOUNDARIES: Predictive Probability scale(Predictive probability that estimated treatment effectat the last analysis exceeds -0.06based on prior distribution having median -0.04and variation parameter 0.0016)
a dTime 1 (N= 425) 0.9784 0.0057Time 2 (N= 850) 0.8066 0.0101Time 3 (N= 1275) 0.3501 0.0085Time 4 (N= 1700) 0.0000 0.0000
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 36
Case Study : Sepsis Trial
Boundaries on various design scales
I Bayesian predictive probabilities for observed estimateson the Futility.8 boundary
I Computed using a non-informative prior (τ =∞)
> 1-changeSeqScale(Futility.8,seqScale("H",priorTheta=-0.04,priorVariation=Inf,threshold=-.06))
STOPPING BOUNDARIES: Predictive Probability scale(Predictive probability that estimated treatment effectat the last analysis exceeds -0.06based on prior distribution having median -0.04and variation parameter Inf)
a dTime 1 (N= 425) 0.9985 0.0018Time 2 (N= 850) 0.8778 0.0092Time 3 (N= 1275) 0.3901 0.0093Time 4 (N= 1700) 0.0000 0.0000
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 37
Case Study : Sepsis Trial
Contour plot of predictive probabilities at 3rd efficacy boundary
−0.10 −0.08 −0.06 −0.04 −0.02 0.00 0.02
Prior Median for θ
0.02
0.04
0.06
0.08
Prio
r S
D fo
r θ
0.990 0.900 0.500 0.200 0.025Power (lower) to detect θ
2.00
0.50
0.20
0.10
0.05
Prio
r In
form
atio
n A
bout
θ a
s P
ropo
rtio
n o
f Inf
orm
atio
n fr
om M
axim
al S
ampl
e S
ize
XO
Pr(θ̂J < −0.06 l M = 3, T = −0.057)
0.10.15
0.2
0.25
0.3
0.350.4
0.45
0.5
0.550.6
0.65
0.7
RCTdesign
IntroductionBayesian operatingcharacteristics
Coarsened Bayes approach
Implementation inRCTdesignPosterior probability scale
Bayesian contour plots inRCTdesign
Bayesian PredictiveProbabilities
UW Grp Seq - Sec 4 - pg 38
Case Study : Sepsis Trial
Probability of observing an economically important estimates oftreatment effect
I For a result corresponding to a crude estimate oftreatment effect of -0.0566 at the third analysis, thepredictive probability of obtaining a crude estimate oftreatment effect less than -0.06 at the final analysis is35.0% under the sponsor’s consensus prior and 39.0%under a noninformative prior
I In either case, such high probabilities of obtaining a moreeconomically viable estimate of treatment effect may beenough to warrant modifying the stopping rule to avoidearly termination at the third analysis with a crudeestimate between -0.0566 and -0.06.