statistical methodology for evaluating a cell mediated immunity-based hiv vaccine

Statistical Methodology for Evaluating a Cell Mediated Immunity-Based HIV

Vaccine

Devan V. Mehrotra* and Xiaoming LiMerck Research Laboratories, Blue Bell, PA

*e-mail: [email protected]

Biostat 578A Lecture 4Adapted from Devan’s presentation at the ASA/Northeastern Illinois Chapter Meeting

October 14, 2004

2

Outline

• Science behind the numbers• Merck’s HIV vaccine project• Proof of concept (POC) efficacy study• Statistical methods• Simulation study• Concluding remarks

3

Worldwide Distribution of HIV-1 Clades (Subtypes)*

Note: *Dominant clades are bolded above; All regions have multiple clades in their populations

B B, BC

B

C

A, B, AB, Other

G

B, F, Other

B, FB, AE

B, Other

AE, B, Other B, AE, Other

B, Other

B

O

B, O

A

A

All

C

C, Other

B, Other

A, Other

G, Other

G, OtherAG

A,C,D

Legend

B dominant + Another

C

O

A

All

B, AE

B, G, OtherC

B, C

F

Other

B

4

T Cell Recognition of Infected Cells

5

HIV Infection: CD4 cell count and Viral Load

6

Merck’s HIV Vaccine Project

• Lead vaccine is an Adenovirus type 5 (Ad5) vector encoding HIV-1 gag, pol and nef genes

• Goal: to induce broad cell mediated immune (CMI) responses against HIV that provide at least one of the following:

Protection from HIV infection: acquisition or sterilizing immunity.

Protection from disease: if infected, low HIV RNA “set point”, preservation of CD4 cells, long term non-progressor (LTNP)-like clinical state.

7

Proof of Concept (POC) Efficacy Study

• Design- Randomized, double-blind, placebo-controlled- Subjects at high risk of acquiring HIV infection- HIV diagnostic test every 6 mos. (~ 3 yrs. f/up)

• Co-Primary Endpoints- HIV infection status (infected/uninfected)- Viral load set-point (vRNA at ~ 3 months after diagnosis of HIV infection)

• Secondary/exploratory endpoints: vRNA at 6-18 months, rate of CD4 decline, time to initiation of antiretroviral therapy, etc., for infected subjects

8

POC Efficacy Study (continued)

• Vaccine Efficacy (VE) =

• Null Hypothesis: Vaccine is same as Placebo Same HIV infection rates (VE = 0) and Same distribution of viral load among infected subjs.

• Alternative Hypothesis: Vaccine is better than Placebo Lower HIV infection rate (VE > 0) and/or Lower viral load for infected subjects who got vaccine

• Proof of Concept: reject above composite null hypothesis with at least 95% confidence

PLACEBO}|infectionPr{HIV VACCINE}|infectionPr{HIV

1

9

Notation for Statistical Methodology

Vaccine Placebo

Number Randomized vN cN

Number I nf ected vn cn

Proportion I nfected v

vv N

np

c

cc N

np

log10(vRNA) of I nf ected Subjects

vnx

x

1

cny

y

1

Let cvv

c ppDNN

r , ,

vn

iixRankS

1

)(

(ranking done af ter pooling x’s and y’s together)

10

Notation (cont’d)

C o m p a r i s o n o f H I V i n f e c t i o n r a t e s

L e t p 1 = o n e - t a i l e d p - v a l u e u s i n g a n e x a c t c o n d i t i o n a l b i n o m i a l t e s t

))1

1,(~|Pr(1 r

nnBinomialBnBp cvv

L e t Z 1 = t e s t s t a t i s t i c f o r c o m p a r i n g i n f e c t i o n r a t e s u s i n g a n a p p r o x i m a t e c o n d i t i o n a l t e s t

)]()1/[(

)1/(1)/(

),|(

,|2

0

01

cv

cvv

cv

cv

nnrr

rnnn

HnnDVar

HnnDEDZ

U n d e r t h e n u l l , Z 1 i s a p p r o x i m a t e l y N ( 0 , 1 )

11

Notation (cont’d)

Comparison of Viral Load (among infected subjects)

Let p2 = one-tailed p-value using an exact conditional Wilcoxon Rank Sum test (PROC TWOSAMPL in Proc-StatXact)

Let Z2 = approximate conditional Wilcoxon Rank Sum test

),,|(

),,|(

0

02 HnnnSVar

HnnnSESZ

vcv

vcv

Under the null, Z2 is approximately N(0,1)

Note: p1 and p2 are stochastically independent

12

Competing Methods for Establishing POC

2- part- model (χ2) (Lachenbruch, 2001): Reject null if 2

1,222

21

2 ZZ

Note: Suitable f or 2-tailed test, but not f or our 1-tailed test of interest

2- part- model (Z) (O’Brien, 1984):

Reject null if 121

2Z

ZZZ

Note: both endpoints get equal weight

Weighted- Z (Follmann, 1995):

Reject null if 12

221

2211 Zww

ZwZwZ

where w1 and w2 are pre-specifi ed weights

Optimal weights: 1221

11 1,

)()()(

wwZEZE

ZEw

13

Optimal Weights for Viral Load Component of Composite Test (w2) under Different Scenarios

True VE (%)

δ = true mean diff. (placebo – vaccine) in log10(vRNA) among infected subjects

.5 .6 .8 1.0 1.3 1.5 2.0

0% ~1 ~1 ~1 ~1 ~1 ~1 ~1

15% .78 .81 .84 .86 .88 .89 .91

30% .62 .65 .71 .74 .78 .79 .81

45% .49 .53 .59 .63 .67 .69 .72

60% .38 .42 .48 .52 .57 .59 .62

75% .28 .31 .37 .41 .45 .48 .51

90% .17 .19 .23 .27 .31 .32 .36

14

Simes test (Simes, 1986): Reject null if 2/),min(),max( 2121 pporpp

Weighted- Simes (Hochberg and Liberman, 1994):

Reject null if 2/),min( ),max(2

2

1

1

2

2

1

1 vp

vp

orvp

vp

where ,2 ,2 2211 wvwv and w1 and w2 are pre- specifi ed weights (same as before)

Methods for Establishing POC (cont’d)

15


Fisher’s combined p- value method (Fisher, 1932): Test is based on 2

12

1

21 ppq 4,log41log4Pr 2

)4( q--PROBCHIqvaluep ee

Reject null if valuep Note: both endpoints get equal weight

Weighted- Fisher’s method (Good, 1955): Test is based on 21

21~ ww ppq

12

2

21

12

11

1 ~~

wwqw

wwqw

valuepww

with 1w and 2w as before 21 ww Reject null if valuep

16

Illustration of Simes, Weighted-Simes, Fisher’s, Weighted-Fisher’s Methods (Hypothetical Examples)

Note: w1 = .14, w2 = .86 for weighted-Simes’ and weighted-Fisher’s methods

Composite p-value (Reject composite null hypothesis?)

p-value f or inf ection

endpoint (p1)

p-value f or viral load

endpoint (p2) Simes’ W-Simes’ Fisher’s W-Fisher’s 0.040

0.047 0.012 0.028

0.040 0.040 (Y) (Y) (Y) (Y)

0.080

0.047 0.037 0.035 0.150 0.040

(N) (Y) (Y) (Y) 0.048 0.028 0.065 0.026

0.500 0.024 (Y) (Y) (N) (Y)

0.040 0.071 0.014 0.063 (N)

0.020 0.100 (Y) (N) (Y) (N)

17

Critical Boundaries: Simes, Weighted-Simes, Fisher’s, Weighted-Fisher’s

Note: w1 = .14, w2 = .86 for weighted Fisher’s method. Boundaries are shown assuming p2 p1

p-value for the Viral Load Endpoint (p2)

p-v

alu

e f

or

the

HIV

-1 I

nfe

ctio

n E

nd

po

int

(p1

)

0.0 0.025 0.050 0.075 0.100 0.125 0.150

0.0

0.1

0.2

0.3

0.4

0.5

Simes (S)Fisher (F)Weighted-Simes (WS)Weighted-Fisher (WF)

F S WFWS

18

Additional Notation for Two Other MethodsBasic Idea: Plug in viral load = 0 for uninfected

subjects Vaccine Placebo

log10(vRNA) of I nf ected Subjects

vnx

x

1

cny

y

1

log10(vRNA) of Uninfected Subjects

vv nN

0

0

cc nN

0

0

Overall average log10(vRNA)

“Burden of illness” per subject (Chang, Guess, Heyse, 1994) v

n

ii

N

xv

1

c

n

ii

N

yc

1

19

Additional Notation for Two Other Methods (cont’d)

Note that

v

n

ii

vv

n

ii

n

xp

N

xvv

11 , where

v

vv N

np

c

n

ii

cc

n

ii

n

yp

N

ycv

11 , where

c

cc N

np

Let cv

cv

NNnn

p = overall HI V infection rate

20

Methods for Establishing POC (cont’d) Burden-of -I llness (BOI ) (Chang, Guess, Heyse, 1994): The diff erence in BOI per subject:

c

n

ii

v

n

ii

N

y

N

xT

cv

11

Let

),|(,|

0

0

HnnTVarHnnTET

Zcv

cvBOI

(see appendix f or details)

Reject null if ZZBOI

21


Overall Wilcoxon Rank Sum Test (af ter plugging in log10(vRNA) = 0 f or all uninfected subjects):

I mplicitly assigns “best rank” (BR) to all uninfected subjects. From Chen, Gould, and Nessly (in press):

22*12 1212

13Z

pp

ppZ

pp

pZ cvBR

11

*1

1

cv

cv

nnpp

ppZ , 2Z is as before (viral load statistic)

*1Z is a score statistic f or comparing two independent

proportions. But it is invalid if the no. of inf ections is fixed in advance since the proportions are correlated!

When both cv pp and (and hence p) are small (e.g. < 5%),

the bulk of the weight in BRZ goes to *1Z .

22

Illustrative Example: Hypothetical Data Placebo Vaccine

Randomized 750 750 I nfected (HI V+) 16 14

297 2,964 52 55

3,281 3,617 275 533

6,098 6,612 556 719

22,641 25,070 1,881 2,181

39,535 49,351 7,187 11,271

72,194 132,388 15,263 39,273

218,419 239,210 53,179 58,534

Observed vRNA at 3 months post HI V+

diagnosis

256,844 266,901

vRNA G. Mean (c/ ml) 24,419 2,280 Median (c/ ml) 32,303 2,031 log10 vRNA Mean 4.3877 3.3580 Median 4.5092 3.3077

INFECTION endpoint: Z1=- 0.3688, p1=0.3561, w1=0.14, VEobs=13% VIRAL LOAD endpoint: Z2=- 2.5150, p2=0.0060, w2=0.86

23

Illustrative Example: Hypothetical Data (cont’d)

Method p-value* POC established? 2-part-model (Z) .0209 Yes

Weighted-Z .0055 Yes Simes .0119 Yes

Weighted-Simes .0069 Yes Fisher’s .0177 Yes

Weighted-Fisher’s .0062 Yes BOI .1463 No

Best-Rank .3453 No * f or the composite null hypotheis

24

Simulation Study

Assumed vRNA distributions f or inf ected subjects: Placebo: log10(vRNA) ~ Normal (log10(30,000), 0.75) Vaccine: Null: log10(vRNA) ~ Normal (log10(30,000), 0.75) Alt.: log10(vRNA) ~ Mixture of normals 20% Normal (log10(27,200) + b, 0.65) 24.3% Normal (log10(27,200) + b - 0.5 , 0.65) 55.7% Normal (log10(27,200) + b – 1.0, 0.65), f or diff erent choices of b; overall SD = 0.91

Overall average diff erence (placebo – vaccine) in log10(vRNA): δ = 1 - b

Pre-specified weights: w1 = 0.14 and w2 = 0.86, assuming VE=15% and δ = 1 log10 copies/ ml (i.e., b = 0)

25

Assumed Distributions for log10(viral laod)

Placebo

μ - δViral Load Set-Point (log10 copies/ml)

SD = 0.75

SD = 0.91

Vaccine

μ

Note: Assumed VL distribution for vaccine is asymmetric and more variable (mixture of vaccine “non-responders” and “responders”)

26

Simulation Study (cont’d)

Total enrollment N = Nv + Nc = 1500, r = Nc:Nv = 1:1 and the number of infections fixed at n = nv + nc

Let = (1 - VE)/ (1 + r - VE); r = 1 [VE = 0 iff = ½]

For a given number of infections n, we drew the number of vaccine inf ections nv f rom Binomial(n, ), and set the number of placebo infections to nc = n - nv

We drew nc and nv viral loads f rom the assumed placebo and vaccine viral load distributions, respectively

For each method, we flagged if the null was rejected

Repeated 5,000 times, calculated type-I error rate and power (under diff erent scenarios) at α = 0.05 (1-sided)

27

Simulation Results: Type-I Error Rate (=5%)T

yp

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90

Total Number of Infections

Two-Part Model (Z)

one-tailed 5% level

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


Weighted Two-Part Model (Z)

+2 S.E. (5,000 iterations)

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


Simes

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


Weighted-Simes

28

Simulation Results: Type-I Error (nominal =5%)

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


Fisher's

one-tailed 5% level

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


Weighted-Fisher's

+2 S.E. (5,000 iterations)

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


Wilcoxon (Best Rank)

Typ

e-I

Err

or

Ra

te (

%)

012345678

10 20 30 40 50 60 70 80 90


BOI (log Scale)

29

Simulation Results: Power ( = 5%, 1-tailed)VE=0%, δ=0.5 VE=0%, δ=1.0

Number of Infections

Po

we

r (%

)

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100

Weighted Fisher'sWeighted-ZWeighted-SimesFisher'sTwo-part (Z)SimesBOIWilcoxon (best rank)


Po

we

r (%

)

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100


30



Po

we

r (%

)

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100



Po

we

r (%

)

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100


31



Po

we

r (%

)

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100



Po

we

r (%

)

0

10

20

30

40

50

60

70

80

90

100

10 20 30 40 50 60 70 80 90 100


32

Number of Infections Required for Establishing POC*

Simes’, Fisher’s, Weighted-Fisher’s methods80% power, =5% (1-tailed)

True (log10 copies/ ml)

0.0 0.5 0.7 0.9 1.0 Vaccine Effi cacy

(%) S, F, WF S, F, WF S, F, WF S, F, WF S, F, WF

0% > 100 93, >100, 76 51, 56, 43 33, 36, 28 28, 31, 23

10% > 100 92, 93, 77 50, 52, 42 33, 36, 27 28, 30, 23

20% > 100 87, 81, 74 49, 48, 42 33, 33, 27 27, 28, 23

30% > 100 77, 67, 71 47, 43, 41 30, 30, 27 27, 26, 23

40% > 100 63, 56, 67 42, 37, 40 30, 27, 27 25, 23, 23

50% 77, 80, > 100 49, 41, 64 35, 30, 39 28, 23, 27 23, 21, 23

60% 47, 49, > 100 35, 30, 58 30, 26, 38 23, 21, 27 23, 18, 23

70% 28, 30, > 100 25, 21, 52 23, 19, 37 20, 16, 27 17, 16, 23

* POC is established if the composite null hypothesis is rejected

33

Challenge for the Merck Vaccine

• Pre-existing immunity to Adenovirus Type 5 may prevent or dampen the T cell response to the HIV proteins

• In the U.S., ~30-50% of people have neutralizing antibodies to Ad-5 virus

• In Southern Africa, ~75-95% of people neutralize Ad-5

• Summary of data from Phase I-II trials– Ad-5 Neut Titers < 18: ~80% vaccinees have a CD8+

ELISpot response– Ad-5 Neut Titers > 1000: ~40% have a response– In responders, geometric mean titer ~200 for

vaccinees with Ad-5 Neut Titers < 18; ~100 for vaccinees with Ad-5 Neut Titers > 1000

34

Concluding Remarks

• For a POC trial of a CMI-based HIV vaccine, Fisher’s (and Simes’) methods are good choices.

• If the composite null hypothesis is rejected at the 5% level, the p-values for the two endpoints can each be assessed separately at the 5% level.

• Challenges for the viral load analysis:- Initiation of antiretroviral therapy < 3

months after HIV+ diagnosis (“missing” vRNA data)

- Important to add “sensitivity analyses” to safeguard against potential selection

bias (e.g., Gilbert et al, 2003).- Estimating causal effect of vaccine on post-

infection viral load (ongoing research)

35

Appendix Conditional variance f or BOI statistic (Chang, Guess, Heyse, 1994):

Let cv

n

i

n

iii

nn

yxa

v c

1 1

1

)( 2

12

v

n

ii

x n

xxs

v

and 1

)( 2

12

c

n

ii

y n

yys

c

Then )//

)((),|(ˆ222

0cv

cyvx

cvcvcv NN

NsNs

NNa

nnHnnTV

36

References• Chang MN, Guess HA, Heyse JF (1994). Reduction in the burden of

illness: a new efficacy measure for prevention trials. Statistics in Medicine, 13, 1807-1814.

• Chen J, Gould AL, Nessly ML. Comparing two treatments by using a biomarker with assay limit. Statistics in Medicine, in press.

• Fisher RA (1932). Statistical methods for research workers. Oliver and Boyd, Edinburgh and London.

• Follman D (1995). Multivariate tests for multiple endpoints in clinical trials. Statistics in Medicine, 14, 1163-1175.

• Gilbert PB, Bosch RJ, Hudgens MG. Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIv vaccine clinical trials. Biometrics, 59, 531-541.

• Good IJ (1955). On the weighted combination of significance tests. Biometrika, 264-265.

• Hochberg Y, Liberman U (1994). An extended Simes’ test. Statistics & Probability Letters, 21, 101-105.

• Lachenbruch PA (1976). Analysis of data with clumping at zero. Biometrische Zeitschrift, 18, 351-356.

• O’Brien PC (1984). Procedures for comparing samples with multiple endpoints. Biometrics, 40, 1079-1087.

statistical methodology for evaluating a cell mediated immunity-based hiv vaccine

Documents

hiv infection rates

low hiv rna

infected subjs

approximate conditional

vaccine proof of concept

distribution of viral

viral load set

test statistic