alpha recycling in confirmatory clinical trials

Proprietary and Confidential © AstraZeneca 2009FOR INTERNAL USE ONLY

1

O Guilbaud, FMS+Cramér Society, AZ-Södertälje, 2010-11-29

Alpha Recyclingin

Confirmatory Clinical TrialsUnifies, Simplifies, Extends

many common MTPs

Olivier GuilbaudSenior Principal Scientist

AstraZeneca R&D, Södertälje, Sweden


2


Outline

• Background

• Splitting, Recycling, Adding, (parts of alpha)

• Graphs and Default Graphs for MTPs

• Rejection Algorithm

• Holm’s MTP for groups of Hs, and a problem to discuss

• Improvements through extensions of the Default Graph

• Summary, and further results:

• (Simultaneous Confidence regions, CTP-formulation of MTPs)


3


Background: Regulatory environment

Citation from EMEA/CPMP’s (2002) Points to Consider on Multiplicity Issues in Clinical Trials :

“ … multiplicity can have a substantial influence on the rate of false positive conclusions …”

”As a general rule it can be stated that control of the family-wise type-I error rate in the strong sense … is a minimal prerequisite for confirmatory claims”


4


Background: Multiple Confirmatory Comparisons

• Several Efficacy Variables • Several Tolerability Variables • Several Comparators (e.g. Placebo, Active 1, Active 2,

…)• Several Doses

• Delta-Noninferiority / Superiority / Delta-Superiority • Several Subgroups/Kinds of Subjects

• Several kinds of Administration (e.g. once daily, twice daily, …)

Confirmatory:Pre-specified family of multiple comparisons (i.e. statistical tests) .Risk of getting any ”false positive” result/conclusion is (no matter how many or which H0s are true – Strong Control)


5


Background: Abbreviations/Terminology

• MTP : Multiple Testing Procedure

• MCP : Multiple Comparison Procedure

• FWER : Family-Wise Error Rate (type-I errors) =Pr[at least 1 true Hi in the Family is rejected by the

MTP](to be controlled to be in the strong sense)

, no matter how many, or which, His in the family are true

In contrast to the weak sense (old) : , if all His in the family are true


6


Background: brief refresher about three basic MTPs

Three basic MTPs for Family {H1, H2, H3} based on raw p-values p1, p2, p3

• Holm MTP (1979)p(1) /3, p(2) /2, p(3) /1

• Fixed-Sequence MTP (old) p1 , p2 , p3

• Fallback MTP (2003/2005)weights w1, w2, w3 p1 w1, p2 (w2+), p3 (w3+)

Now, to the Alpha-Recycling framework


7


Two articles in Statistics in Medicine, February 2009

Only this now


8


Idea: Splitting and Recycling test-mass (= part of )

Bonferroni:

split of Fixed-Sequence: recycling of Combination of

split & recycling of

Throughout: Raw p-value available for each null hypothesis H


9


Idea: Adding test-mass from different paths

Parallel-gatekeeping MTP: 2 graphs for same MTP

Default graph

Important for rejection AlgorithmIts sequences reflect the possible rejection paths


10



Fallback MTP for 3 Hs: 2 graphs for same MTP

Default graph



11



Holm’s MTP for 3 Hs: 2 graphs for same MTP

Default graph



12


Rejection Algorithm - based on Default-Graph Seqs

This MTP controls (in the strong sense) the FWER to be !

Step 1. For each ”first” H in Sequences,

(a) add-upp test mass from ”its” Sequences(b) test H at this added-up level

If nothing is rejected, then Stop; otherwise:

Reduce Sequences by deleting all rejected H from Sequences, and go to Step 2 with remaining reduced Sequences

Step r = 2, 3, …. For each ”first” H in remaining reduced Sequences,

(a) add-upp test mass from ”its” Sequences(b) test H at this added-up level

If nothing is rejected, then Stop; otherwise:

Reduce Sequences by deleting all rejected H from Sequences, and go to Step r + 1 with remaining reduced Sequences


13


Comments

• number of sequences can be increased(no restriction)

• length of sequences can be increased(no restriction)

• if not all sequences contain all Hs, thenimprovements are possible (by increasing length of relevant sequences)

• interpretability of MTP is important(also when improvements are considered)


14


A refresher – Bonferroni & Holm for groups of Hs

Step1: Bonferroni-version as above

Step2: If one group is entirely rejected, but not the other, then try again

in the other at increased level

1 2

3 4

Bonferroni for 2 groups of 2 Hs, with

Fixed-Seq testing within group

/2 /2

Holm for 2 groups of 2 Hs, with Fixed-Seq testing within group:

1 2

3 4

/2 /2

1

1 2

3 4

/2

1


15


Problem to discuss with your neighbour

1 2

3 4

Bonferroni for 2 groups of 2 Hs, with

Fixed-Seq testing within group

/2 /2

(is already in default-graph form):

Extend sequences of default graph to get recycling improvement equivalent toHolm for 2 groups of 2 Hs, with Fixed-Seq testing within group

1 2

3 4

/2 /2


16


Recent Example discussed by FDA statisticians

• Confirmatory family with 4 null hypotheses :

H1 = (Null hypothesis concerning non-inferiority for primary variable) H2 = (Null hypothesis concerning superiority for primary variable) H3 = (Null hypothesis concerning non-inferiority for secondary variable) H4 = (Null hypothesis concerning superiority for secondary variable)

H1

H2 H3

H4

w (1-w)


17



Alpha-recycling applied to show Non-inf & Superiority for Primary & Secondary Variables

H1

H2 H3

H4

w (1-w)

Basic version

H1

H2 H3

w (1-w)

H1

H4

Reformulation in Recycling terms

Very easy to test with rejection algorithm !

But why not recycle also after the last H in each sequence ?


18



Alpha-recycling applied to show Non-inf & Superiority for Primary & Secondary Variables - Simple improvement

Improvement Reformulation in Recycling terms

H1

H2 H3

w (1-w)

H1

H4H3

H2H4

H1

H2 H3

H4

w (1-w)

Very easy to test with rejection algorithm !This improved MTP is ”-exhaustive” !

May be questionedby some (not me)!

May be questionedby some (not me)!


19


Sequences can be extended/added for more rejections

Improved Fallback MTP for 3 Hs: 2 graphs for same MTP

Default graph

This improved MTP is ”-exhaustive” !


20


Sequences can be extended/added for more rejections

Improved Parallel-gatekeeping : 2 graphs for same MTP

Default graph

This improved MTP is ”-exhaustive” !


21


XXX example: 2 primary & 2 second vars, 2 age stata

1

2 3

4

5 6

8

9 11

7

10 12

Original proposal

Simple improvement possible ( Holm instead of Bonf after H8 ):

Add sub-seq (11, 12) to Default-graph sequences ending with (9, 10) Add sub-seq (9, 10) to Default-graph sequences ending with (11, 12)

1

2

7

8

9

10

/81

3

7

8

9

10

/84

5

7

8

9

10

/84

6

7

8

9

10

/81

2

7

8

11

12

/81

3

7

8

11

12

/84

5

7

8

11

12

/84

6

7

8

11

12

/8

Default graphReformulation in Recycling terms

to show strong control of FWER


22


Compact notation for MTP in XXX example

1

2 3

4

5 6

8

9 11

7

10 12

Original proposal

Compact notation for this recycling MTP (B here is Bonf-operator):

B(H1 B(H2, H3), H4 B(H5, H6)) H7 H8 B(H9 H10, H11 H12)

Can be expanded (using certain rules) to 8 sequences of Hs above

1

2

7

8

9

10

/81

3

7

8

9

10

/84

5

7

8

9

10

/84

6

7

8

9

10

/81

2

7

8

11

12

/81

3

7

8

11

12

/84

5

7

8

11

12

/84

6

7

8

11

12

/8

Default graphReformulation in Recycling terms


23


YYY example: 3 doses vs. 0, 1 primary & 1 second var

1

3

1

5

2

5

2

4

6 6

/4 /4 /4 /4

Reformulation in Recycling termsDefault graph

1 2

3 45

6

Basic version

/2 /2

Idea: To give lowest dose a chance if at least 1 of the 2 larger doses works for the primary variable


24


YYY example – Improvement 1

1

3

1

5

2

5

2

4

6 6

/4 /4 /4 /4


5

6

5

6

1 2

3 45

6

Improvement 1

/2 /2

Increases level for H5 and H6 if H3 and/or H4 are rejected


25


YYY example – Improvement 2

1 2

3 45

6

Improvement 2

1/2 1/2

Gives H1 or H2 a (Holm-type) second opportunity to be rejected if not rejected initially

1

3

1

5

2

5

2

4

6 6

/4 /8 /8 /4


5

6

5

6

1

5

6

/8

2

5

6

/8

2

4

2 2

4

1 1

3 1

3

3

4 3

4

3 4


26


Summary, and further results

• Easy to formulate (cf. Closed-Testing Pocedures) • Easy to perform (even manually)

• Easy to calculate multiplicity-adjusted p-values (algorithm similar to that for rejections)

• Easy to obtain weights of Bonferroni test for any intersection hypothesis HI of corresponding CTP(useful for contruction of Conf regions)

• Easy to construct new MTPs (for new problems)

• Unifies many common MTPs

• Compact algebraic notation available(to describe and derive default graphs)


27


Corresponding Simultaneous Confidence Regions ???

• Holm MTP (1979)p(1) /3, p(2) /2, p(3) /1

• Fixed-Sequence MTP (old) p1 , p2 , p3

• Fallback MTP (2003/2005)weights w1, w2, w3 p1 w1, p2 (w2+), p3 (w3+)

Solution inJASA 1999

Open until2007

Three basic MTPs for Family {H1, H2, H3} based on raw p-values p1, p2, p3


28


Corresponding Simultaneous Confidence Regions !!!

Amazing:

Simultanouslypresented at MCP 2007in Vienna

28 years after Holm (1979)

ExtensionsandRelations

Confidence regions require weights that can be obtained directly from rejection algorithm of recycling approach


34


Outline

• Background

• Splitting, Recycling, Adding, (parts of alpha)

• Graphs and Default Graphs for MTPs

• Rejection Algorithm

• Holm’s MTP for groups of Hs, and a problem to discuss

• Improvements through extensions of the Default Graph

• Summary, and further results:

• (Simultaneous Confidence regions, CTP-formulation of MTPs)

alpha recycling in confirmatory clinical trials

Documents