issues of simultaneous tests for non-inferiority and superiority tie-hua ng*, ph. d. u.s. food and...
TRANSCRIPT
Issues of Simultaneous Tests for Non-Inferiority and Superiority
Tie-Hua Ng*, Ph. D.U.S. Food and Drug Administration
Presented atMCP 2002
August 5-7, 2002Bethesda, Maryland
_______
* The views expressed in this presentation are not necessarily of the U.S. Food and Drug Administration.
2
Simultaneous Tests for Non-Inferiority and Superiority
• Multiplicity adjustment is not necessary– Intersection-union principle (IU)
• Dunnett and Gent (1996)
– Closed testing procedure (CTP)• Morikawa and Yoshida (1995)
• Indisputable
3
A Big Question
Is Multiplicity Adjustment Necessary?
4
Is Is MultiplicityMultiplicityAdjustment Adjustment Necessary?Necessary?
5
Outline• Assumptions and Notations
• Switching between Superiority and Non-Inferiority
• Is Simultaneous Testing Acceptable?
• Use of Confidence Interval in Hypothesis Testing --- Pitfall
• Problems of Simultaneous Testing
• Conclusion
6
Assumptions/Notations• Normality and larger is better• T: Test/Experimental treatment (t)
• S: Standard therapy/Active control (s): Non-Inferiority Margin (> 0)• For a given d (real number), define
– Null: H0(d): T S - d – Alternative: H1(d): T > S - d
• Non-Inferiority: d = • Superiority: d = 0
7
Non-Inferiority (d = )
H0(): T S - against H1(): T > S -
H0() H1()
•°
TBoundaryWorse
Better
Mean Response
S
8
Superiority (d = 0)
H0 (0): T S against H1 (0): T > S
H0(0)H1(0)
•°
TBoundary
Worse
Better
Mean Response
S
9
Switching between Superiority and Non-Inferiority
CPMP (Committee for Proprietary Medicinal Products), European Agency for the Evaluation of Medicinal Products
Points to Consider on Switching Between Superiority and Non-Inferiority, 2000.
http://www.emea.eu.int/htms/human/ewp/ewpptc.htm
10
Switching between Superiority and Non-Inferiority (2)
• Non-Inferiority Trial– If H0() is rejected, proceed to test H0(0)
– No multiplicity issue, closed testing procedure
• Superiority Trial– Fail to reject H0(0), proceed to test H0()
– No multiplicity issue
– Post hoc specification of
11
Switching between Superiority and Non-Inferiority (3)
• Non-inferiority Trial– Intention-to-treat (ITT)– Per protocol (PP)
• Superiority Trial– Primary: Intention-to-treat (ITT)– Supportive: Per protocol (PP)
• Assume ITT = PP
12
Simultaneous Testing
One-sided 100(1 - )% lower Confidence Interval for T - S
Test is worse
Test is betterMean Difference (T – S)
0-
Superiority
Non-inferiority
Neither
13
Simultaneous Testing (2)
• Multiplicity adjustment is not necessary– Dunnett and Gent (1996)
• Intersection-Union (IU):
Superiority: Both H0() and H0(0) are rejected
– Morikawa and Yoshida (1995)• Closed Testing Procedure (CTP):
Test H0(0) when H0()H0(0) is rejected
14
Simultaneous Testing (3)
• Discussion Forum (October 1998)– London
– PSI (Statisticians in Pharmaceutical Industry)
• Is Simultaneous Testing of Equivalence [Non-Inferiority] and Superiority Acceptable?– Superiority trial:
• Fail to reject H0 (0)
• No equivalence/non-inferiority claim
– Ok: Morikawa and Yoshida (1995)
• Ref: Phillips et al (2000), DIJ
15
Is Simultaneous
TestingAcceptable?
16
Use of Confidence Interval in Hypothesis Testing
H0(d): T S - d (at significance level )
One-sided 100(1-)% lower CI for T-S
Reject H0(d) if and only if the CI excludes -d
Test is worse
Test is better
Mean Difference (T – S)
-d
Reject H0(d)
Do not reject H0(d)
17
Use of Confidence Interval in Hypothesis Testing (2)
• If CI = (L, ), then H0(d) will be rejected for all -d < L.
• A Tricky Question– Suppose CI = (-1.999, ), L = -1.999
• H0(2): T S - 2 is rejected (d=2) since -d < L• Can we conclude that T > S - 2?
• Yes, if H0(2) is prespecified.• No, otherwise.
18
Use of Confidence Interval in Hypothesis Testing (3)
Post hoc specification of
H0(d) is a
No No
19
Simultaneous Testing: Problems
H0(d1) and H0(d2), for d1 > d2
One-sided (1 - )100% lower CI for T - S
Test is worse
Test is betterMean Difference (T – S)
-d2-d1
Reject H0(d2)
Reject H0(d1)
Neither
20
Simultaneous Testing: Problems (2)
H0(d1), H0(d2) and H0(d3), for d1 > d2 > d3
One-sided (1 - )100% lower CI for T - S
Test is worse
Test is betterMean Difference (T – S)
-d3-d1
Reject H0(d3)Reject H0(d2)
None
-d2
Reject H0(d1)
21
Simultaneous Testing: Problems (3)
H0(d1), H0(d2),…, H0(dk), for d1 > d2 > … > dk
One-sided (1 - )100% lower CI for T - S
Test is worse
Test is betterMean Difference (T – S)
-dk-d1
Reject H0(dk)
Reject H0(d2)
None
-d2
Reject H0(d1)
…
…
.
.
.
-d3
22
Simultaneous Testing: Problems (4)
• Choose k large enough Pr[-d1 < Lower limit < -dk] close to 1
Max |dk - dk-1| < a given small number
Simultaneous testing of H0(di), i = 1,…, k
Post hoc specification of H0(d)
23
Simultaneous Testing: Problems (5)
Number of Nested hypotheses
Exploratory (many H0(d))
Confirmatory (one H0(d))
1 2 3 4 …………. k …………
Simultaneous H0() and H0(0)
24
Simultaneous Testing: Problems (6)
• What is wrong with IU and CTP?
• Nothing
• Pr[Rejecting at least one true null] • What kind of problems?
25
Simultaneous Testing: Problems (7)
• Post hoc specification of H0(d)
Let -d0 = 100(1 - )% lower limit -
Reject H0(d0), since -d0 < lower limit
Repeat the same trial independently
Pr[Rejecting H0(d0)] = 0.5 +
26
Simultaneous Testing: Problems (8)
• Simultaneous testing of many H0(d)
– Repeat the same trial independently
– Low probability of confirming the finding
• 1st trial: Reject H0(dj) but not H0(dj+1)
• 2nd trial: Pr[Rejecting H0(dj)] is low (e.g., 0.5+)
27
Simultaneous Testing: Problems (9)
• Simultaneous testing of H0() and H0(0)?
• Confirm the finding = 2 Known variance Let T - S Significance level = 0.025 80% power for H0() (at = 0)
28
Simultaneous Testing: Problems (10)
f() = Pr[Rejecting H0() | ]
f0() = Pr[Rejecting H0(0) | ]
29
Simultaneous Testing: Problems (11)
• Test one null hypothesis H0()
• Suppose that H0() is rejected
• Repeat the same trial independently
• Pr[Rejecting H0() again] = f()
30
Simultaneous Testing: Problems (12)
• Test H0() and H0(0) simultaneously
• Suppose that H0() or H0(0) is rejected
• Repeat the same trial independently• Pr[Rejecting the same null hypothesis again] =
[1 - w()] · f() + w() · f0()
= f() - f0() [1 – f0()/f()],
where w() = f0()/f()
31
Simultaneous Testing: Problems (13)
[1 - w()] · f() + w() · f0()
where w() = f0()/f()
Simultaneous tests in the 2nd trial
32
Simultaneous Testing: Problems (14)
• Ratio: 1 – [f0()/f()] [1 – f0()/f()]
• Ratio may be as low as 0.75
33
Conclusion
• Many H0(d): Problematic
• Not type I error rate
• H0() and H0(0): Acceptable?
• If “zero tolerance policy”: No
• If 25% reduction cannot be tolerated: No
• If 25% reduction can be tolerated: Yes
34
Is Simultaneous Testing of
H0() and H0(0) Acceptable?
35
You be the judge
36
References• Dunnett and Gent (1976), Statistics in Medicine, 15,
1729-1738.
• Committee for Proprietary Medicinal Products (CPMP; 2002). Points to Consider on Switching Between Superiority and Non-Inferiority. http://www.emea.eu.int/htms/human/ewp/ewpptc.htm
• Morikawa T, Yoshida M. (1995), Journal of Biopharmaceutical Statistics, 5:297-306.
• Phillips et al., (2000), Drug Information Journal, 34:337-348.
37