statistical issues in randomized controlled trial (rct) · • the interpretation of statistical...

1

Statistical Issues in Randomized Controlled Trial (RCT)

Jaranit KaewkungwalFaculty of Tropical Medicine

Mahidol University

Design: RCTs• Subjects are randomly assigned to either an experimental or control

group

– Control group: receives no intervention or receives the standard or conventional intervention

– May be > 1 experimental or control group

• Assess the effects of a preventive or therapeutic agent, treatment, procedure, or service

population

group 1

group 2

Outcome

Outcome

new treatment

control treatment

• “Gold standard” of causal studies

– Provide strongest possible evidence of causation that any study designs can deliver

• Due to controlled conditions & randomization of subjects & can be replicated by investigators using the same study protocol

2

Receive A Do not receive A

Study group A

Do not receive A Do not receive B

Control group B

Randomisation

Participants

Outcome known Outcome unknown

Follow for outcome

Non-participants

Informed Consent

Study or Experimental Population

Reference Population

Design: RCTs

Outcome

(Receive B)

Outcome

Types of Trial: Confirmatory Trial

• Trial is usually designed to provide firm evidence of efficacy or safety of investigative drug/treatment.

• Primary objective, is always predefined, and is the hypothesis that is subsequently tested when the trial is complete.

• Confirmatory trials are intended to provide firm evidence in support of claims and hence adherence to protocols and standard operating procedures is particularly important

• The interpretation of statistical measures of uncertainty of the treatment effect and treatment comparisons should involve consideration of the potential contribution of bias to the p-value, confidence interval, or inference.

Source: E9 Statistical Principles for Clinical Trials

3

Design: Therapeutic Trial

EligibleAge >=30

Participation No Participation

Random Assignment

Intensive Insulin

Diabetes Patients

Ineligible

Standard Insulin

Retinopathy (y/n) Retinopathy (y/n)

Design: Preventive Trial

EligibleHIV negative

Age >=18


Random Assignment

Placebo

High risk population (IDUs attending 17 BMA

Clinics)

Ineligible

HIV Vaccine

HIV positive (y/n) HIV positive (y/n)

4

Design: Field Trial

EligibleHIV negative

Age >=18


Random Assignment

Placebo

Healthy Population (Volunteers in 2 Provinces)

Ineligible

HIV Vaccine

HIV positive (y/n) HIV positive (y/n)

Salk Polio Vaccine Trial (Field Trial)Summer of 1916: epidemic struck New York City

-27,000 people paralyzed-9,000 people dead-Epidemic every summer

until vaccination initiated in 1955

-Children (grades 1-5) inoculated in U.S., Canada and Finland-Treated 440,000 with vaccine and 210,000 received placebo

-April 12, 1955: Announcement of successful field trials-April 1955: Five million children vaccinated with Salk vaccine

5

Types of Trial: Exploratory Trial• The rationale and design of confirrmatory trials nearly

always rests on earlier clinical work carried out in a series of exploratory studies.

• Like all clinical trials, exploratory studies should have clear and precise objectives. However, in contrast to confirmatory trials, their objectives may not always lead to simple tests of predefined hypotheses.

• In addition, exploratory trials may sometimes require a more flexible approach to design so that changes can be made in response to accumulating results.

• Their analysis may entail data exploration; test of hypothesis may be carried out, but the choice of hypothesis may be data dependent. Such trials cannot be the basis of the formal proof of efficacy, although they may contribute to the total body of relevantevidence.


Other Types of Trial

• Factorial designineligible

exposed to treatment A exposed to treatment B exposed to treatments A & B unexposed to treatments A & B

Random Assignment

participation no participation

eligible

Source Population

ineligible

exposed to treatment B

exposed to treatment A

exposed to treatment A

exposed to treatment B

Random Assignment

participation no participation

eligible

Source Population

• Cross-over design

6

Strengths and Limitation of RCTs

• Strengths– Ability to demonstrate causal associations when

appropriately designed• Use of controlled conditions• Powerful techniques – randomization & blinding

– Increase subject comparability – Decrease potential bias and confounding

– Investigators control level of exposure• Permits them to establish precise doses or procedures

most appropriate to experiment

Strengths and Limitation of RCTs• Limitation

– Limited applicability• Ethical concerns limit use • This is why RCTs focus on interventions that have the potential to reduce

rather than increase the risks of morbidity or mortality

– Bias that can result from differential rates of compliance, withdrawal, or losses to follow-up between experimental and control groups

• Can alter the magnitude or direction of the relationship between the intervention and study outcome

– Long-term commitment– Considerable expenses necessary to implement and complete a

trial– Some address narrowly focused questions in artificial settings

• Applicability to real-world situations may not always be clear

– Reliance on volunteers and use of strict exclusion criteria • Make it difficult to recruit sufficient subjects for a study• Can limit external validity of findings

7

Drug Development Studies(Phase I–IV)

Drug Development Process

8



9

Phase I

• This is the first trial of a new active ingredient or new formulation in humans, often carried out in healthy volunteers.

• Its purpose is to establish a preliminary evaluation of the safety, and the pharmaco-kinetic and, where possible, pharmaco-dynamic profile of the active ingredient in humans

Phase II

• These trials are performed in a limited number of subjects and are often, at a later stage, of a comparative (e.g. placebo-controlled) design. Their purpose is to demonstrate therapeutic activity and to assess the short-term safety of the active ingredient in patients suffering from a disease or condition for which the active ingredient is intended.

• This phase also aims at the determination of appropriate dose ranges or regimens and (if possible) clarification of dose-response relationships in order to provide an optimal background for the design of extensive therapeutic trials.

10

Phase III (Efficacy Trial)

• Trials in larger (and possibly varied) patient groups, with the purpose of determining the short-and long-term safety/efficacy balance of formulation(s) of the active ingredient, and of assessing its overall and relative therapeutic value.

• The pattern and profile of any frequent adverse reactions must be investigated and special features of the product must be explored (e.g. clinically relevant drug interactions, factors leading to differences in effect such as age).

Phase IV• Studies performed after marketing of the pharmaceutical

product. Trials in phase IV are carried out on the basis of the product characteristics for which the marketing authorisation was granted and are normally in the form of post-marketing surveillance, or assessment of therapeutic value or treatment strategies.

• Although methods may differ, these studies should use the same scientific and ethical standards as applied in pre-marketing studies.

• After a product has been placed on the market, clinical trials designed to explore new indications, new methods of administration, new combinations, etc. are normally considered as trials on new pharmaceutical products.

11

Efficacy Trial - RV144 (2009)

ประสิทธิผล0.279 – 0.192

0.279

Efficacy Trial - RV144 (2009)

12

Effectiveness trial - Microbicides (2010)

ประสิทธิผล9.1 – 5.6

9.1

38 vs. 60

38 vs. 60

Effectiveness Trial - Microbicides (2010)

13

Effectiveness Trial - PrEP (2010)

• Pre ก่อน• Exposure สัมผ้ส• Prophylaxic การกนิยา ป้องกนัการตดิเช้ือ

ยาตา้นไวรัส (antiretroviral medication) 2 ตวั –tenofovir disoproxil fumarate (TDF) และ emtricitabine (FTC)

ประสิทธิผล47% (22-64%)

Effectiveness Trail - PrEP (2010)

14

Statistical Issues in Designing & Conducting RCT

• Good Clinical Practice (GCP) • Selecting the samples• Determining sample size requirement• Allocating Subjects

into Groups (Random allocation)

• Blinding/Masking• 3 C’s

Good Clinical Practice

• Good Clinical Practice (GCP)

A standard for clinical studies which encompasses the design, conduct, monitoring, termination, audit, analyses, reporting and documentation of the studies and which ensures that the studies are scientifically and ethically sound and that the clinical properties of the pharmaceutical product (diagnostic, therapeutic or prophylactic) under investigation are properly documented.

WHO: Guidelines for Good Clinical Practice (GCP) for Trials on Pharmaceutical Products.WHO Technical Report Series, No. 850, 1995. Geneva: WHO

15

Regulatory & Guidelines

• International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH)

• The European Agency for the Evaluation of Medicinal Products(EMEA)

• FDA Center for Drug Evaluation and Research (CDER)

• Human Subject Protections- Office of Human Subjects Research, NIH(OHSR)

• WORLD MEDICAL ASSOCIATION

• Standard operating procedures for clinical investigators ( WHO GCP SOP)

GCP & Computer / Database Management Systems


16

GCP – E6


GCP – E2A (Data Management

& Reporting)


17

GCP – E2B(R2) (Data Elements &

Transmission)


GCP – E2B(R3) (Data Elements &

Transmission)


18

GCP – E9(Data Analysis)


Selecting Sample• Selecting the Sample

– Define reference population• General population to which investigators hope to generalize the findings• Depends on problem or outcome being investigated• Usually impossible to identify and enumerate all members

– Identify experimental population• Practical representation of reference population• Consists of discrete sub-sample of reference population that can be

identified and counted• Usually focuses those in a small geographic area during a finite time period• Most important: its ability to produce valid results for hypothesis being

tested - Must be large enough & likely to produce enough end points (study outcomes) to permit valid statistical comparisons between experimental and control groups

– Choose study sample from experimental population• Include fully informed, willing participants who meet the predetermined

eligibility criteria • Considering issues: the study objectives, possible effects on internal and

external validity, potential harm or benefit to subjects, and issues related to convenience and efficacy (optimize conditions for successful testing of effects of an intervention)

19

Population

• In the earlier phases of drug development, the choice of subjects for a clinical trial may be heavily influenced by the wish to maximize the chance of observing specific clinical effects of interest, and hence they may come from a very narrow subgroup of the total patient population for which the drug may eventually be indicated.

• However, by the time the confirmatory trialsare undertaken, the subjects in the trials should more closely mirror the target population.


Levels in Subject

Selection &

Generalization

Level of selection Generalization

Interpretation of results

Reference populationTarget population

the population to whichthe results can be applied

Source/Sample population the population defined ingeneral terms and enumerated ifpossible, from which eligiblesubjects are drawn

Eligible subjects the population of subjectseligible for inclusion in thestudy; should be definedprecisely and counted

Study participants the individuals who contributedata to the study; the resultsapply directly only to thesesubjects

Thai People; Subtype B/E

High-Risk PeopleCSW IDUCSW IDU

IDU @ 17 BMA Clinics

4700 IDU s - Methadone maintenance

- HIV negative

- Age 18 - 50

- Willing to participate

- etc.2500 IDUs

20

Sample Size

• The number of subjects in a clinical trial should always be large enough to provide a reliable answer to the questions addressed.

• This number is usually determined by the primary objective of the trial.

• If the sample size is determined on some other basis, then this should be made clear and justified. For example, a trial sized on the basis of safety questions or requirements or important secondary objectives may need larger numbers of subjects than a trial sized on the basis of the primary e$cacy question (see ICH E 1 a).


Sample Size

• Sample size calculations should refer to the number of subjects required for the primary analysis.

• If this is the “full analysis set”, estimates of the effect size may need to be reduced compared to the “per protocol set”


21

Determinants of sample size (Hypothesis Testing)

Four factors determine the required sample size:1. Standard deviation, (continuous), or the expected success rate for the control group, p1 (categorical).

2. The difference between groups that we wish to detect, . Note: Effect size: the size of the smallest effect that is clinically

important.– E.g., RR of 1.5 for risk of CHD in patients with

hypertension (50% increased risk)» Ho: RR=1.0; H1: RR>=1.5

3. The false positive error rate, or significance level of the test, (usually 0.05).

4. The false negative error rate, (usually 0.1 or 0.2), more commonly expressed as (1-), the power (0.8 or 0.9). This is for a specified (alternative hypothesis).

Difference between two proportionsHo:

221

22112

11

)(

)]1()1([}{2

pp

ppppzzn

Formula: Categorical outcome Where

p1 = expected/known proportion in the control group

p2 = expected proportion in the intervention group (= p1 + )

Type I err Priori Info.

Effect Size

Power(1-Type II err)

Difference between means

2

2211

}{22

zz

n

Formula: Continuous outcomeHo: 1 = 2

Type I err Priori Info.

Effect Size

Power(1-Type II err)

Where

σ = expected/known standard deviation in the control group

= expected different standard deviation in the intervention group

22

In a randomised clinical trial, the placebo response is anticipated to be 25%, and the active treatment response 65%.

How many patients are needed if a two-sided test at the 5% level is planned, and a power of 80% is required?

2)65.025.0(

))65.01(65.0)25.01(25.0(849.7

n

so n=21 per group.

Example: RCT- Placebo vs. Treatment

221

2211

)(

)]1()1([849.7

pp

ppppn

4.2016.0

415.0849.7

Placebo Active Trtmt

R NR R NR

25% 75% 65% 35%

Example of Intervention Study

23

Selection of control groups

Selection of control groups

24

Example of Sample Size Adjustment·

Randomised, double-blind, placebo-controlled study to determine whether steroids reduce the incidence and severity of nephropathy in Henoch Schonlein Purpura (HSP)

Main research questions

Do steroids reduce the incidence and severity of nephropathy in childhood HSP?

Are ACE genotype polymorphisms predictive of progressive nephropathy in HSP?

Henoch-Schonlein Purpura (HSP) is the commonest small vessel vasculitis of childhood. Long term prognosis is related to progressive renal insufficiency. There is no conclusive evidence that steroids will alter the course of the disease. We will address this. In conjunction, we will evaluate the association between insertion and deletion polymorphisms of the ACE gene and progressive nephropathy in treated and untreated groups.

Data analysis/Sample size

Formal statistical input into the study has been provided by the Research and Development Support Unit at Southmead hospital, Bristol. To test the hypothesis that treatment with prednisolone 2mg/Kg for a period of 14 days reduces the incidence of proteinuria at a set point (12 months) after initial presentation. We will require a study of 320 patients (160 in each group). This calculation is based on the premise that 15% of children in the untreated group are likely to develop proteinuria during the 12 month period, compared with 5% in the treated group. This sample size will provide 80% power for testing the hypothesis at the 5% level of statistical significance, and assumes the difference will be analysed using a continuity corrected chi-squared test. Allowing a dropout rate of 15%, 184 patients will need to be randomised to each treatment arm (prednisolone or placebo).

Sample Size Monitoring

BACK

Figure 3. Flow diagram of a trial of chiropractic manipulation of the cervical spine for treatment of episodic tension-type headache. The diagram includes the number of patients actively followed up at different times during the trial. Adapted from Bove and Nilsson

25

Randomization• Randomization introduces a deliberate element of

chance into the assignment of treatments to subjects in a clinical trial.

• During subsequent analysis of the trial data, it provides a sound statistical basis for the quantitative evaluation of the evidence relating to treatment effects.

• It also tends to produce treatment groups in which the distributions of prognostic factors, known and unknown, are similar.

• In combination with blinding, randomization helps to avoid possible bias in the selection and allocation of subjects arising from the predictability of treatment assignments.


Block Randomization

• Although unrestricted randomization is an acceptable approach, some advantages can generally be gained by randomizing subjects in blocks.

• This help to increase the comparability of the treatment groups, particularly when subject characteristics may change over time, as a result, for example, of changes in recruitment policy.

• It also provides a better guarantee that the treatment groups will be of nearly equal size.


26

Example: Block Randomization

Stratified Randomization• In multi-centre trials the randomization procedures

should be organized centrally. It is advisable to have a separate random scheme for each centre, that is, to stratify by centre or to allocate several whole blocks to each centre.

• More generally, stratification by important prognostic factors measured at baseline (for example, severity of disease, age, sex, etc.) may sometimes be valuable in order to promote balanced allocation within strata; this has greater potential bene"t in small trials.

• The use of more than two or three strati"cation factors is rarely necessary, is less successful at achieving balance and is logistically troublesome.


27

Randomization Process• Details of the randomization that facilitate

predictability (for example, block length) should notbe contained in the trial protocol.

• The randomization schedule itself should be filed securely by the sponsor or an independent party in a manner that ensures that blindness is properly maintained throughout the trial.

• Access to the randomization schedule during the trial should take into account the possibility that, in an emergency, the blind may have to be broken for any subject.

• The procedure to be followed, the necessary documentation, and the subsequent treatment and assessment of the subject should all be described in the protocol.


Blinding or Masking• Applying the Intervention and Assessing the Outcomes During

Follow-up– Assessment of outcomes during follow-up period may be subject to

measurement bias (e.g., outcome reporting & compliance) especially when subjects are aware of which group they have been assigned to.

– Ways to minimize these problems:• Single-blinded study

– Blinding (masking) subjects to their own group assignment– Minimizes bias introduced by subjects– No effect on bias on part of investigators

• Double-blinded study– Neither the subjects nor the investigators are aware of subjects’ group

assignments– Overcome both sources of bias– Use a sealed code for assignments that is only broken at conclusion of

study

Single blind Double blind Open-label

28

Blinding or Masking

– Useful in reducing confounding that occurs during follow-up period• Investigators pay more attention to experimental

group subjects – suggest other ways that they can improve their condition– Increases probability that the intervention

appears successful due to unrecognized confounding by these other measures

• Control group subjects seek other treatments – These “cointerventions” can confound the

study results during the follow-up period

Blinding or Masking

– Not always feasible• Sometimes, it is obvious which group is which

– E.g., Back surgery vs. physical therapy• Sometimes, it is difficult to produce a suitable

placebo– E.g., Side effects of intervention

– Most important when the outcome being assessed is subjective

– E.g., Pain, quality of life– Less important when outcome is truly objective

– E.g., Overall mortality, infection• Because bias is unlikely to have an effect on

reporting of outcome

29

3 C’s

• Compliance (or Adherence)– The extent to which the patient's behaviour (taking

medications, diets, or other life-style) changes coincides with clinical prescription.

– Although non-compliance suggests a kind of wilful neglect of good advice, in medicine other factors also contribute

– Patients may misunderstand which drugs and does are intended

– Run out of prescription medications– Confuse various preparations of the same drug– Have no money or insurance to pay for drugs.

– Compliance is particularly important in medical care outside hospital.

3 C’s• Co-intervention

– The application of additional diagnostic or therapeutic procedures to members of either or both the experimental and the control groups.• After randomisation, patients may receive a variety

of interventions other than the ones being studied.

• Contamination– The inadvertent application of the experimental

procedure to members of the control group, or inadvertent failure to apply the procedure to members of the experimental group.• E.g., exchange of study treatment regimens among

study participants

30

Statistical Issues in Analyzing RCT

• The statistical analysis plan• Analysing data in RCTs• Types of Variables• Missing & Outliers• Data Transformation

The statistical analysis plan• The statistical analysis plan may be written as a

separate document to be completed after finalizing the protocol.

• In this document, a more technical and detailed elaboration of the principal features stated in the protocol may be included

• The plan may include detailed procedures for executing the statistical analysis of the primary and secondary variables and other data.

• The plan should be reviewed and possibly updated as a result of the blind review of the data and should be finalized before breaking the blind.

• Formal records should be kept of when the statistical analysis plan was "nalized as well as when the blind was subsequently broken.


31

Data Analysis Sets in RCT

Prespecification of the Analysis

• When designing a clinical trial the principal features of the eventual statistical analysis of the data should be described in the statistical section of the protocol.

Analysis Sets

• The set of subjects whose data are to be included in the main analyses should be defined in the statistical section of the protocol.

• In addition, documentation for all subjects for whom trial procedures (for example, run-in period) were initiated may be useful.


Types of Data Analysis

• Analyzing data in RCTs

– According to the treatment to which the patients were randomised or to the one they actually received.

– The correct presentation of results depends on the question being asked.

• Intent to treat• Per-protocol• As treated

32

Full Analysis Set• The intention-to-treat principles implies that the

primary analysis should include all randomized subjects.

• Compliance with this principle would necessitate complete follow-up of all randomized subjects for study outcomes.

• When the full analysis set of subjects is used, violations of the protocol that occur after randomization may have an impact on the data and conclusions, particularly if their occurrence is related to treatment assignment.

• In the full analysis set, it is justified to include those with failure to satisfy major entry criteria (eligibility violations), for example - the failure to take at least one dose of trial medication and the lack of any data postrandomization.


Full Analysis Set• Subjects who fail to satisfy an entry criterion may be

excluded from the analysis without the possibility of introducing bias only under the following circumstances:– (i) the entry criterion was measured prior to randomization;– (ii) the detection of the relevant eligibility violations can be

made completely objectively;

– (iii) all subjects receive equal scrutiny for eligibility violations (this may be di$cult to ensure in an open-label study, or even in a double-blind study if the data are unblinded prior to this scrutiny, emphasizing the importance of the blind review);

– (iv) all detected violations of the particular entry criterion are excluded.


33

Per Protocol Set• The “per protocol” set of subjects, sometimes described

as the “valid cases.”, the “efficacy” sample or the “evaluable subjects” sample, defines a subset of the subjects in the full analysis set who are more compliant with the protocol and is characterized by criteria such as the following:– (i) the completion of a certain prespecified minimal exposure

to the treatment regimen;– (ii) the availability of measurements of the primary

variable(s);– (iii) the absence of any major protocol violations including the

violation of entry criteria.

• The precise reasons for excluding subjects from the per protocol set should be fully defined and documented before breaking the blind in a manner appropriate to the circumstances of the specific trial.


Data Analysis: Intention-to-treat, Per-protocol & As-treated

Source: Sally Hollis, Fiona Campbell, What is meant by intention to treat nalysis?Survey of published randomised controlled trials, BMJ VOLUME 319 11 SEPTEMBER 1999 www.bmj.com

34

Example: RV144 (2009)

ITT Population

Modified ITT Population

Data Analysis:Eligible vs. Ineligible Participants

Source: D. L. DEMETS, Statistical issues in interpreting clinical trials, Journal of Internal Medicine 2004; 255: 529–537

35

Data Analysis:Compliant vs. Not Compliant Participants

Source: D. L. DEMETS, Statistical issues in interpreting clinical trials, Journal of Internal Medicine 2004; 255: 529–537

Primary and Secondary Variables

• The primary variable (target' variable, primary endpoint) should be the variable capable of providing the most clinically relevant and convincing evidence directly related to the primary objective of the trial.

• There should generally be only one primary variable. This will usually be an efficacy variable, because the primary objective of most confirmatory trials is to provide strong scientiดรc evidence regarding efficacy.

• Safety/tolerability may sometimes be the primary variable.

• Measurements relating to quality of life andhealth economics are further potential primary variables.


36


• The primary variable should be specified in the protocol, along with the rationale for its selection. Redefinition of the primary variable after unblinding will almost always be unacceptable, since the biases this introduces are di$cult to assess.

• When the clinical effect defined by the primary objective is to be measured in more than one way, the protocol should identify one of the measurements as the primary variable on the basis of clinical relevance, importance, objectivity, and/or other relevant characteristics, whenever such selection is feasible.



• Secondary variables are either supportive measurements related to the primary objective or measurements of e!ects related to the secondary objectives.

• Their predefinition in the protocol is also important, as well as an explanation of their relative importance and roles in interpretation of trial results.

• The number of secondary variables should be limited and should be related to the limited number of questions to be answered in the trial.


37

Composite Variables• If a single primary variable cannot be selected from

multiple measurements associated with the primary objective, another useful strategy is to integrate or combine the multiple measurementsinto a single or composite' variable, using a predefined algorithm.

• The primary variable sometimes arises as a combination of multiple clinical measurements (for example, the rating scales used in arthritis, psychiatric disorders.)

• When a composite variable is used as a primary variable, the components of this variable may sometimes be analysed separately, where clinically meaningful and validated.

• When a rating scale is used as a primary variable, it is especially important to address such factors as content validity, interrater and intrarater reliability.

Global Assessment Variables

• In some cases, “global assessment” variables are developed to measure the overall safety, overall efficacy, and/or overall usefulness of a treatment.

• This type of variable integrates objective variables and the investigator's overall impression about the state or change in the state of the subject, and is usually a scale of ordered categorical ratings.

• Global assessments of overall efficacy are well established in some therapeutic areas, such as neurology and psychiatry.


38

Multiple Primary Variables• It may sometimes be desirable to use more than one

primary variable, each of which (or a subset of which) could be su$cient to cover the range of effects of the therapies.

• The planned manner of interpretation of this type of evidence should be carefully spelled out.

• It should be clear whether an impact on any of the variables, some minimum number of them, or all of them, would be considered necessary to achieve the trial objectives.

• If the purpose of the trial is to demonstrate effects on all of the designated primary variables, then there is no need for adjustment of the type I error, but the impact on type II error and sample size should be carefully considered.


Surrogate Variables• When direct assessment of the clinical benefit to the

subject through observing actual clinical efficacy is not practical, indirect criteria (surrogate variables) may be considered.

• Commonly accepted surrogate variables are used in a number of indications where they are believed to be reliable predictors of clinical benefit.

• There are two principal concerns with the introduction of any proposed surrogate variable. – It may not be a true predictor of the clinical outcome of

interest.

– The proposed surrogate variables may not yield a quantitative measure of clinical benefit that can be weighed directly against adverse e!ects.


39

Surrogate Variables

• In practice, the strength of the evidence for surrogacy depends upon

– (i) the biological plausibility of the relationship,

– (ii) the demonstration in epidemiological studies of the prognostic value of the surrogate for the clinical outcome and

– (iii) evidence from clinical trials that treatment e!ects on the surrogate correspond to e!ects on the clinical outcome.


Missing Values and Outliers• Missing values represent a potential source of bias in a

clinical trial. Hence, every effort should be undertaken to fullfil all the requirements of the protocol concerning the collection and management of data.

• A trial may be regarded as valid, none the less, provided the methods of dealing with missing values are sensible, and particularly if those methods are predefined in the protocol.

• Unfortunately, no universally applicable methods of handling missing values can be recommended.

• An investigation should be made concerning the sensitivity of the results of analysis to the method of handling missing values, especially if the number of missing values is substantial.


40

Missing Values and Outliers

• A similar approach should be adopted to exploring the influence of outliers, the statistical definition of which is, to some extent, arbitrary.

• If no procedure for dealing with outliers was foreseen in the trial protocol, one analysis with the actual values and at least one other analysis eliminating or reducing the outlier e!ect should be performed and di!erences between their results discussed.


Source: Sally Hollis, Fiona Campbell, What is meant by intention to treat analysis?Survey of published randomised controlled trials, BMJ VOLUME 319 11 SEPTEMBER 1999 www.bmj.com

41

Data Transformation• The decision to transform key variables prior to analysis is

best made during the design of the trial on the basis of similar data from earlier clinical trials.

• Transformations (for example, square root, logarithm) should be specified in the protocol and a rationale provided, especially for the primary variable(s).

• The general principles guiding the use of transformations to ensure that the assumptions underlying the statistical methods are met are to be found in standard texts; conventions for particular variables have been developed in a number of specific clinical areas.

• The decision on whether and how to transform a variable should be influenced by the preference for a scale which facilitates clinical interpretation.


Data Preparation:Data Transformation

42

Data Preparation: Data Transformation

0

5

10

15

20

0 1000 2000 3000 4000

Value Count

1 310 6100 9200 11400 18800 51600 23200 1

Value Log Count

1 0 310 6100 9200 11400 18800 51600 23200 1

Value Log Count

1 0 310 1 6100 9200 11400 18800 51600 23200 1

Value Log Count

1 0 310 1 6100 2 9200 11400 18800 51600 23200 1

Value Log Count

1 0 310 1 6100 2 9200 2.3 11400 2.6 18800 2.9 51600 3.2 23200 3.51 1

0

5

10

15

20

0 1 2 3 4

Examples of the Logarithmic transformation:

Subgroup Analysis

Source: John Fletcher, Subgroup analyses: how to avoid being misled, BMJ | 14 july 2007 | Volume 335

43

• Preliminary Results of the Phase III Efficacy Trial of AIDSVAX B/B February 23, 2003

All Volunteers 98/1679 (5.8%) 191/3330 (5.7%) 3.8% (-22.9 to 24.7%)

White & Hispanic 81/1508 (5.4%) 179/3003 (6.0%) -9.7% (-42.8 to 15.7%)

Black/Asian/Other 17/171 (9.9%) 12/327 (3.7%) 66.8% (30.2 to 84.2%)*

Black 9/111 (8.1%) 4/203 (2.0%) 78.3% (29.0 to 93.3%)**

Asian 2/20 (10.0%) 2/53 (3.8%) 68.0% (-129.4 to 95.5%)

Other 6/40 (15.0%) 6/71 (8.5%) 46.2% (-67.8 to 82.8%)

Vaccine Efficacy

Weighted cohort Placebo Inf./total Vaccine Inf./total VE (95.12%CI)

** p <0.02* p <0.01

Subgroup Analysis

Effects of gp120 HIV Vaccine among US Volunteers

Statistical Analysis Practices in Clinical Trial

Adapted from lecture on

“Data and Safety Monitoring Boards (DSMB) for Clinical Trials”Shrikant I Bangdiwala, Dept of Biostatistics, Univ of North Carolina at Chapel Hill

Faculty of Tropical Medicine, 25 July 2006

“Statistical Considerations in the Determinations of Futility and Efficacy in HIV Vaccine Trials”

Michael D. Lock, VaxGen“Role of the Data and Safety Monitoring Board (DSMB)”

Donald Stablein. The EMMES CorporationWHO Meeting, Pattaya, Chonburi, 12 June 2007

Jaranit Kaewkungwal

Data Management UnitFaculty of Tropical Medicine

Mahidol University

44

Typical structure of a single site trial

SponsorData and SafetyMonitoring BoardInstitutional

Review Board

physiciansnursesassistants

Clinicalsupport

biostatisticiansprogrammersassistants

Biostatisticalsupport

coordinatorassistants

Technicalsupport

manageraccountantsecretaryasssitants

Administrativesupport

directorprocessors

Laboratorysupport

Co-investigators

Principal Investigator

Structure of clinical trial

Typical structure of a multicenter trial

SponsorData and SafetyMonitoring Board

Publicationssubcommittee

directorbiostatisticiansprogrammersassistants

Statisticalcoordinating

center

ABC.......K

Sites

directorprocessors

Centrallaboratories

Executivesubcommittee

Governancesubcommittee

Eventadjudication

subcommittee

SteeringCommittee

Institutional review boards

Structure of clinical trial

45

Ethical requirements – Protocol approval

Ethical considerations dictate the need for approving a clinical trial protocol prior to its startProtection of rights of the subjects

“Equipoise” of interventions – outcomes and adverse effectsConsent processAdequate care during and after trial

Design issuesAppropriate design for intended outcome – statistical powerFeasibility

Monitoring study resultsInternally vs externally trial monitoring of study results

Internal: monitoring by investigators: – Investigators: progress, quality assurance (QA)

External: monitoring by various organizations:– Regulatory agencies: compliance with rules

(e.g. the FDA in the USA), GCP, serious adverse events

– Funding agency: progress – DSMB: progress, validity, QA, patient advocacy,

adverse events, efficiency analyses

46

DSMB Defined

A committee charged with reviewing accumulating information on safety/efficacy and making recommendations on trial conduct and continuation. Often the only group with access to unblinded study results.

3 Basic Models

DSMB Members

Independent

Independent

Not Independent

Analysis Statistician

Independent

Trial Statistician

Trial Statistician

47

Ethical requirements - Monitoring

Ethical considerations dictate the need for monitoring and considering the possible early termination of a clinical trialConsiderations based on the information (analytic)

unexpected or unanticipated toxicity or side-effectsunequivocal evidence of treatment difference in efficacyzero tendency and unlikely possibility of a result (futility)

Considerations based on the conduct (administrative)inadequate recruitmentpoor compliance, adherence or follow-uppoor quality dataloss of financing

External considerations (subjective)results from other trialsinformation from clinical practicechanges in current treatment

Data monitoring & analysis support

The monitoring aspects dictates that they be performed by an objective individual or institution with no conflicts of interest;

thus the analyses are done by the biostatistical support:

IndividualStatistical Coordinating Center (SCC) and/or Data Coordinating Center (DCC)

48

DSMB: Meetings

SCC prepares report and distributes prior to meeting

Open section: members of DSMB, study investigators, biostatisticianClosed section: members of DSMB, biostatistician

Content of reportOPEN – overall, and BY site

Study progress: recruitment and eligibility, follow-up, compliance and adherenceQuality assurance: staff training & certification, equipment calibration & standardization, reliability & validity of instrumentsData management: missing & incomplete forms, timeliness, entry errors, missing data, out of range and inconsistent dataAnalysis: comparability of groups at baseline

49

Content of reportCLOSED – overall and BY intervention group

Study progress: follow-up, compliance & adherence, event adjudication processPatient care: adverse events at individual & group levelsData management: missing & incomplete forms, missing dataAnalyses: follow-up descriptive statistics; outcome variables – primary & secondary endpoints; interim analyses

Timing of DSMB meetings

X X X X X …… X

Start of study End of studyEstablish quality control Definitive analyses

Recruitment of patients Intervention and Follow-up

50

Analyses for the DSMB

Individual level informationGroup and site level information, looking for trends in adverse events

Patients trust that the study will not be continued if equipoise is no longer believed or if study will be inconclusive

Timing of DSMB meetings

X X X X X …… X

Start of study End of studyEstablish quality control Definitive analyses

Recruitment of patients Intervention and Follow-up

51

t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

±1.96•Z1

Hypothetical example

±1.96

Level of significance

Z0.05 =

Probability of Type I error

t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

±1.96•Z1

•Z2


52

t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

±1.96•Z1

•Z2

•Z3


t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

•Z1

•Z2

•Z3

•Z4

±1.96


53

Statistical implication

Ethical considerations dictate that the DSMB review periodic tests of the primary study hypotheses during the conduct of the study and not only at the programmed conclusion

Increases the probability of a Type I error ()

How do we adjust the analyses?

Implications of multiple looks at data

• Increase in the probability of falsely rejecting the null hypothesis

Overall significance level when H0 is true inrepeated testing of accumulated data

N um be r o f repea ted tests (K ) N om ina l le ve l o f s ign ifican ce

1 2 3 4 5 10 25 50 200

1 1 1 .8 2 .4 2 .9 3 .3 4 .7 7 .0 8 .8 12 .6

5 5 8 .3 10 .7 12 .6 14 .2 19 .3 26 .6 32 .0 42 .4

10 10 16 .0 20 .2 23 .4 26 .0 34 .2 44 .9 52 .4 65 .2

54

Statistical methodology

Sequential testing approachesSequential probability ratio test (Wald 1947)Repeated significance test (Armitage et al 1969)

Group sequential approach Conditional power approachesConfidence intervals (Jennison & Turnbull)Modifying critical values

• Ad-hoc methods (Haybittle 1971, Peto 1972)• Pocock (1977) – based on repeated significance testing• O’Brien & Fleming (1979)

Alpha-spending function approach (Lan & DeMets 1983)

Example: cutpoints for K=10, =0.05, 1-β=0.80

k PocockO'Brien-

FlemingHaybittle-

PetoWang-Tsiatis

=0.25

1 ±2.555 ± 109.56 ± 3 ± 15.93

2 ± 2.555 ± 34.65 ± 3 ± 8.96

3 ± 2.555 ± 15.49 ± 3 ± 5.99

4 ± 2.555 ± 8.49 ± 3 ± 4.43

5 ± 2.555 ± 5.37 ± 3 ± 3.53

6 ± 2.555 ± 3.80 ± 3 ± 2.97

7 ± 2.555 ± 2.94 ± 3 ± 2.61

8 ± 2.555 ± 2.46 ± 3 ± 2.39

9 ± 2.555 ± 2.20 ± 3 ± 2.26

10 ± 2.555 ± 2.087 ± 2.021 ± 2.199

Group sequential approach

55

t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

•Z1

•Z2

•Z3

•Z4

±1.96Pocock


t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

•Z1

•Z2

•Z3

•Z4

±1.96Pocock

•Z5


56

t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

•Z1

•Z2

•Z3

•Z4

±1.96

Pocock

Haybittle-Peto

•Z5


t1 t2 t3 t4 t5

5

4

3

2

1

0

-1

-2

-3

-4

-5

•Z1

•Z2

•Z3

•Z4±1.96

Pocock

Haybittle-Peto

O-Brien-Fleming

•Z5


57

Example

Beta-blocker Heart Attack Trial (BHAT)

stopped early

Six interim logrank s tatis tics and O'Brien-flem ing cutpoints for BHAT Study

1.68

2.242.37 2.3 2.34

2.82

0

1

2

3

4

5

6

Jun-78 May-79 Oct -79 Mar -80 Oct -80 Apr -81 Oct -81 Jun-82

D a t e s o f D S M C M e e t i n g s

Interim analysis

58

Interim analysis

Interim analysis

59

Multiple Analyses Require Adjustments

Several methods are available to limit the chance of reaching erroneous conclusion

Rules like this one make it hard to stop trial at early analyses

Futility Analyses using Conditional Power

Conditional power (CP) is a one approach to assessing futility

CP at time t is the probability that trial will have a (statistically significant) positive outcome given the amount of information collected up to t

Stop trial if CP is low, e.g. < 0.1

60

Factors Affecting Conditional Power

Conditional power is partially determined by assumptionsAssumed future trajectory of VE is importantExample on left assumes no change - may be too pessimisticExample on right relies on assumptions made prior to start of trial

Factors Affecting Conditional Power

Conditional power depends on when analysis occursMore certainty in analysis when there are more dataExample: Conditional power lower with 2/3 of information compared to 1/2 since it is more certain that trial will not meet its objectives

61

The End Randomized Controlled Trial

statistical issues in randomized controlled trial (rct) · • the interpretation of statistical...

Documents