University of Dundee School of Medicine

Protocol Development and Protocol Development and Statistical Analysis PlansStatistical Analysis PlansPetra RauchhausTCTU Clinical Trials Statistician

Importance of the Protocol

• Provide rationale for the trial• Define trial goals and processes• Define methods of analysis/ reporting• Enable scientific and ethical review• Provide a “Trial Roadmap”

Funders

Policy makers

Journals

Systematic Reviewers

Ethics

EthicsHealthcare providers

Trialists Patients

Journals

Importance of the Protocol• GCP Requirement

• Ethics Committee requires a protocol for submission

• Part of the EU Clinical Trials Register (EUDRACT)

• Ensures in Multi-Centre Trials that all centres perform the study in the same way

• Journals require a registered protocol for publication

• Not only for CTIMPS, Non-CTIMPS also benefit from a good protocol

What could go wrong?• Missing details of basic trial design (uncontrolled/

controlled/ randomized)

• Imprecise or missing description of the primary outcome in the protocol

• Sample Size calculation not reported

• Limited methodological information

• Interventions not well defined

• Planned subgroup analyses missing

• Favourable reporting of positive outcomes

• Adverse events suppressed in reports

Lack of general information59

34

25

40 41

0

10

20

30

40

50

60

70% Inadequate information

Allocation Concealment

Blinding Primary Outcome

Power Calc.

Adverse Events Reporting SystemChan AW et al, BMJ 2008; Al-Marzouki S et al, Lancet 2008

Lack of statistical information

Primary Outcome Analysis

Handling Missing Data

Handling deviations

Adjusted Analyses

Subgroup Analyses

Chan AW et al, BMJ 2008; Al-Marzouki S et al, Lancet 2008

Protocol standardsThere is a number of support documents:

• ICH Guideline E6 defines the protocol structure (15 sections with several sub-points each)

• SPIRIT (Standard Protocol Items for Randomized Trials) initiative by statisticians, journal editors and PIs

• CONSORT guidelines to report trials

• EQUATOR Network:http://www.equator-network.org/

• TASC SOP 14: Writing a protocol

• Protocol Template on the TASC website:http://www.tasc-research.org.uk/_page.php?id=208

Definition of a protocolPre-Trial Document containing transparent description of:

• Background and objectives

• Population and interventions

• Methods and statistical analysis

• Ethical and administrative aspects

Title• A title uniquely identifies the project

• It should summarize the aim and methods of the trial

• Important information (e.g. randomized, double-blind, parallel group) should be included in the title

• Indexers on websites such as PubMed may not classify a report correctly if the authors do not explicitly report information in the title

• “A Prospective Randomized Study of Medial Patellofemoral Ligament (MPFL) Reconstruction “

Synopsis• Brief overview over the study aims and

conduct• Should contain sufficient information about a

trial to serve as an accurate record of its conduct

• Should accurately reflect what is included in the full protocol and should not include information that does not appear in the body

Background• The Declaration of Helsinki states that biomedical

research involving people should be based on a thorough knowledge of the scientific literature

• Thus, the need for a new trial should be justified in the introduction

• Explain the scientific background and rationale for the trial

• Report any evidence of the benefits and harms• Ideally, it should include a reference to a systematic

review of previous similar trials or a note of the absence of such trials

Objectives• Objectives are statements what the researcher means

to do• Objectives can be seen as smaller problems in the

larger research area• E.g. “Improving cancer care” is a large research area

which is too broad to be tested within a trial.Impact of physiotherapy on QOL of late stage lung cancer patients is testable within a trial.

• Ensure that objectives are specific, measurable and clinically important

• Changing objectives can sometimes make a trial better

Outcomes• Is the measurable part of the objective• Ensure that the outcome is appropriate to the

objective it serves. • Define clearly what the outcome is and how it will be

measured• If outcomes are measured several times, specify

time point of interest• If possible, use validated and measurable outcomes• If there is more than one assessor, state how many

there are and how discrepancies in measurement will be handled

Trial Design• Define the type of trial, e.g. parallel group, cross-

over or factorial• Define the conceptual framework, e.g. superiority,

non-inferiority, equivalence or other • If a less common design is employed, authors are

encouraged to explain their choice• This is especially important because it might have

implications on sample size or analysis• Include allocation ratio if more than one group, and

unit of allocation (patient, practice, lesion)

Eligibility Criteria• Should be well defined and appropriate to the trial• Define which patient groups are involved and how

they relate to the objectives• Eligibility criteria which are too narrow can

jeopardize the study• Eligibility criteria too wide can invalidate the

outcomes• E.g.: Including Stage IV Cancer patients in a study

examining the effectiveness of two different treatments might fail, as the diseases is too advanced already to make a difference

Sites and Locations

• Goes hand in hand with the eligibility criteria, as certain subjects need certain locations

• E.g.: primary care, hospital wards, specialized units• Healthcare institutions vary in their organisation,

experience, and resources• Social, economic, and cultural environment and the

climate may also affect a study’s validity• Especially important in multicentre trials,

particularly in international studies

Interventions• Describe all interventions including controls in

great detail• It must be possible to be reproduced if necessary• If you compare to “usual practice” describe what

that means, do not assume everyone knows• If interventions are variable, e.g. adaptation of

radiation doses or drug regimes, define rules of application

• In dose-escalation studies, define stopping rules

Sample Size• Sample size calculations are based on previous

trials measuring the outcome• Ensure that the patient population matches the trial

population• Where no previous trials are available, sample size is

often based on assumptions• Sample size is only as accurate as the assumptions• Where more than one outcome is present, sample

size is calculated for the primary outcome usually• It is possible to use the largest sample size to get

the best power

Interim Analysis• Interim Analysis can diminish the trial power• Error rates increase as the number of analysis

increases• E.g. doing 5 interim analysis requires a p-value of

0.01 rather than 0.05, and can give an error rate of 19% rather than 5%

• Use only when necessary• Some trials require interim analysis, e.g. for a DMC• If possible, separate the DMC analysis from the

main analysis

Randomisation• Randomized trials are the gold standard• Randomization requires a program to be written• Sequence generation must be reproducible at any

stage• Define criteria for stratification and minimisation• Try to avoid predictable block sizes• If possible, blinding should be employed • Blinded studies require an independent statistician• Minimization is dynamic and therefore less

predictable

Allocation• Allocation concealment is not blinding• Define how the allocation is applied to the subjects• Define how to conceal allocation until the subject is

included into the trial• Ensure that the person doing the screening is not

familiar with the allocation sequence• Decide whether to include a subject into the trial

before the allocation• If possible, use a third party to allocate subjects

Statistical Analysis• Statistical analysis must be described• Descriptive statistics should be defined for an

overview over the data• Define the appropriate methods for the data• Describe briefly missing or spurious data• Keep the description of the statistical analysis short• Mention checks of normality and independence• Do not hesitate to involve a statistician with this part of

the protocol• A detailed statistical analysis plan (SAP) should be

written during the course of the trial

Statistical Analysis Plan (SAP)

Statistical Analysis Plan (SAP)

• It is critical link between the conduct of the clinical trial and the clinical study report. 

• General statistical analysis is defined in the clinical protocol

• The SAP contains a more technical and detailed elaboration of the analysis

• Recommended by the CONSORT guidelines and ICH Guideline E9 (Statistical Principles for Clinical Trials)

Why write an SAP?

Study Design (Clinical Protocol)

Study Analysis(SAP)

Study Methods(Data CollectionTrial conduct)

Implement the Trial as outlinedin the Protocol

Establish Good StatisticalPractice

Analysis of the plannedstudy design, adapted to

the study methods

Study Analysis provides checks on the original design

GCP requirements• The statistical authorship of the SAP should be clear• Version and date should be clearly defined• The SAP should be reviewed/ updated immediately

before the blinded code is broken or before analysis begins in an unblinded trial

• The SAP should be signed off by the PI/ CI and the Statistician (and other members of the study team where applicable)

• Changes in the SAP after study end should be justified and fully documented in the statistical report

When to write an SAP• The SAP is written during the trial, after the clinical

protocol is final• It must be finalized and signed off before the end of the

trial to avoid bias• If the study is blinded, it must be finalized before the

blind is broken• The SAP should be reviewed and possibly updated as a

result of the blind review of the data• In adaptive trials, it must be finalized before the first

interim analysis • Regulatory factors, such as a special protocol

assessment at the FDA, may affect the timing

Changes in Study Methods• Protocol Amendments during the trial

• Change in the planned treatment (new developments in therapy or guidelines)

• Recruitment does not go as planned

• Early termination of the trial can change patient numbers

• Adding or removing a group

• Addition or removal of a planned test or procedure

• Changes in the outcomes or how they are measured

SAP Contents• A brief description of the purpose• The study rationale as laid out in the protocol• Definition of analysis populations (usually ITT) • How subject data will be summarized (descriptive

statistics or counts/ percents) • Which statistical tests will be used on which data• The statistical methods to be used for the endpoints• When and how to impute missing or partial data• Mocks (or shells) of all unique TLF's • Quality control of the analysis

Writing an SAP• Refer to TASC SOP 05 (Statistical Analysis Plans for

Clinical Trials of Investigational Medicinal Products)http://www.tasc-research.org.uk/_page.php?id=266

• Follow the section headings laid out in the SOP• Contact the study statistician if present• If no study statistician is present, TASC statisticians

can review the SAP• Distribute the SAP to all members of the study team

that can contribute• Finalize the SAP before the study is finished

Benefits• Clear Protocol and SAP show that a study was done

according to GCP standards

• Avoid biased analysis by defining the study populations before study end

• Defined handling of missing data, outliers and data deviations make the analysis more transparent

• Clearly defined subgroup analysis ward off data dredging

• The study report and resulting papers will be more likely to be of high quality

Any Questions?