1 challenges and opportunities for 21 st century pharmaceutical statisticians christy chuang-stein,...

1

Challenges and Opportunitiesfor 21st Century

Pharmaceutical Statisticians

Christy Chuang-Stein, PhD

Statistical Research and Consulting Center

Pfizer Inc

delete these guides from slide master before printing or giving to the client


2

Outline Evolving role of statisticians in pharmaceutical

industry

The external environment

Responses to the changes in the environment

Develop partnerships and collaborations

Examples of collaborations

A new metric for determining sample size for a confirmatory trial

Summary



3

Evolving Role of Stat in Pharma Industry

Little use of Statistics ==>

“Required” use of Clin Statistics ==>

Tactical use of Statistics ==>

Strategic use of Statistics & “Statistical

Thinking”

1955 2009 and beyond



4

Industry Perspective: “Then” Hired some statisticians to get things through the

regulatory agency (mostly in the US)

“Number crunchers” to get analyses done

“Blessed” clinical trial designs with minimal intellectual participation except sample size

Clinical and manufacturing focus

Very little input outside of “necessary”, low involvement in non-clinical areas

Statisticians played a secondary role



5

Statistician’s Role: Now and the Future

Full and equal partner with basic, clinical & regulatory scientists.

Focus on experimental design and development strategy.

Application of statistical thinking throughout the life cycle of a pharmaceutical product.

Parallel development in other disciplines such as epidemiology, genomics, biomarker development, and risk management expands statistician’s contributions.



6

Statisticians in Pharma Industry

Being technically smart is not enough: Understand the broad clinical, regulatory and public-

health context Communicate statistical strengths and weaknesses

Proactive, not passive: Design: Options available, decision analysis Execution: Quality control and risk mitigation Analysis: Planned and unplanned, strengths/weaknesses Interpretation: Pre-planned or data-driven Presentations/Publications: Keeping audience in mind.

Statistics is a Collaborative Science!!



7

The External Environment Desire for more transparency in clinical research

and clinical reporting

A perceived need for more public “control” over the search for valuable medicines

Loss of public confidence in the clinical research process and the pharmaceutical industry

Loss of public confidence in the regulators

An increasing demand for product safety



8

Our External Stakeholders Health care providers

Patient care givers

Regulators

Journal editors

Shareholders

Health care payer

And most importantly - THE PATIENTS



9

Data Disclosure Desire for more “transparency” in the clinical

research process Protocols Results Logistics

Registries of trials (www.clinicaltrials.gov)

Results from trials involving marketed products are posted (www.clinicalstudyresults.org). Some companies have extended this to all trials and maintained their own trial results websites.



10

How Can Statisticians Help?

Leverage the drive for transparency.

Develop partnerships with academia and government.

Within the limits allowed by internal rules and external regulations, share our processes and approaches to the outside world.



11

Leverage the Drive for Transparency Improve study designs

All studies will be public and designs will be subject to public scrutiny.

Improve communication

Provide informative displays and presentations of results that are customized for the audience.

Endorse the use of common standards

Maximize the efficiency of collecting and sharing data via the use of common standards.



12

Develop Partnerships Foster partnerships among academia, industry and

government.

Reasons: Solve complex problems by sharing resources and

knowledge Integrate ideas and increase credibility through peer

review Create solutions that take into consideration differing

perspectives

Belief: Working together is more effective than working in

isolation



13

Examples of Collaborations Adaptive Designs

Multiple Co-primary Endpoints

Active Control Study

QT Effect of a Pharmaceutical Product

Methods to Handle Missing Data

Dichotomizing Continuous Endpoint

Biomarker Quantification

Predictive Model (Pre-clinical to Clinical)

Safety Data Evaluation

Multi-regional Clinical Trials



14

Adaptive Designs

PhRMA Definition

uses accumulating data to decide on how to modify aspects of the study

without undermining the validity and integrity of the trial

ADWG: Adaptive Design Working Group

Validity means

providing correct statistical inference (such as adjusted p-values, estimates and confidence intervals)

minimizing operational bias

Validity means

providing correct statistical inference (such as adjusted p-values, estimates and confidence intervals)

minimizing operational bias

Integrity means

preplanning, as much as possible, based on intended adaptations

maintaining confidentiality of data

Integrity means

preplanning, as much as possible, based on intended adaptations

maintaining confidentiality of data



15

Drug Development ProcessL

evel

of

un

cert

ain

ty

Development process

PopulationEnd pointsDose rangeDosing frequencyExposureDrug formulation

Sample sizeAdd treatmentDrop treatmentRandomization fractionPopulation

Sample size Modify deltaRandomization fractionDrop treatmentPopulation

Exploratory phase Confirmatory phase



16

Adaptive Design Working Group The Group was formed in 2005 of industry and academic

members to investigate and facilitate opportunities for wider acceptance and usage of adaptive designs and related methodologies.

The Group met face-to-face about twice a year, has published numerous papers & given multiple presentations.

It has met with regulators in US, Canada, Europe and Japan to share experience and discuss issues of concern.

It continues to hold monthly telecons and sponsor a monthly Key Opinion Leader lecture series.

Is planning a workshop with the US Food and Drug Administration (FDA) to discuss a forthcoming FDA guidance on adaptive designs.



17

10 Workstreams of ADWG Education Effort

ICH harmonization

Data monitoring issues and processes

Good adaptive practices

Case studies to share experience

Software user requirements

Material manufacturing and supply

Communications

Key Opinion Leader lecture series

Documentation



18

Multiple Co-Primary Endpoints

There are many disorders where regulators require a new treatment to demonstrate statistically significant benefit on multiple endpoints before accepting the efficacy of the new treatment.



19

Definition of “Co-Primary” Endpoints These endpoints describe the disease in the sense

that an experimental drug that does not show superiority over placebo on all these endpoints is not a viable treatment for the disease.

This could arise because clinicians cannot agree on which endpoint (among several endpoints) is the most relevant one. So, they want evidence of effect on all of them.

In other words, they want statisticians solve the problem for them!



20

How Prevalent Is the Problem?Migraine (4) Alzheimer’s (2)

Acute Pain (3) Acne (3)

Fibromyalgia (3) Male Pattern Baldness (2)

Low Back Pain (3) Vaginal Atrophy (4)

Erectile Dysfunction (3) Organ Transplantation (2)

Osteoarthritis (2) BPH (2)

Menopausal Symptoms (4) Primary Biliary Cirrhosis (4)

Fracture Healing (2) Multiple Sclerosis (2)

Offen et al. Drug Information Journal (2007) 41(1):31-46.



21

Two Examples Alzheimer’s Disease

Alzheimer’s Disease Assessment Scale - Cognitive (ADAS-Cog)

Clinician Interview-Based Impression of Change (CIBIC)

Vaginal Atrophy Self-reported frequency of the moderate or severe

symptoms that were most bothersome

Vaginal pH

% of vaginal parabasal cells from the maturation index

% of vaginal superficial cells from the maturation index



22

The Hypotheses Assume (…) represents the mean effect of a

new treatment over a placebo on K co-primary endpoints, positive i implying positive treatment effect.

The null (H0) and alternative (H1) hypotheses are

H0: 0 or 2 0 or … or 0

H1: > 0 and 2 > 0 … and K > 0

Usual requirement: false positive rate 2.5% (one-sided).



23

Current Regulatory Practice Test H0i vs H1i (the effect on the ith endpoint), each

at the one-sided 2.5% level, i = 1, …, K.

H0i: i 0

H1i: i > 0

Declare efficacy only if all H0i (i = 1, …, K) are rejected at the one-sided 2.5% level. This is the intersection-union test (IUT).

IUT is necessary to control the false positive rate at the one-sided 2.5% level over the entire null space.



24

Overall Statistical Power

Number of co-primary endpoints

Correlation 2 3 4 9

0 64% 51% 41% 13%

0.2 66% 55% 47% 24%

0.5 69% 61% 56% 40%

0.8 73% 69% 66% 57%

*Power for each endpoint is 80%, one-sided =0.025



25

Correlation Coefficient (AD) Alzheimer’s Disease

Alzheimer’s Disease Assessment Scale - Cognitive (ADAS-Cog)

Clinician Interview-Based Impression of Change (CIBIC)

Correlation coefficient between ADAS-Cog and CIBIC is 0.22.



26

Correlation Coefficient (VA)

Vaginal pH % Parabasal Cells

% Superficial Cells

Freq of Symptoms

0.20 0.11 -0.06

Vaginal pH __ 0.32 -0.13

% Parabasal Cells

0.32 __ -0.26

% Superficial Cells

-0.13 -0.26 __



27

% Increase on Sample Size

Number of co-primary endpoints

Correlation 2 3 4 9

0 31 49 62 96

0.2 29 46 58 91

0.5 25 39 49 74

0.8 17 27 32 48

Assume same effect size on each endpoint, tested at one-sided 2.5% level. The objective is to have an 80% overall power.



28

Possible Solutions Optimal solution is a medical one

Reduce to a single dimension– Choose a single primary endpoint– Create a composite endpoint such as ACR20 for

rheumatoid arthritis

Prioritize endpoints, requiring only a positive “trend” in some endpoints. A positive trend could be, e.g. a two-sided P-value < 0.15.

Propose statistical approaches that could provide rationales for adopting a higher significance level for testing each endpoint.



29

ACR20 (-50, -70) Definition At least 20% (50%, 70%) improvement in

# of swollen joint count, AND # of tender joint count, AND At least three of the following

– Patient’s global assessment of disease activity– Physician’s global assessment of disease activity– Patient’s assessment of pain (VAS, Likert, NRS)– Acute-phase reactants (ESR, CRP)– Patient’s assessment of disability (HAQ)

Often the primary endpoint to evaluate RA treatments.



30

Adopting a Higher Sig Level (1-sided)Chuang-Stein et al (2007), Stat in Med, 26:1181-1192.

Correlation 2 3 4

0 0.036 0.055 0.082

0.2 0.036 0.052 0.075

0.4 0.035 0.048 0.066

0.6 0.032 0.043 0.055

0.8 0.030 0.037 0.044



31

Impact on Sample Size (K = 2) The entry in black (red) corresponds to % increase in

sample size under the new (current) approach.

Correlation 80% Power 90% Power

0 18% 31% 12% 23%

0.2 16% 29% 11% 22%

0.4 14% 26% 10% 20%

0.6 14% 22% 11% 18%

0.8 11% 16% 8% 13%



32

Impact on Sample Size (K = 3) The entry in black (red) corresponds to % increase in

sample size under the new (current) approach.

Correlation 80% Power 90% Power

0 19% 49% 11% 36%

0.2 18% 46% 11% 34%

0.4 17% 41% 11% 31%

0.6 15% 35% 11% 27%

0.8 12% 25% 9% 21%



33

A More Serious Issue A common question – Why can’t a sponsor just

increase the sample size to get an adequate overall power?

Statistical power gives the probability of concluding a treatment effect when the true treatment effect is the amount used to size the study.

For a trial to be successful, the new treatment needs to deliver the above assumed effect on all endpoints.

A greater demand of a new treatment!



34

A New Sample Size Paradigm Relevant to trials expected to provide confirmatory

evidence for product registration.

The primary objective of a confirmatory trial is to confirm that the effect of the new treatment is statistically better than a placebo. Recall that statistical power gives the probability of concluding a treatment effect when the true treatment effect is equal to the effect used to size the study.

But, we don’t know for sure if the treatment could deliver on the assumed effect.



35

An Example A small PoC study, n = 25 per group, estimated

treatment effect over a placebo = 2.5 units, estimated standard deviation = 7.14 units. So, estimated effect size = 0.35 (2.5/7.14).

The temptation is to design the next study to detect an effect size of 0.35 based on a two-sided test at the 5% significance level.

Need 128 subjects per group for 80% power, or 172 per group for 90% power.

Source: Chuang-Stein (2006), Pharmaceutical Statistics, 5(4):305-309.



36

Question

What is the probability that we will have a successful trial? Define a successful trial as obtaining a P-value less than 5%. Other criteria can be used.

Is it 80% if we enroll 128 subjects per group, or 90% if we enroll 172 subjects per group?



37

A Frequentist-Bayesian Approach

We know there is variability in our estimate for the treatment effect. Assume we can describe this variability by

| PoC data N(2.5 ; (2/25) (7.14)2)

We could obtain “average success prob” by

ddataPoCPvalueP ) |( )|05.0Pr( Success of Prob



38

A Frequentist-Bayesian Approach The success prob is 63.3% for 128 subjects per

group (80% power); and 67.7% for 172 subjects per group (90% power). These are much lower than the corresponding statistical power.

If the PoC study had 70 subjects per group (instead of 25) with the same estimated effect and standard deviation, then the prob of success is 69.2% for 128 subjects per group and 75.6% for 172 subjects per group.



39

Observations If the required number is much higher than the number

obtained under the traditional approach, this could be a signal that our prior information on treatment effect is not sufficiently strong.

The above suggests that we need to select appropriate metrics to make decisions.

For proof-of-concept study, the metric might be the precision of the treatment effect estimate.

For a dose-response study, the metric might be the probability for selecting the right dose.

For a confirmatory trial, a better metric might be the probability of a successful trial.



40

Summary Pharmaceutical industry, as a whole, is facing

unprecedented challenges. These challenges also create a lot of opportunities.

A successful statistician needs to be responsive to these challenges. Statisticians need to be able to anticipate these challenges and offer solutions!

It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. - Charles Darwin

1 challenges and opportunities for 21 st century pharmaceutical statisticians christy chuang-stein,...

Documents

slide master

secondary role slide

product safety slide

statisticians role

clinical reporting

evolving role of statisticians

sample size clinical

clinical research process