Demonstrating the impact of statistics in the preclinical area using Bayesian analysis with informative priors

8th-10th October 2014

Non Clinical Statistics Conference

Ros Walley, John Sherington, Joe Rastrick, Gill Watt

2

ן Strategy to demonstrate impact • Features of Bayesian designs• Selling points

ן Preclinical setting• Contrast with Clinical• Related issues

ן Case study

ן First steps

ן Types of control groups

ן Bayesian methodology

ן Summary to date

Outline

Co-branding logo

3

Strategy to demonstrate impact

Features of Bayesian designs

• Explicit way of combining information sources

• Forces early agreement as to relevance of all information sources

• Reduce costs and resources (animal numbers) through informative priors/predictive distributions

• Reduce costs and resources through interim analysis

• Allows more relevant statements to be made at the end of the study e.g. the probability the response rate for drug A is more than 10% better than drug B

• Ranking compounds

• Comparing a combination with its components

• Flexibility in estimation. E.g. one can analyse on the log scale and estimate differences on the linear scale

• Allows for a wide variety of models to be fitted and can address issues such as lack of convergence or outlier-prone data

• Model averaging, model selection

4

Strategy to demonstrate impact

“Selling points of Bayesian methods”

High impact

Focus on in vivo studies that are run again and again

Saving even a few animals per study results in large savings, easily demonstrated

Ground-breaking

Quick search in the literature suggests little use in vivo except some focused applications:

• PK & PK/PD models

• SNPS/genes – pathway analysis. Including a Nature reviews article called “The Bayesian revolution in genetics”

• Lookup proteins

Complements clinical strategy

*

5

Preclinical setting

Contrast with clinical

Preclinical Clinical

Data recording Focus on individual study Hard to pool

Standardized, locatable Easy to pool

Software Graphpad PRISM, EXCEL SAS, R

Relevant data for priors

In house, Run to same protocol; any changes noted.Fantastic source.

Usually from publications. Differences between protocols & populations. Summary level data only.

Size of experiment For in vivo studies, typicallyN=3-10 per group

Typically, N=20-100 per group for a PoC.

Protocol finalisation A set of experiments may be further optimised during their running

Reluctance to trigger a protocol amendment

Novel stats methods

Very traditional approaches used. Little stats support.

Bayesian and adaptive methods implemented and published.

Data interpretation Common to compare compounds across experiments

Focus on individual experimental compound

Preclinical setting

Related issues to consider

Bayesian designs for repeated studies

Allied designs: modelled approach, QC charts

Getting hold of the data; in the right

format

Different output:

internally & externally

Supplement a group with prior info analysis

issues

Rely entirely on historic data for

one group

7

Case study

Background

Mouse model to study inflammatory response after a challenge.

Measures a number of cytokines at 2.5 or 3hrs.

Includes test compounds + three control treatments:

• Negative group (no challenge).

• ‘Positive’ group (challenge, no treatment).

• Comparator with known efficacy

Main comparisons of interest:

• Test compound vs. +ve group (untreated) – Does test compound reduce the response?

Data on 19 studies available (reasonably consistent protocol).

Most frequent in-vivo assay in this therapeutic area.

Data typically presented an experiment at a time, in PRISM

8

First steps

Strategy to smooth the introduction of Bayesian methods

QC charts to demonstrate reproducibility over time

Promoting interval estimates rather than a focus on p-values

Before committing to reducing animals, show “what would have happened if we had done this in the last study” by removing observations from the data set

Advertising and speaking to stakeholders

Publication strategy:

1. A statistical paper

• case studies with details removed

2. For each assay, publish the meta-analysis of the historic data and the planned future data analysis

• preferably in the selected journal to be used for data for new compounds

3. Publish data relating to specific compounds

• having de-risked by the preceding 2 publications

9

First steps

QC charts for control groups

Well received.

Introduces the idea of expt-to-expt variation.

Intuitively, the relevance of the historic controls depends on the size of the study to study variation.

Bayesian analysis can use the historic control information, down-weighting it according to the amount of experiment-to-experiment variation

Low expt-to-expt variation High expt-to-expt variation

10

Types of control groups

■ Not used for formal statistical comparisons. Example uses:

• To ensure challenge is working; to establish a “window”; to check consistency with previous studies; to convert values to %.

• Replace group with a range from a predictive distribution

■ Used for formal comparison vs. test compounds/doses

• Used as the comparison in t-tests ..etc

• Combine down-weighted historic data with the current experiment

11

Bayesian methodology

Outline

■ Analyse historic control treatment group data, excluding the last study.• Bayesian meta-analysis

■ Analyse the last study• Show what would have happened if we had “bought into” the Bayesian approach; omit

animals if necessary

■ Possible options for future studies:• Omit all/some animals from all/some control groups.

• Use historic data as prior information combined with observed data in a Bayesian analysis.

• Use historic data to give a predictive distribution for control group. i.e. don’t include that treatment group in current study.

■ Statistical model based on meta-analytic predictive methodology in Neuenschwander et al

12

Bayesian meta-analytic predictive approach

Statistical model based on Neuenschwander et al (2010)

■ Historic data:, h=1…H:

Observed study data: h | θh, σa2~ N(θh, σa

2)

Study means: θ1… θH ~ N(μ, τ2)

■ Predictions for next study, denoted by *:

True study mean θ* ~ N(μ, τ2) prior for next study

Observed study mean *| θ*, σa2~ N(θ*, σa

2/n*) predictive dn for next study

■ Priors and hyperpriors

μ has vague normal prior

Τ has vague half normal prior (sensitivity analysis carried out)

σa2 has vague gamma prior

13

Bayesian methodology (2)

“Replacing” control groups with predictive distributions

New study data: 8 per group (for the

other groups)

Traditional analysis of current study

Bayesian analysis of historic control

data

QC-chart like limits for control group

Overlay Bayesian analysis in data presentationsSuitable for control groups not

used in statistical comparisons

14

Case study

Predictive distribution

15

Bayesian methodology (4)

Full Bayesian analysis

This is a simplification of the exact analysis

Bayesian analysis of current study

Results and conclusions

New study data: 8 per group

Bayesian analysis of historic control

data

Effectively N control animals with mean, m

Suitable for any control groups but requires a Bayesian analysis for each data set.

16

Case Study

Full Bayesian analysis

The Bayesian analysis gives narrower confidence intervals. It is comparable to using 3 extra animals (11 instead of 8)

__________A@3mg/kg

___________A@30mg/kg

__________B@3mg/kg

___________B@30mg/kg

17

Summary to date

Predictive approach developed for two models

Biologists very positive; implementation being considered

Potential saving: one control group per study

Full Bayesian approach developed for one model

Modest savings in numbers of animals

Software issue; potentially lengthening turnaround times for rapid screens

Biologists suggested starting with something slightly more low-key

QC charts provided an excellent introduction to between and within study variation

Bayesian methods can reduce required resource (animal numbers)

To have the greatest impact, focus on studies repeated very frequently

Even for well controlled in vivo experiments study-to-study variation is not negligible

,

18

References

• Neuenschwander, B., Capkun-Niggli, G., Branson, M. and Spiegelhalter, DJ. Summarizing historical information on controls in clinical trials., Clin Trials 2010 7: 5

• Di Scala L, Kerman J, Neuenschwander B. Collection, synthesis, an interpretation of evidence: a proof-of-concept study in COPD. Statistics in Medicine 2013; 32: 1621–1634.

• Evans, M, Hastings, N and Peacock, B (2000). Statistical Distributions. 3rd Edition. New York: Wiley

• Beaumont and Rannala, The Bayesian revolution in genetics. Nature Reviews Genetics 5, 251-261 (April 2004)

Questions?

19

Thanks!