Setting the cut point bar

An industry perspective

16th May 2016

Manoj Rajadhyaksha

Bioanalytical Sciences

Manoj.rajadhyaksa@regeneron.com

Introduction

What are most of us observing with

respect to ADA assay cut points (ACPs)?

Mock study specific cut point case study:

For illustration

What is relevant?: For discussion

Background

Provided by Dr. Ron Bowsher

ACP history and evolution

– Clinical Immunogenicity Risk Experience Phase

– Stratification of products in Risk Assessment Categories Phase High, Moderate, Low Association of bioanalytical strategies with the expected risk

of the clinical outcome to the patient

– Initial Proposals for establishing ADA assay cut points Use of statistics in context of data obtained Balanced experimental design and elimination of outliers Taking into account the analytical and biological variability To ensure application of objective criteria (not subjective eg.

50% for CCP)

– Industry experience of bioanalytical outcome using these ACPs.

General observations in clinical studies in industry

(Personal Survey)

– With respect to Human monoclonal antibody drugs

– Validation screen and confirmation cut points are established using

normal human sera or commercial patient sera

Validation screen cut point is established with a 5% FP rate

– Early clinical phase false positive rate observed ~4-8% – FP: defined as % ADA positives in baseline samples of clinical

study population

Less apparent, small “n”

– Later phase clinical false positive rate observed ~25-50%

Reverse observation: Clinical FP rate drops to < 1%

If observed in only 1 study, data comparison across studies is problematic

Mock Case study for illustration:

– Human monoclonal antibody drug

– Assay cut point values established as recommended Shankar G et al. 2008. Journal of Pharmaceutical and Biomedical Analysis

– Validation screen cut point established with a 5% FP rate Bridging ADA assay on MSD platform

screen and confirmation cut point established using normal human subjects

(n=50)

Floating Screen CP multiplicative factor ~1.5 (Parametric/Non-parametric)

1% FP rate Confirmation CP ~20% Inh

Floating Titer CP factor ~ 1.4 (Parametric)

– Late stage clinical analysis Bridging ADA assay on MSD platform (No change in Life Cycle Management)

Observed false positive rate in clinical study baseline samples ~15-20%

Mock in-study cut point analysis

– In study cut point analysis: screen and confirmation cut point established using baseline patient samples

~400 values from ~ >100 baseline study samples (For ACP: Screen, Confirmation and Titer)

– Cut point analysis out put: No statistical difference in least square means for the analyst effect

statistically significant difference detected among run means

Floating Screen CP multiplicative factor ~ 1.8

1% FP rate Confirmation CP ~26% Inh

Floating Titer CP multiplicative factor ~ 1.9

Validation Clinical Study

Screen CP ~1.5 ~1.8

Confirmation CP ~20% ~26%

Titer CP ~1.4 ~1.9

Obsvd FP rate ~23% ~4.7%

Observations: Mock study

Comm. NHV

Comm. Indication A; Clinical Indication A

Comm. Indication B; Clinical Indication B

NHV Comm

A

Comm

B

Clin

A

Clin

B

Observations: Mock study

Pop 1 Pop 2 P Use

HNV Clin A < 0.001 ×

HNV Clin B < 0.001 ×

Comm

B

Clin B < 0.001 ×

Comm

A

Clin A 0.5237 √

Published Observations

Modified from Gerry Kolaitis et al poster. BMS

This direct comparative analysis shows that depending on the indication, the pre-

study validation CP v/s the population specific in-study CP, provide different FP

rates for a given study population.

Contributing Factors

– Validation ACP established without target clinical population

Disease state matrix is very rare or very expensive

Availability of a surrogate matrix – not representative of target population

– NHV or commercial samples heterogeneous in composition

sub- populations of TN, TP and FP

No patient stratification or criteria

Stage of disease at which sample is taken

– Multimeric targets or receptors fluctuate through course of disease

– Clinical trial (target) population little less heterogeneous

Inclusion criteria and exclusion criteria

Stratification based on study objective (Baseline characteristics)

– Drug development franchise

Multiple indications pursued for the same drug and same MOA

– Patient related factors

Age group (Pediatric and geriatric populations)

Concomitant medications

Sex

Pre-existing antibodies

Impact of “clinically not-relevant” cut point

– Impact on resources

several rounds of re-analysis

– Impact on timelines

Factors to be considered, tested, mitigated – Any potential matrix interference? – Any Target interference? – Any pre-existing antibodies?

Larger amount of samples going into confirmation assay

For very large long term studies (> 5000 patients) the issue may be realized mid-way in the trial – forcing lot of rework

– Impact on overall study costs

Bioanalysis/re-analysis if done at CROs– larger amount of samples going into confirmation assay

Statistical analysis done at CROs

Impact of “clinically not-relevant” cut point

– Impact on product distinction

Two products with similar MOA Equivalent efficacy, safety profile, dosing regimens

Dramatic difference in reported immunogenicity incidence in label

– Baseline FP rate: 0% Drug A v/s 8% Drug B

– Treatment Emergent: 1% Drug A v/s 20% Drug B

Does study specific cut point resolve all

issues?

– Significant interference in baseline matrix needs to be resolved Target interference by multimeric targets

Anti-allotype, anti-carbohydrate, anti-frame work, pre-existing antibodies

– Subsequent study populations may be significantly different Different indications pursued for the same drug

Different geographical sectors included/excluded in a given trial

Different study designs

– Monotherapy, Combination Therapy +/- immune modulators

– These are bioanalytical cut points that reflect analytical and biological

variability (May not be clinically relevant) “conservative” and “non-conservative” cut point

– A sample positive in the assay does not mean it has necessary

“levels” of ADA to cause a meaningful clinical impact (PK, safety,

efficacy)

– Concept of a clinically relevant cut point

14

Variable outcome of different methods

(Mock Data)

Non-parametric

Simple Parametric

Robust Parametric

Quantile lower bound

Least Conservative

Most Conservative

What is relevant? (Mock Data)S

ign

al/

No

ise

1

2

3

Points to Ponder

– Should assay validation cut points be “fit for purpose” to assess

validation parameters? Used for FIH or FIP trials with small “n”

– As applicable, can relevant (in study) cut points be set during pivotal

studies when the “target population is accessible”?

– Is application of the most conservative statistical method for ACPs for

all drug products valid? Can cut point rules be established based on “risk assessments”?

– Less conservative for low risk

– More conservative for high risk

– Is ADA impact on PK clinically relevant?

– Is there a need to build a consensus approach based more on

“biological” data and assisted by “statistical” approach? Various sponsors developing and presenting their own cut point assessment

rules

Acknowledgements

REGN BAS Team

Al Torri

Thomas Daly

Giane Sumner

Matthew Andisik

Michael Partridge

Ching Ha Lai

Weiping Shao

B2S Team

Wendell Smith

Rocco Brunelle

Ron Bowsher