real world evidence - cddf

Real World Evidence: Opportunities & Challenges

Amy Abernethy, MD PhD

FLATIRON HEALTH

WWW.FLATIRON.COM

4% vs. 96%

Research & Development

Approval & Coverage Decisions

Clinical Practice



Clinical Practice

18% 2001

90%+ 2015

Electronic health record adoption in the U.S. has surged since 2001.



Clinical Practice

Clinical experience data Real–world evidence Clinical annotation for the biologic sciences Quality monitoring & healthcare administration

• Aggregation

• Organization

• Completeness

• Quality & Reliability

• Timeliness

• Access & Security

Patient Reported Outcomes

EHR Labs

Insurance

Treatment Plans Rad

Onc

Surgery Notes

Therapy

Biomarkers

Pathology

Molecular Data

Clinical Trials

Patient Adherence

Notes

Demographics

Hospitalizations

Sites of Metastases

Lab Values

Ancillary Services

Staging

Family History

Comorbidities

Build solutions for oncology

OncoAnalyticsTM OncoEMRTM Academic Outcomes

Database

Cloud-based oncology-specific Electronic

Medical Record (EMR)

EMR-agnostic analytics portal that allows

administrators and physicians to explore and

analyze practice data

Continuously-aggregated retrospective registry to facilitate research, and

quality and process improvement

Research-Grade Data

Demographics

Diagnosis

Visits

Labs

e-Prescribing

Pathology Report

Discharge Notes

Radiology Report

Physician Notes

Electronic Health Record

Structured Data Unstructured Data Outside Practice

Hospital

Lab

Structured Data

Processing

Unstructured Data

Processing

Standard EHR Data

Processing the Electronic Health Record

All terms are mapped to a common vocabulary, standard across all centers

Matching algorithms can predict matches for ~90% of terms

Data processing engine transcodes terms in real-time

Any unmatched term is flagged for clinical review by Flatiron MD/RN

2220 Blood Serum Albumin g/dL QD25001600 ALBUMIN/GLOBULIN RATIO QD (calc) QD25001400 ALBUMIN QD g/dL QD50058600 ALBUMIN % QD50055700 ALBUMIN g/dL CL3215104 Albumin % (EPR) % LC001081 ALBUMIN, SERUM (001081) g/dL LC003718 Albumin, U % LC001488 Albumin g/dL LC133751 Albumin, U % CL3215162 Albumin%, Urine % CL3215160 Albumin, Urine mg/24hr 3234 ALBUMIN SS g/dL LC133686 Albumin, U % QD50060710 MICROALBUMIN mg/dL QD50061100 MICROALBUMIN/CREATININE RATIO,

RANDOM URINE mcg/mg creat

QD85991610 ALBUMIN relative % 50058600 ALBUMIN UPEP RAND % CL3210074 ALBUMIN LEVEL g/dL QD86008211 ALBUMIN/GLOBULIN RATIO (calc) LC149520 Albumin g/dL QD45069600 PREALBUMIN mg/dL QD900415245 ALBUMIN, SERUM mg/dl QD900429745 ALBUMIN g/dL CL3215124 Albumin Electrophoresis g/dL LC016931 Prealbumin mg/dL QD50060900 MICROALBUMIN, 24 HOUR UR mcg/min QD85994821 ALBUMIN,SERUM g/dL CL3213320 PREALBUMIN mg/dL QD85995225 PROTEIN ELECTROPHORESIS

ALBUMIN g/dL

1751-7 Albumin [Mass/volume] in Serum or Plasma g/dL

Structured Data Processing

Leverage unstructured data processing to drive accuracy and completeness of data elements

• KRAS testing status • KRAS test result • Date sample was collected • Date sample was received in lab • Date result was provided to physician

Capturing Key Data From Unstructured Notes

Technology leverages people & capabilities

Chart abstraction Expensive Inaccurate Slow

Technology-Enabled Abstraction using Patient Manager®

Abstractor feedback and continuous QA

EHR Source Data

Technology-enabled abstraction Use technology to

make people more efficient

Human review of every data point

Centralized infrastructure allows for rigorous QA/QC

Pathology report

Radiology report

Physician notes

High Efficiency Chart Abstraction

Variable Structured data only Flatiron data completeness

Smoking status 0%1 94%

Histology 37% 99%2

Stage 61% 95%

ALK results (of those tested) 9% 100%3

EGFR results (of those tested) 11% 99%3

Data quality must be a focus

Site of Met Inter-abstractor Agreement Kappa

Bone 97% 0.93

Brain 96% 0.91

Liver 92% 0.83

Lung 94% 0.87

Processed EHR Data

Mortality data

Progression data Claims data

Genomic data

Derived variables Patient reported data

A dataset is an amalgamation of many patient stories

Diagnosed with Stage II NSCLC

Develops metastatic disease

Tested for EGFR and ALK

Undergoes surgery for early-stage disease

Starts 1L therapy

Progresses on 1L, tested for PD-L1 and/or re-tested for EGFR

Starts 2L therapy, deteriorates and is hospitalized Death





Starts 1L therapy



Organize datasets around patient stories

• Patient age • Gender • Race • Insurance • Stage

• Regimen name • Duration of

therapy • Dosage • Concomitant

meds

Structured EMR data

• Date of death





Starts 1L therapy




• Patient age • Gender • Race • Insurance • Stage • TNM staging

• Sites of metastases

• Biopsy date • Type of test

conducted • Turnaround

time for test • Number of

unsuccessful tests

• Test result, if successful

• Type of EGFR mutation

• Date of surgery



meds • Response

• Biopsy date • Lab name • Test result • T790M

mutation

Structured EMR data Unstructured EMR data

• Date of death





Starts 1L therapy









unsuccessful tests



• Date of surgery



meds • Response


mutation

Structured EMR data Unstructured EMR data External datasets

• Date of death

• Date of death





Starts 1L therapy



• Date of hospitalization

• Cost of care




• Time to recurrence




unsuccessful tests



• Date of surgery



meds • Response • Line of therapy


mutation

Structured EMR data Unstructured EMR data External datasets Algorithms

• Date of death

• Date of death

• Consensus date of death





Starts 1L therapy



• Date of hospitalization

• Cost of care • Line of

therapy






Starts 1L therapy



Curating the complete picture

Patient Demographics Stage Diagnosis Date

Biomarkers Treatment Response Hospital admissions

Mortality

A

B

C

D





Starts 1L therapy






Mortality

A

B

C

D

EGFR





Starts 1L therapy






Mortality

A

B

C

D

EGFR

• Abstracted by Sue Smith on 1/5/16 at 10:10am • Biomarker documents were reviewed • Medical record from West Florida Cancer Clinic • Quality of PDL1 abstraction

• Completeness is 99% • Sue Smith is 96% accurate at last testing • Inter-abstractor agreement 97% • Kappa 0.93

• Audit trail for any changes • Dataset freeze and storage

1.1M+ Always on

Up to date

Nationally representative cohorts Linkable

Single data model

Processed EHR Data

Mortality data

Progression data Claims data

Genomic data

Derived variables Patient reported data

§ Disease Growth

Flatiron-FMI Clinicogenomics (CG) Registry

March ‘16 Jun ‘16 Estimated

Dec’16 Lung cancer 770 1093 ~1500

Breast cancer - ~800 ~1100

Link date FH FMI

Total overlap (name, DOB,

gender) 8/20/2015 2,243,287 51,156 5,098 9/26/2015 4,750,096 53,777 7,967 1/20/2016 5,726,817 64,896 10,450 4/12/2016 6,944,503 73,185 12,478

§ Broad Cohort Growth

>50% growth in total

overlap over the past 9 months

Patient Reported Outcomes

EHR Labs

Insurance

Treatment Plans Rad

Onc

Surgery Notes

Therapy

Biomarkers

Pathology

Molecular Data

Clinical Trials

Patient Adherence

Notes

Demographics

Hospitalizations

Sites of Metastases

Lab Values

Ancillary Services

Staging

Family History

Comorbidities

Organized Readily Analyzable

High quality Linked

Processed EHR data is now:

So….what’s next?

4/30/16

N=25,958

12% 25%

69%

2.6% 12.1% 42%

28.3% 995/3519 1.4%

11% 1st line

48.1% 2nd line+ 2.7%

NSCLC: Comparison of CG Registry to TCGA


DATA FROM FH-FMI NSCLC CG Registry

CG Registry: Genomic Landscape of NSCLC


DATA FROM FH-FMI NSCLC CG Registry

Reproduces and extends findings of the The Cancer Genome Atlas project

Overall Survival by EGFR Mutation

Cohort Selection Diagnosed with advanced NSCLC on or after 1/1/2011 Two clinic visits on or after 1/1/2011 Valid EGFR Test

EGFR Mutation N

Wild-Type 10362

L858R Point Mutation in Exon 21 644

Other EGFR Mutation 1413

Overall Survival by EGFR Mutation

Wild-Type L858R Point Mutation in Exon 21 Other EGFR Mutation

100%

75 75%

50%

25%

50%

0 1 2 3 4 5

Time [Years]

Surv

ival

from

Adv

ance

d D

X

Real-time analytic solutions


PRELIMINARY ANALYSIS: DATA UNDER REVIEW

• Analysis at scale

• Real world endpoints

• Data quality defined at the variable level

• RWE regulatory use cases

• Prospective data collection

• Pragmatic clinical trials run in the EHR and using RWE

WHAT ELSE?

real world evidence - cddf

Documents