epad: a p latformfor motivation for developing epad s ‐b c i...

Copyright Daniel Rubin 2016 1

EPAD: A PLATFORM FORSTANDARDS‐BASED COLLABORATIVE

IMAGE MANAGEMENT ANDANALYSIS

Daniel Rubin MD, MSMete Akdogan, PhD

Cavit Altindag Ozge YurtseverEmel Alkim, PhD

Department of Biomedical Data Science, Radiology, and Medicine (Biomedical Informatics)

Stanford University

Motivation for developing ePAD

Fragmentation of resources for imaging research◦ Image archives◦ Image viewing/annotation tools◦ Image analysis tools

Lack of tools to integrate image‐ and non‐image data ◦ Data related to images (“annotations”) recorded

separately from images◦ Image labels /ROIs recorded in disparate formats

Opportunity to create a platform ◦ Image archive, project management◦ User‐friendly image viewing/annotation ◦ Image analysis, pipelines cross‐experiment analysis◦ Modular, extensible (plugins)

Copyright © Daniel Rubin 2019

The electronic Physician Annotation Device (ePAD) ePAD is a web based system for managing, viewing,

annotating, and analyzing images Built originally to manage human image data,

recently extended to support animal studies Organizes images in projects◦ Cohorts in a lab◦ Can serve as an image registry for multiple labs

Image viewing/annotation◦ Akin to Osirix but using a web browser with no software

installation◦ Tools to automate annotation (segmentation)◦ Adopts standards for interoperability

Incorporates image processing plugins to integrate analyses as part of image management worfklow

CentOSVirtual Machine

ePAD Viewer & Semantic Annotator

ePADWeb Client

AIM XML (MongoDB)

ePADWeb

Services

Client‐side Plugins

MongoDB

ePAD system architectureServer‐side Plugins

Image processing(segmentation,

quantitation)

Analysis and applications

ePAD GUI AIM‐compliant annotation; supports AIM templates Plugins for quantifying lesion features

Template

ROI Valu

es

Rubin, Willrett, O'Connor, Hage, Kurtz, Moreira, Translational Oncology 7(1):23-35, 2014http://epad.stanford.edu

Quantitative image features

Annotations linked to images

Qualitative image features

Annotation and Image Markup (AIM)

• Emerging standard for semantic annotation of images– Imaging observations

– Anatomy

– ROIs

– Etc

• Makes the image contents explicit, computer‐accessible

• Developed by National Cancer Imaging Program at NCI

Rubin DL, et. al: Medical Imaging on the Semantic Web: Annotation and Image Markup, AAAI 2008.


AIM captures both semantic and quantitative features

Finding: massLocation: Lung, LULLength: 2.3cmWidth: 1.2cmMargins: spiculatedCavitary: YCalcified: NSpatial relationships:Abuts pleural surfaceinvades aortaTexture: {T1, T2, T3,…}Shape: {S1, S2, S3,…}

Controlled terminology:

CAVITARY MASS


ePAD Template Builder


ePAD in MIPS: Cohort/study management


Projects

Studies

Series

ePAD in MIPS: Descriptive ROIs (imported from Osirix)

Annotation Tools: Smart Paintbrush Annotation Tools: ROI interpolation


Uploading Images/Annotations

Images

◦ Upload a zip in ePAD web viewer

◦ DICOM push

◦ Drop in server folder

Annotations

◦ Can import annotations from Osirix

◦ Can import from DICOM‐SR

◦ Can import AIM

Can do custom imports via ePAD API


Exporting Annotations

ePAD Plugins

Image pre‐processing

◦ Lesion segmentation

Pre‐defined radiomics features (standard and novel quantitative image biomarkers)

◦ Linear dimension, volume, SUV, statistics

◦ JJvector > 400 features

◦ QIFE > 500 features through QIFP (QIIR QRR015)

Integration with QIFP for training machine learning applications


Example plugin: Automated segmentation

LesionSeg 2D: • Single slice• Input: A seed polygon• Output: Lesion contour (as coordinates

in AIM/DICOM-SR)

LesionSeg 3D: • Input: A seed longaxis or polygon and

start and end slice numbers• Output: Lesion volume (as DICOM

Segmentation Object-DSO)

ePAD enables collecting radiomics features in radiology workflow


You can extract many different kinds of features to correlate with outcomes

Example plugin: ePAD plugin to compute novel imaging biomarker ADLA: The Attenuation Distribution across the Long Axis

Automatically computed from linear annotation of cancer lesion


Computing treatment response directly from images in ePAD

Application: Assessment of treatment effectiveness• Automated waterfall plot (summary of best response in all patients in a cohort)

• User selects the imaging biomarker to use for assessing response (e.g., “RECIST” vs. “ADLA”)

• Generated by querying image annotations

Example study using ePAD: Predicting recurrence in lung cancer

Depeursinge, et. al, and Rubin, Med Phys 42(4):2054-63, 2015

Rieszwavelets

for texture

AIM enables AI development by mining image data


e.g., discover better imaging biomarkers of treatment response..

Evaluation ‐ Comparison of Waterfall Plots on Evaluation Dataset ePAD compares different imaging biomarkers by

showing differences in waterfall plots generated by each imaging biomarker

Using linear measure (RECIST)

Using SD of pixel intensity

`Image

feature matrix

`

Survival,response to therapy,

etc.AI/ML modelAI/ML model

Imageviewing and annotation

Imagefeature extraction

ePAD is integrated into QIFP

http://qifp/stanford.edu


http://qifp.stanford.edu

From ePAD into radiomics pipelines ePAD can Interface to Other Resources

Kheops

XNAT

NCI TCIA

NCI Imaging Data Commone

ACR AI‐Lab


Future of ePAD: Federated AI Learning

There are barriers to data sharing

Instead of bringing the data to the algorithm, bring the algorithm to the data

ePAD located in different labs can work collaboratively without sharing data

J Am Med Inform Assoc 25(8):945-954, 2018

Conclusion

ePAD fills gaps in current tools for collection annotated image data for radiomics and AI

• Freely available, web based, standards‐based

• Extensible plugins for pre‐processing images

• Automated computation of radiomics image features as part of routine image viewing workflow

• Integration with QIFP for developing AI models

• Geared to fostering an ecosystem of collaborative acquisition of image annotations and image analysis

Acknowledgements

Funding support

NCI QIN grants U01CA142555,1U01CA190214, 1U01CA187947


Thank you.

Contact info:[email protected]

epad: a p latformfor motivation for developing epad s ‐b c i...

Documents