tools to analyze morphology and spatially mapped molecular data - information technology for cancer...

NCI Information Technology for Cancer Research CA18092401

Stony Brook: Joel Saltz PI, Tahsin Kurc, Yi Gao, Allen Tannenbaum, Fusheng Wang, Liangjia Zhu, Ivan Kolesov, Romeil Sandhu, Erich Bremer, Jonas AlmeidaEmory: Adam Marcus, Ashish Sharma, Dan Brat, Fadlo Khuri, Rick Cummings, Roberd BostickOak Ridge National Lab: Scott Klasky, Dave PugmireYale: Michael Krauthammer

Tools to Analyze Morphology and Spatially Mapped Molecular Data

Integrative Multi-scale Analysis in Biomedical Informatics

• Predict treatment outcome, select, monitor treatments

• Computer assisted exploration of new classification schemes

• Integrated analysis and presentation of observations, features analytical results –human and machine generated

Pipelines, Database, Data modeling, Visualization

• Specific Aim 1 Analysis pipelines for multi- scale, integrative image analysis.

• Specific Aim 2: Database infrastructure to manage and query image data, image analysis results.

• Specific Aim 3: HPC software that targets clusters, cloud computing, and leadership scale systems.

• Specific Aim 4: Develop visualization middleware for 2D/3D image and feature data and for integrated image and “omic” data.

Quantitative Imaging in Pathology

quip.bmi.stonybrook.edu

Integrative Search linking Pathology and “omics”

Jonas Almeida

caMicroscope/MongoDB - Multiple Algorithm Comparison

Why we need multiple algorithm comparison

Heatmap – Depicts Agreement Between Algorithms

Nuclear Segmentation Algorithms

Algorithm v1a

Algorithm v1

Algorithm v2

Algorithm v1 & v1a

Algorithm v1 Color normalizationChannel decomposition into Hematoxylin and Eosin

Regional level set evolution to extract dark spots

Algorithm v1 & v1a

Algorithm v1a

Algorithm v1 Color normalizationChannel decomposition into Hematoxylin and Eosin

Regional level set evolution to extract dark spots

Color normalizationChannel decomposition into Hematoxylin and Eosin

Regional level set evolution to extract dark spots

Hierarchical mean shift to de-clump

Algorithm v1 & v1a

Algorithm v1a

Algorithm v1Color normalizationChannel decomposition into Hematoxylin and Eosin

Regional level set evolution to extract dark spots

Color normalizationChannel decomposition into Hematoxylin and Eosin

Regional level set evolution to extract dark spots

Hierarchical mean shift to de-clump

Algorithm v2

Additional nuclear recognition criteriaHigh sensitivityCorrect detection of epithelial nuclei, and/or nuclei with clearing

Slightly lower specificity

CNN Based Local Classification for Heterogeneity and MicroenvironmentMultiple Instance Learning

Data Management and Spatial Analyses

Algorithm comparison metrics – Jaccard, DICE and others - over trillionobject spatial datasetsHeatmaps to provide graphical depiction of algorithm

differences/similaritiesCan download markupsData model -- markups, annotations, algorithm provenance, specimen, etc.Support for complex relationships and spatial query: multi-level

granularities, relationships between markups and annotations, spatial andnested relationshipsImplemented in a variety of ways including optimized CPU/GPU,

Hadoop/HDFS, Javascript and IBM DB2 (Wang, Saltz, Kurc)Additional Support: NLM/NCI: Integrative Analysis/Digital Pathology R01LM011119, R01LM009239 (Dual PIs Joel Saltz Fusheng Wang NSF CAREER award

Tool for Heatmap Computation

Tahsin KurcYang Yang ZhuFusheng WangJoel Saltz

Human Computer – Generate Ground TruthYi GaoLiangjia ZhuAllen Tannenbaum

Low Power

• Fast GrowCut segmentation• Intensity insufficient: need user

guidance• Boundaries are most time

consuming for user

Medium Power

• Adaptive thresholdingsegmentation

• Allow for global user input (influence parameter settings)

Crypt/Nuclear Segmentation

• Variational active contour• Context is crucial

Initial (Early!) Prototype

Confocal/Super resolution nuclear morphometry (Slicer!)Ken Shroyer, Yi Gao, Tahsin Kurc, Joel Saltz • Pancreatic Fine Needle

Aspirate• Correlative studies

linking fine needle aspirate cell data, “omic” and Radiology imaging data

• Leverages Marcus foundation virtual biopsy effort

Define thresholds of morphologic characteristics in for normal versus overtly malignant ductal cells. Apply thresholds for the analysis of cytologic features “atypical or “suspicious for carcinoma, with the underlying aim of providing objective data to reduce diagnostic uncertainty.

Cells first prepared via Papanicolaou stain – identified as not suspicious

Preliminary Work

Cells first prepared via Papanicolaou stain – identified as suspicious

Results: one nucleus

Figure 1: 3D confocal imaging and the computed concavity of the nucleus morphology.• A,B, C: three orthogonal views of one nucleus from a healthy cell. Red contour depicts the automatically generated surface around the

nucleus.• D: three-dimensional surface view• E:overlay the concavity color-map over the surface. A region with more red-oriented color indicates more significant concaveness. • Same for F-J for a cancer cell nucleus.

Normal ductal cell nuclei

More ductal cell nuclei

Cancer cell nuclei

VLDB 2012, 2013Spatial Query, Change Detection, Comparison, and Quantification

Spatial Centric – Pathology Imaging “GIS”Point query: human marked point inside a nucleus

.

Window query: return markups contained in a rectangle

Spatial join query: algorithm validation/comparison

Containment query: nuclear featureaggregation in tumor regions

Fusheng Wang

Thanks!