tools to analyze morphology and spatially mapped molecular data - information technology for cancer...
TRANSCRIPT
![Page 1: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/1.jpg)
NCI Information Technology for Cancer Research CA18092401
Stony Brook: Joel Saltz PI, Tahsin Kurc, Yi Gao, Allen Tannenbaum, Fusheng Wang, Liangjia Zhu, Ivan Kolesov, Romeil Sandhu, Erich Bremer, Jonas AlmeidaEmory: Adam Marcus, Ashish Sharma, Dan Brat, Fadlo Khuri, Rick Cummings, Roberd BostickOak Ridge National Lab: Scott Klasky, Dave PugmireYale: Michael Krauthammer
Tools to Analyze Morphology and Spatially Mapped Molecular Data
![Page 2: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/2.jpg)
Integrative Multi-scale Analysis in Biomedical Informatics
• Predict treatment outcome, select, monitor treatments
• Computer assisted exploration of new classification schemes
• Integrated analysis and presentation of observations, features analytical results –human and machine generated
![Page 3: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/3.jpg)
Pipelines, Database, Data modeling, Visualization
• Specific Aim 1 Analysis pipelines for multi- scale, integrative image analysis.
• Specific Aim 2: Database infrastructure to manage and query image data, image analysis results.
• Specific Aim 3: HPC software that targets clusters, cloud computing, and leadership scale systems.
• Specific Aim 4: Develop visualization middleware for 2D/3D image and feature data and for integrated image and “omic” data.
![Page 4: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/4.jpg)
Quantitative Imaging in Pathology
quip.bmi.stonybrook.edu
![Page 5: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/5.jpg)
Integrative Search linking Pathology and “omics”
Jonas Almeida
![Page 6: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/6.jpg)
caMicroscope/MongoDB - Multiple Algorithm Comparison
![Page 7: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/7.jpg)
Why we need multiple algorithm comparison
![Page 8: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/8.jpg)
Heatmap – Depicts Agreement Between Algorithms
![Page 9: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/9.jpg)
![Page 10: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/10.jpg)
![Page 11: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/11.jpg)
![Page 12: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/12.jpg)
![Page 13: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/13.jpg)
![Page 14: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/14.jpg)
Nuclear Segmentation Algorithms
Algorithm v1a
Algorithm v1
Algorithm v2
![Page 15: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/15.jpg)
Algorithm v1 & v1a
Algorithm v1 Color normalizationChannel decomposition into Hematoxylin and Eosin
Regional level set evolution to extract dark spots
![Page 16: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/16.jpg)
Algorithm v1 & v1a
Algorithm v1a
Algorithm v1 Color normalizationChannel decomposition into Hematoxylin and Eosin
Regional level set evolution to extract dark spots
Color normalizationChannel decomposition into Hematoxylin and Eosin
Regional level set evolution to extract dark spots
Hierarchical mean shift to de-clump
![Page 17: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/17.jpg)
Algorithm v1 & v1a
Algorithm v1a
Algorithm v1Color normalizationChannel decomposition into Hematoxylin and Eosin
Regional level set evolution to extract dark spots
Color normalizationChannel decomposition into Hematoxylin and Eosin
Regional level set evolution to extract dark spots
Hierarchical mean shift to de-clump
Algorithm v2
Additional nuclear recognition criteriaHigh sensitivityCorrect detection of epithelial nuclei, and/or nuclei with clearing
Slightly lower specificity
![Page 18: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/18.jpg)
CNN Based Local Classification for Heterogeneity and MicroenvironmentMultiple Instance Learning
![Page 19: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/19.jpg)
Data Management and Spatial Analyses
Algorithm comparison metrics – Jaccard, DICE and others - over trillionobject spatial datasetsHeatmaps to provide graphical depiction of algorithm
differences/similaritiesCan download markupsData model -- markups, annotations, algorithm provenance, specimen, etc.Support for complex relationships and spatial query: multi-level
granularities, relationships between markups and annotations, spatial andnested relationshipsImplemented in a variety of ways including optimized CPU/GPU,
Hadoop/HDFS, Javascript and IBM DB2 (Wang, Saltz, Kurc)Additional Support: NLM/NCI: Integrative Analysis/Digital Pathology R01LM011119, R01LM009239 (Dual PIs Joel Saltz Fusheng Wang NSF CAREER award
![Page 20: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/20.jpg)
Tool for Heatmap Computation
Tahsin KurcYang Yang ZhuFusheng WangJoel Saltz
![Page 21: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/21.jpg)
Human Computer – Generate Ground TruthYi GaoLiangjia ZhuAllen Tannenbaum
![Page 22: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/22.jpg)
Low Power
• Fast GrowCut segmentation• Intensity insufficient: need user
guidance• Boundaries are most time
consuming for user
![Page 23: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/23.jpg)
Medium Power
• Adaptive thresholdingsegmentation
• Allow for global user input (influence parameter settings)
![Page 24: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/24.jpg)
Crypt/Nuclear Segmentation
• Variational active contour• Context is crucial
![Page 25: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/25.jpg)
Initial (Early!) Prototype
![Page 26: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/26.jpg)
![Page 27: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/27.jpg)
![Page 28: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/28.jpg)
![Page 29: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/29.jpg)
![Page 30: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/30.jpg)
![Page 31: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/31.jpg)
Confocal/Super resolution nuclear morphometry (Slicer!)Ken Shroyer, Yi Gao, Tahsin Kurc, Joel Saltz • Pancreatic Fine Needle
Aspirate• Correlative studies
linking fine needle aspirate cell data, “omic” and Radiology imaging data
• Leverages Marcus foundation virtual biopsy effort
![Page 32: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/32.jpg)
Define thresholds of morphologic characteristics in for normal versus overtly malignant ductal cells. Apply thresholds for the analysis of cytologic features “atypical or “suspicious for carcinoma, with the underlying aim of providing objective data to reduce diagnostic uncertainty.
![Page 33: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/33.jpg)
Cells first prepared via Papanicolaou stain – identified as not suspicious
Preliminary Work
![Page 34: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/34.jpg)
Cells first prepared via Papanicolaou stain – identified as suspicious
![Page 35: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/35.jpg)
Results: one nucleus
Figure 1: 3D confocal imaging and the computed concavity of the nucleus morphology.• A,B, C: three orthogonal views of one nucleus from a healthy cell. Red contour depicts the automatically generated surface around the
nucleus.• D: three-dimensional surface view• E:overlay the concavity color-map over the surface. A region with more red-oriented color indicates more significant concaveness. • Same for F-J for a cancer cell nucleus.
![Page 36: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/36.jpg)
Normal ductal cell nuclei
![Page 37: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/37.jpg)
More ductal cell nuclei
![Page 38: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/38.jpg)
Cancer cell nuclei
![Page 39: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/39.jpg)
VLDB 2012, 2013Spatial Query, Change Detection, Comparison, and Quantification
![Page 40: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/40.jpg)
Spatial Centric – Pathology Imaging “GIS”Point query: human marked point inside a nucleus
.
Window query: return markups contained in a rectangle
Spatial join query: algorithm validation/comparison
Containment query: nuclear featureaggregation in tumor regions
Fusheng Wang
![Page 41: Tools to Analyze Morphology and Spatially Mapped Molecular Data - Information Technology for Cancer Research](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d34971bb61eb49748b4723/html5/thumbnails/41.jpg)
Thanks!