visualization-based brain mapping jim brinkley (pi) structural informatics group dept biological...
TRANSCRIPT
![Page 1: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/1.jpg)
Visualization-based Brain Mapping
Jim Brinkley (PI)
Structural Informatics Group
Dept Biological Structure
University of Washington, Seattle
Funded by the Human Brain ProjectNIDCD
NIMH
![Page 2: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/2.jpg)
Goals
• An information framework for brain mapping
• Organized around structure– Spatial– Symbolic
• Implemented in a distributed, web-accessible system
• Optimized for language mapping
![Page 3: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/3.jpg)
Photo
![Page 4: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/4.jpg)
Visual Brain Mapper (VBM)
![Page 5: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/5.jpg)
Requirements for visualization-based brain mapping
• Accurate reconstruction of brain surface as seen at neurosurgery
• Different problem than surface reconstruction for unfolding, flattening, analysis
![Page 6: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/6.jpg)
Reconstructing the surface
• Isosurface following
• Needs cleanly segmented cortex: the skull-stripping problem
![Page 7: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/7.jpg)
Approaches to skull-stripping
• Manual
• Deformable models
• 3-D region growing/mathematical morphology
![Page 8: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/8.jpg)
A shape-based approach
• Represent “knowledge” of expected shape and range of variation of cortical envelope
• Use to guide the search for gray-CSF boundary
• Infer the boundary where none is present
• Allow user corrections
![Page 9: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/9.jpg)
Representing shape
• Geometric constraint networks
• Set of local constraints
• Interact together to generate a global shape description
![Page 10: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/10.jpg)
Constraint Network
Vi (Vi1, Vi2,…)
Vj (Vj1, Vj2,…)
Cij
![Page 11: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/11.jpg)
Geometric Constraint Network
Vi (Vi1, Vi2,…)
Vj (Vj1, Vj2,…)
Cij
![Page 12: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/12.jpg)
Radial contour modelStructural component
Ri [Ri,inner … Ri,outer ]
Rj [Rj,inner, Rj,outer ]
Cij
![Page 13: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/13.jpg)
Radial contour modelShape component
Ri
Rj
Cij = [Ri/Rjmin … Ri/Rjmax]
![Page 14: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/14.jpg)
Constraint Propagation
![Page 15: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/15.jpg)
Radial surface model
![Page 16: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/16.jpg)
Radial surface model neighbors
![Page 17: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/17.jpg)
Shape-based skull stripping
• A module of VBM
• Load previously trained shape model
• Load MRI, MRV, MRA volumes
• Find cortical envelope
• Mask skull
• Find cortical surface
• Find veins and arteries
![Page 18: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/18.jpg)
Mid-sagittal plane
![Page 19: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/19.jpg)
Anterior Commissure
![Page 20: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/20.jpg)
Posterior commissure
![Page 21: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/21.jpg)
Cortical bounding box
![Page 22: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/22.jpg)
Initial model
![Page 23: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/23.jpg)
Initial model - end-on
![Page 24: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/24.jpg)
Initial model - lateral view
![Page 25: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/25.jpg)
Propagation from poles
![Page 26: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/26.jpg)
Define one radial
![Page 27: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/27.jpg)
Propagation from one radial
![Page 28: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/28.jpg)
Error
![Page 29: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/29.jpg)
Correct error
![Page 30: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/30.jpg)
Generate mask
![Page 31: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/31.jpg)
Generate mask
![Page 32: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/32.jpg)
Error in mask
![Page 33: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/33.jpg)
Generate mask
![Page 34: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/34.jpg)
Generate isosurface
![Page 35: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/35.jpg)
Extract veins
![Page 36: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/36.jpg)
Mask veins
![Page 37: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/37.jpg)
Vein isosurface
![Page 38: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/38.jpg)
Extract arteries
![Page 39: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/39.jpg)
Mask arteries
![Page 40: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/40.jpg)
Artery isosurface
![Page 41: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/41.jpg)
Vessels
![Page 42: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/42.jpg)
All
![Page 43: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/43.jpg)
Map
![Page 44: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/44.jpg)
WorkflowSelectPatient
Align
SegmentCortex
SegmentVeins
SegmentArteries
Map
FindEnvelope
MaskVolume
Iso-Surface
SelectModel
DefineLandmarks
Propagate & Search
![Page 45: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/45.jpg)
Workflow ArchitectureSelectPatient
Align
SegmentCortex
SegmentVeins
SegmentArteries
Map
(setq workflow (send class-workflow :new :name "BRAIN MAPPER" :steps (list select-step align-step segment-cortex-step segment-veins-step segment-arteries-step mapping-step ) ) )
![Page 46: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/46.jpg)
SpatialKnowledge
3-D Models
Images3-D Maps
SymbolicModelBuilder
Visual Brain Mapper
Authoring Programs
Spatial Information
Internet
ExperimentManager
End User Web Interfaces
Brain MapVisualizer Analyzer
Graphics Server
KnowledgeServer
Structural Information Servers
RepositoryManager
SymbolicKnowledge
Metadata
Symbolic Information
![Page 47: Visualization-based Brain Mapping Jim Brinkley (PI) Structural Informatics Group Dept Biological Structure University of Washington, Seattle Funded by](https://reader035.vdocuments.site/reader035/viewer/2022062422/56649f1d5503460f94c3384c/html5/thumbnails/47.jpg)
Contributors
• Kevin Hinshaw• Jeff Prothero• Andrew Poliakov• Richard Martin• David Corina• Ken Maravilla• George Ojemann
• Rex Jakobovits• Kate Mulligan• Hector Lettich• Linda Shapiro• Cornelius Rosse