the use of augmented reality in the operating room: a reviewtmm/courses/533-09/projects/... ·...
TRANSCRIPT
![Page 1: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/1.jpg)
The Use of Augmented Reality in the Operating
Room: a ReviewJohn Bartlett
![Page 2: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/2.jpg)
Scope of Survey
Introduction to Augmented Reality
Information Visualization in AR
AR in Medicine
Minimally-Invasive Surgery
Medical Volume Visualization
![Page 3: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/3.jpg)
Previously Covered
Intro to AR
Why we need AR in the operating room
Overview of AR display options
Medical procedures that could benefit
![Page 4: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/4.jpg)
Goal for Today
Impossible to summarize entirety of survey
Instead, present 3 of most significant results:Virtual depth perception in ARAR-guided needle insertionUncertainty in medical volume visualization
Conclude presentation with final thoughts
![Page 5: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/5.jpg)
Virtual Depth Perception in AR
![Page 6: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/6.jpg)
Virtual Depth Perception
Essential for immersion
Problems with optical displays: display set at fixed depth in 3D worlddepths underestimated with HMDs“X-ray vision” uses purposely conflicting depth cues
![Page 7: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/7.jpg)
Depth Cuesbinocular disparity
binocular convergence
accommodative focus
atmospheric haze
linear perspective and foreshortening
motion parallax
occlusion
height in visual field
shading
texture gradient
![Page 8: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/8.jpg)
Experiment: Perceptual Matching
![Page 9: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/9.jpg)
Lesson
Conflicting depth cues create problems for user depth judgement
Standard optical see-through display not ideal for 3D x-ray vision
Solution: add semi-transparent LCD panel or use video display
![Page 10: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/10.jpg)
AR-guided Needle Insertion
![Page 11: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/11.jpg)
Needle Insertion
Accuracy important for biopsies and ablation
Want to avoid sensitive structures, e.g. nerves
MRI guidancemost machines too bulky for patient access
CT guidanceprolonged radiation exposure
![Page 12: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/12.jpg)
AR-enhanced Needle and Targets
MRI or CT data is overlaid using HMD
7-cm virtual cylinder extends from needle tip
Target areas are virtually highlighted
![Page 13: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/13.jpg)
Needle Insertion:Summary
AR technique is ready for implementation
Limitations:complexity of apparatusrequires immobilization of patient or motion correction
Likely reserved for special cases in CTCT-guidance already common
![Page 14: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/14.jpg)
Uncertainty in Medical Volume Vis.
![Page 15: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/15.jpg)
Uncertainty in Medical Volume Vis.
Disregarding this can be life-threateninge.g. could misjudge aortic width and insert stent of wrong size
Radiologists manually adjust parameters to sample range of feasible values
time-consumingnot guaranteed to be comprehensive
![Page 16: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/16.jpg)
Probabilistic Animation
Perform fuzzy classification of tissues
Fraction of frames a sample appears as type X represents probability that X is correct
More effective than traditional static rendering
Sensitivity lens used to localize animation
![Page 17: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/17.jpg)
Probabilistic Animation: Example
![Page 18: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/18.jpg)
Conclusions
![Page 19: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/19.jpg)
Conclusions
Many applications of AR to medicine
Different tools suited for different procedures
Many techniques still need refinement
Physicians will determine what gets adopted
![Page 20: The Use of Augmented Reality in the Operating Room: a Reviewtmm/courses/533-09/projects/... · Introduction to Augmented Reality Information Visualization in AR AR in Medicine](https://reader034.vdocuments.site/reader034/viewer/2022042303/5ece2682cebd7c0f84040e9e/html5/thumbnails/20.jpg)
Thank You