gpu-enabled ultrasound imaging - nvidiaon-demand.gputechconf.com/gtc/2018/presentation/s8764... ·...
TRANSCRIPT
![Page 1: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/1.jpg)
Dr. Christoph Hennersperger
Research Manager, Technical University of MunichResearch Fellow, Trinity College Dublin
CTO, OneProjects
GPU-Enabled Ultrasound ImagingReal-Time, Fully-Flexible Data Processing
![Page 2: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/2.jpg)
1/27/2016 | Slide 2GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Manual Navigation
Reliability on Operator
Complex Diagnostics
![Page 3: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/3.jpg)
1/27/2016 | Slide 3GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Flexible Acquisition
Guidance by/for Expert
Intelligent Imaging
![Page 4: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/4.jpg)
Dr. Christoph Hennersperger
Research Manager, Technical University of MunichResearch Fellow, Trinity College Dublin
CTO, OneProjects
Real-Time, Fully-Flexible Data Acquisition
Data Processing Toward Improved Diagnostics
![Page 5: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/5.jpg)
1/27/2016 | Slide 5
Brief Background on Ultrasound Imaging Workflow
Delay & Focus
Carrier signal
Shape signal
Pulse shape
Active elements
Beamforming Electronic delays for focusing Applicable in transmit and
receive
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
B-mode image
ScanlineScanline RF data
Scanline envelope
Scanline
Processing of received data Radiofrequency data (RF)
scanline signals Envelope detection
(Hilbert transform) Subsampling (decimation)
Postprocessing for visualization Subsequent filters pipeline
(e.g. speckle reduction) Reduction of dynamic range
(Log-compression) Transformation of scanlines to
image (Scan-conversion)
![Page 6: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/6.jpg)
1/27/2016 | Slide 6
The Need for a Software Defined Ultrasound Framework
Most ultrasound systems have limited flexibility Implementation of major processing on DSPs, FPGAs or ASICs Change of specific points require significant changes
Most ultrasound systems are closed systems Access to images only through PACS Proprietary access and interfaces
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Not usable for fast prototyping or research
![Page 7: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/7.jpg)
1/27/2016 | Slide 7
Software Defined Ultrasound Platform for Real-time Applications
Mission: Provide framework to allow covering research aspects from low-level US to high-level applications
Key Design Properties Data and module-driven approach Fully software-defined platform (with GPU)
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Fully flexible design Fast prototyping Transparent storage Fully real-time
![Page 8: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/8.jpg)
1/27/2016 | Slide 8
General Ultrasound Processing Layout
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Transmit Beamforming(Aperture, Delays) Transmit & Receive
Receive Beamforming (Delay and Sum
with Apodization)
Envelope detection incl. Frequency Compounding Log-Compression Scan-Conversion
GPU accelerated
Controllable via
![Page 9: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/9.jpg)
1/27/2016 | Slide 9
Beamforming on GPU
Parallelization for individual scanlines and samples Aperture defines input data (number of channels)
Delay and sum beamformer in SUPRA
Blocks operate on receive scanlines Blocks process individual rx scanlines Shared memory over local aperture (memory access)
Thread operate on receive samples Individual threads process samples of scanlines Local thread performs DAS over aperture (x,y) and depth (z) using
shared memory
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
+
τ2τ1 τ3 τ4 τ5 τ6 τ7 τ8
![Page 10: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/10.jpg)
1/27/2016 | Slide 10
Imaging with SUPRA
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Fast Acquisitions
Storage of Full Data
Direct Configuration
![Page 11: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/11.jpg)
1/27/2016 | Slide 11
Qualitative Evaluation to Proprietary Scanline Imaging
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Cep
haso
nics
SUP
RA
Point Phantom Muscle Fibers Carotid Transverse Carotid Longitudinal
![Page 12: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/12.jpg)
1/27/2016 | Slide 12
From Scanline to Planewave Imaging
64 scanlines Max 300 Hz
64 Angles 300 Hz
10 Angles 1925 Hz
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Scanline Planewave Planewave
![Page 13: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/13.jpg)
1/27/2016 | Slide 13
Real-time Capabilities of Framework
Results for NVIDIA Jetson TX2, mobile GTX and GTX 1080
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
![Page 14: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/14.jpg)
1/27/2016 | Slide 14GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Towards Improving Diagnostic Outcomes with US
![Page 15: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/15.jpg)
1/27/2016 | Slide 15
Ultrasound - Unique Abilities and Challenges
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Data interpretation Image Graph (network) Continuous signals
![Page 16: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/16.jpg)
1/27/2016 | Slide 16
Ultrasound - Unique Abilities and Challenges
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Data interpretation Image Graph (network) Continuous signals
Understanding of physics Signals from reflection Acoustic tissue properties
![Page 17: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/17.jpg)
1/27/2016 | Slide 17
Ultrasound - Unique Abilities and Challenges
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Data interpretation Image Graph (network) Continuous signals
Understanding of physics Signals from reflection Acoustic tissue properties
Challenges and artefacts Shadowing and enhancement Nonlinearity of tissue propagation Interference of waves
![Page 18: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/18.jpg)
1/27/2016 | Slide 19
Modeling Ultrasound as Arbitrarily Sampled Data
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
• Overlapping US-slices• Resampling to regular grid
![Page 19: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/19.jpg)
1/27/2016 | Slide 20GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
• Overlapping US-slices• Resampling to regular grid• Process on regular graph
Loss of information regarding acquisition!
Modeling Ultrasound as Arbitrarily Sampled Data
![Page 20: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/20.jpg)
1/27/2016 | Slide 21
Modeling Ultrasound as Arbitrarily Sampled Data
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Better: Define graph on original samples
• Graph nodes represent US samples• Graph edges represent spatial
structure
Construct edges in “local” coordinates
![Page 21: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/21.jpg)
1/27/2016 | Slide 23
Modeling Ultrasound as Arbitrarily Sampled Data
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Better: Define graph on original samples
• Graph nodes represent US samples• Graph edges represent spatial
structure
Construct edges in “local” coordinates
1) Zu Berge, C. S., Declara, D., Hennersperger, C., Baust, M., & Navab, N. (2015, October). Real-time uncertainty visualization for B-mode ultrasound. IEEE SciVis 2015.2) Hennersperger, C., Mateus, D., Baust, M., & Navab, N. (2014, September). A quadratic energy minimization framework for signal loss estimation from arbitrarily sampled ultrasound data, MICCAI 20153) Virga, S., Zettinig, O., Esposito, M., Pfister, K., Frisch, … & Hennersperger, C. (2016, October). Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms, IEEE IROS 2016
![Page 22: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/22.jpg)
1/27/2016 | Slide 24
Modeling Ultrasound as Arbitrarily Sampled Data
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Better: Define graph on original samples
• Graph nodes represent US samples• Graph edges represent spatial
structure
Construct edges in “local” coordinates
1) Virga, S., Göbl, R., Baust, M., Navab, N., & Hennersperger, C. (2018). Use the force: deformation correction in robotic 3D ultrasound. International journal of computer assisted radiology and surgery, 1-9.
![Page 23: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/23.jpg)
1/27/2016 | Slide 25
Connecting the Dots
Improving Diagnostic Outcomes Domain specific knowledge to improve
diagnostic benefit Data science approaches with need for
data (end to end)
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
SUPRA as enabling technology• Fully open, flexible, and real-time• Tool for rapid demonstration and R&D
![Page 24: GPU-Enabled Ultrasound Imaging - NVIDIAon-demand.gputechconf.com/gtc/2018/presentation/s8764... · 2018-04-11 · Software Defined Ultrasound Platform for Real-time Applications](https://reader033.vdocuments.site/reader033/viewer/2022042420/5f377a408acf1212ac7ae432/html5/thumbnails/24.jpg)
1/27/2016 | Slide 26
Next: Try it Yourself!
GPU-Enabled Ultrasound Imaging | Christoph Hennersperger
Even without an ultrasound machine: https://github.com/IFL-CAMP/supra
Rüdiger Göbl
Christoph Hennersperger
Nassir Navab
This project received funding from the European Union’s H2020 research and innovation programme (No 688279