elif albuz vision software - nvidiaon-demand.gputechconf.com/siggraph/2014/presentation/sg... ·...
TRANSCRIPT
![Page 1: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/1.jpg)
GPUS FOR AUTONOMOUS DRIVING ELIF ALBUZ VISION SOFTWARE
![Page 2: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/2.jpg)
WHY AUTONOMOUS? Safety
Efficient utilization of resources (roads, time, fuel)
Mobility
![Page 3: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/3.jpg)
TRENDS & REGULATIONS
![Page 4: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/4.jpg)
SELF-DRIVING NEEDS.. Sensor and image data processing
Interpretation of visual sensor data
COMPUTER VISION
Power efficiency Safety critical processing
![Page 5: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/5.jpg)
HIGH COMPUTATIONAL COMPLEXITY Computer Vision
Detect pedestrians, road plane, traffic signs, lanes,..
Sensor fusion Path planning and control
![Page 6: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/6.jpg)
HIGHLY PARALLEL +
HIGH COMPUTATIONAL COMPLEXITY =
GPU
![Page 7: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/7.jpg)
NVIDIA Automotive
6.2 M Cars on the road
More coming…
20+ Brands 100+ Models
![Page 8: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/8.jpg)
UNIFY GPU & TEGRA ARCHITECTURE
MOBILE ARCHITECTURE
Maxwell
Kepler
Tesla
Fermi
Tegra 3
Tegra 4
GPU ARCHITECTURE
TEGRA K1
![Page 9: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/9.jpg)
TEGRA K1 192-Core Super Chip 326 GFLOPs CUDA
![Page 10: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/10.jpg)
ONE ARCHITECTURE 1X
~200 Cores 10X
~2000 Cores 100X
~20,000 Cores
CUDA
![Page 11: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/11.jpg)
ONE ARCHITECTURE MULTIPLE WAYS TO AUTONOMOUS DRIVING
1X ~200 Cores
10X ~2000 Cores
100X ~20,000 Cores
Massive Vision Learning 3D Map processing
High-end Vision functionality Development and prototyping
Embedded Vision Deployment
CUDA
![Page 12: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/12.jpg)
TEGRA K1 DEVELOPMENT PLATFORMS
JETSON TK1 PRO gigE, USB3.0, HDMI, CANBUS
running Vibrante Linux
AUTOMOTIVE GRADE
JETSON TK1 gigE, USB3.0, HDMI
running Linux4Tegra
$192
![Page 13: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/13.jpg)
COMPUTER VISION ON CUDA
Feature Detection / Tracking ~30 GFLOPS @ 30 Hz
Object Recognition / Tracking
~180 GFLOPS @ 30 Hz
3D Scene Interpretation ~280 GFLOPS @ 30 Hz
![Page 14: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/14.jpg)
VISUAL SUPERCOMPUTING WITH TEGRA K1
![Page 15: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/15.jpg)
COMPUTER VISION ON GPUS
NPP OpenCV VisionWorks NVIDIA Performance
Primitives for basic image & vision processing
Hundreds of functions,
accelerating OpenCV and other libraries
Open Source Computer Vision Libraries
> 900 functions, widely adopted for prototyping
NVIDIA Computer Vision Library
High performance functions and algorithms
![Page 16: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/16.jpg)
SOFTWARE STACK
Application Code
Sample Pipelines
Tegra/Kepler dGPU
CUDA
VisionWorks Framework
OpenVX
VisionWorks APIs
…
Classifier Corner Detection
Feature Tracking
Hough Detection
![Page 17: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/17.jpg)
SAMPLE PIPELINES Application Code
Demo & Sample Pipelines
Feature Tracker
Hough Detection
Feature Tracker Hough
Circle/Line Object Tracker Optical Flow
Denoising
![Page 18: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/18.jpg)
PRIMITIVES
Absolute Difference Accumulate Image Accumulate Squared Accumulate Weighted Add Affine Warp And BoxFilter Canny Edge Detector Channel Combine Channel Extract Color Convert Convert Depth Convolution Dilation Filter
Erosion Filter Fast Corners Gaussian Filter Gaussian Image Pyr. Harris Corners Histogram Histogram Equalization Integral Image Magnitude Mean Std Deviation Median Filter Min Max Locations Multiply Not Optical Flow (LK)
Or Perspective Warp Phase Node Remap Scale Image Sobel 3x3 Subtract Table Lookup Threshold Xor
Bilateral Filter Convert To Gray Corner FAST Corner Harris Fast NLM Denoising Find Homography HOG (Hist. of Oriented Gradients) Hough Circles Hough Lines IME (Iterative Motion Estimation)
Integral Histogram Median Flow Optical Flow Farneback RANSAC Scharr3x3 Soft Cascade Detector Stereo Block Matching Object Tracker Algorithm SLAM Algorithm
Additional NVIDIA Primitives All OpenVX Primitives
![Page 19: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/19.jpg)
THE ROAD TO THE SELF DRIVING CAR DEPENDS ON VISUAL SUPERCOMPUTING
![Page 20: ELIF ALBUZ VISION SOFTWARE - NVIDIAon-demand.gputechconf.com/siggraph/2014/presentation/SG... · 2014-08-13 · ELIF ALBUZ . VISION SOFTWARE . WHY AUTONOMOUS? Safety Efficient utilization](https://reader033.vdocuments.site/reader033/viewer/2022050517/5fa0a307a1d22734ff0a7315/html5/thumbnails/20.jpg)
QUESTIONS?