introduction of computer vision benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04...
TRANSCRIPT
![Page 1: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/1.jpg)
Introduction of Computer Vision Benchmark for Autonomous Driving
Automotive Cyber-Physical Systems Cheng-Yang Fu
01/04/2015
![Page 2: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/2.jpg)
State-of-the-Art• Localization, Path Planning, and Obstacle avoidance
![Page 3: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/3.jpg)
State-of-the-Art• Heavy usage of Valodyne and detailed Maps
May 2014 Embedded Vision Summit Keynote Presentation: "Self-Driving Cars," Nathaniel Fairfield, Google
![Page 4: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/4.jpg)
State-of-the-Art• Heavy usage of Valodyne and detailed Maps
May 2014 Embedded Vision Summit Keynote Presentation: "Self-Driving Cars," Nathaniel Fairfield, Google
![Page 5: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/5.jpg)
State-of-the-Art• Heavy usage of Valodyne and detailed Maps
May 2014 Embedded Vision Summit Keynote Presentation: "Self-Driving Cars," Nathaniel Fairfield, Google
Do we have cheaper solutions?
![Page 6: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/6.jpg)
KITTI Vision Benchmark Suite• Goal : Autonomous driving cheap sensors
• Problem for Computer vision• Stereo, optical flow , Visual Odometry
Karlsruhe Institute of Technology (KIT) and Toyota Technological Institute at Chicago (TTI-C)
![Page 7: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/7.jpg)
KITTI Vision Benchmark Suite• Goal : Autonomous driving cheap sensors
• Problem for Computer vision• Stereo, optical flow , Visual Odometry• Object Detection, Recognition, and Tracking
Karlsruhe Institute of Technology (KIT) and Toyota Technological Institute at Chicago (TTI-C)
![Page 8: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/8.jpg)
KITTI Vision Benchmark Suite• Goal : Autonomous driving cheap sensors
• Problem for Computer vision• Stereo, optical flow , Visual Odometry• Object Detection, Recognition, and Tracking
• scene Understanding
Karlsruhe Institute of Technology (KIT) and Toyota Technological Institute at Chicago (TTI-C)
![Page 9: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/9.jpg)
KITTI Vision Benchmark : Data Collection
• Two stereo rigs(1392x512 pixels, 54 cm , 90º opening)
• Velodyne laser scanner, GPS +IMU
• 6 hours at 10 frame per second
![Page 10: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/10.jpg)
Annotation• 3D object labels, Occlusion labels
![Page 11: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/11.jpg)
Evaluation
http://www.cvlibs.net/datasets/kitti/index.php
![Page 12: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/12.jpg)
3D Reconstruction Goal: given 2 cameras mounted on top of the car, reconstruct theenvironment in 3D.
![Page 13: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/13.jpg)
Stereo• Input Data : Stereo Images
• Output : Depth
![Page 14: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/14.jpg)
![Page 15: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/15.jpg)
DispletsDisplets: Resolving Stereo Ambiguities using Object
Knowledge (CVPR 2015)
![Page 16: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/16.jpg)
Optical Flow• Input Data : Two Temporally Adjacent Images
• Output : Depth
![Page 17: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/17.jpg)
![Page 18: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/18.jpg)
VC-SFView-Consistent 3D Scene
Flow Estimation over Multiple Frames
, ECCV 2014
![Page 19: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/19.jpg)
![Page 20: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/20.jpg)
Odometry:motivation• Localization is crucial for autonomous systems
• GPS has limitations in terms of reliability and availability
• Place recognition techniques use image features or depth maps and a database of previously collected images
![Page 21: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/21.jpg)
Odometry:motivation• Use visual input to localize a vehicle
![Page 22: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/22.jpg)
Visual Odometry
[M. Brubaker, A. Geiger and R. Urtasun, CVPR13 best paper runner up award]
![Page 23: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/23.jpg)
Visual Odometry
[M. Brubaker, A. Geiger and R. Urtasun, CVPR13 best paper runner up award]
• Velocity • Direction
![Page 24: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/24.jpg)
![Page 25: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/25.jpg)
![Page 26: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/26.jpg)
![Page 27: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/27.jpg)
![Page 28: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/28.jpg)
![Page 29: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/29.jpg)
Detection • Car, Pedestrian, Cyclist
detection
• Object Detection and Orientation Estimation
![Page 30: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/30.jpg)
![Page 31: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/31.jpg)
Regionlets for Generic Object Detection, ICCV 2013
![Page 32: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/32.jpg)
Tracking• Car and Pedestrian classes
![Page 33: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/33.jpg)
![Page 34: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/34.jpg)
Road Estimation• UU - Urban unmarked
• UM - Urban Marked
• UMM - Urban Multiple Marked lanes
![Page 35: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/35.jpg)
![Page 36: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/36.jpg)
DDN : Deep Deconvolutional Networks for Scene Parsing, arXiv
![Page 37: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/37.jpg)
http://blog.peltarion.com/2014/06/22/deep-learning-and-deep-neural-networks-in-synapse/
Deep Neural Networks
![Page 38: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/38.jpg)
http://blog.peltarion.com/2014/06/22/deep-learning-and-deep-neural-networks-in-synapse/
Deep Neural Networks : Why Popular ? • GPU• Large scale Data
![Page 39: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/39.jpg)
![Page 40: Introduction of Computer Vision Benchmark for …anderson/teach/comp790a/slides/2015-04...2015/04/04 · Introduction of Computer Vision Benchmark for Autonomous Driving Automotive](https://reader034.vdocuments.site/reader034/viewer/2022042318/5f0802097e708231d41fde00/html5/thumbnails/40.jpg)
Reference• KITTI website : http://www.cvlibs.net/datasets/kitti/
• First International Workshop on Computer Vision for Autonomous Driving
• Visual Scene Understanding for Autonomous Systems
• Raquel Urtasun, University of Toronto • Regionlet
• http://www.xiaoyumu.com/project/kitti • DDN
• http://arxiv.org/pdf/1411.4101v1.pdf