![Page 1: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/1.jpg)
Augmented Vehicular Reality (AVR)
Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan
![Page 2: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/2.jpg)
Reliable Autonomous Driving
Human drivers can achieve high reliability• ~ 100,000,000 miles between fatalities
Can autonomous cars reach the same level of reliability?2
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 3: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/3.jpg)
Stereo Camera
Reliability with Advanced Sensing
LiDAR
Radar
3
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 4: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/4.jpg)
4
![Page 5: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/5.jpg)
ShadowSensors limited by
Line-of-sight
5
Car
![Page 6: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/6.jpg)
6
![Page 7: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/7.jpg)
7
![Page 8: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/8.jpg)
Extended Vision
8
![Page 9: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/9.jpg)
Extended Vision Improves Safety
9
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 10: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/10.jpg)
10
![Page 11: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/11.jpg)
AVR Introduction AVR Design AVR Evaluation AVR Future Work
11
![Page 12: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/12.jpg)
12
![Page 13: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/13.jpg)
13
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 14: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/14.jpg)
14
AVR: Augmented Vehicular Reality
![Page 15: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/15.jpg)
• Accurate Positioning
• Extremely Limited V2V Bandwidth
• Low Latency
AVR: Challenges
dx
r
15
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 16: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/16.jpg)
AVR Contributions
• Accurate Positioning• Relative Positioning using 3D Feature Map
• Perspective Transformation
• Extremely Limited V2V bandwidth • Dynamic Object Extraction
• Adaptive Motion Vector Transmission
• Low Latency• Pipeline Optimization
• Motion Prediction to Hide Latency
16
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 17: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/17.jpg)
AVR Contributions
• Accurate Positioning• Relative Positioning using 3D Feature Map
• Perspective Transformation
• Extremely Limited V2V bandwidth • Dynamic Object Extraction
• Adaptive Motion Vector Transmission
• Low Latency• Pipeline Optimization
• Motion Prediction to Hide Latency
17
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 18: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/18.jpg)
3D Perception: The Point Cloud
. Voxel • Position (x, y, z)• Color (R, G, B)• Reflection Intensity
18
AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 19: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/19.jpg)
Relative Positioning
?
AVR Introduction AVR Design AVR Evaluation AVR Future Work
19
![Page 20: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/20.jpg)
Sparse Features for LocalizationStatic Features
20AVR Introduction AVR Design AVR Evaluation AVR Future Work
20
![Page 21: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/21.jpg)
Collecting the 3D feature map
21AVR Introduction AVR Design AVR Evaluation AVR Future Work
21
AVR uses SLAM to build a 3D feature map
![Page 22: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/22.jpg)
Localization using sparse 3D map
22AVR Introduction AVR Design AVR Evaluation AVR Future Work
22
![Page 23: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/23.jpg)
Perspective Transformation
23AVR Introduction AVR Design AVR Evaluation AVR Future Work
23
![Page 24: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/24.jpg)
AVR Contributions
• Accurate Positioning• Relative Positioning using 3D Feature Map
• Perspective Transformation
• Extremely Limited V2V bandwidth • Dynamic Object Extraction
• Adaptive Motion Vector Transmission
• Low Latency• Pipeline Optimization
• Motion Prediction to Hide Latency
24
24AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 25: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/25.jpg)
The Bandwidth
Data Size Bandwidth at 30fps
Full Point Cloud 4.91 MB 1178.4 Mbps
A Single Car 0.33 MB 79.2 Mbps
DSRC 802.11g
6 ~ 27 Mbps 6 ~ 54 Mbps
25AVR Introduction AVR Design AVR Evaluation AVR Future Work
25
![Page 26: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/26.jpg)
Dynamic Object Detection
C1
C2
26AVR Introduction AVR Design AVR Evaluation AVR Future Work
26
![Page 27: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/27.jpg)
Dynamic Object Detection
V C2+VC2
27AVR Introduction AVR Design AVR Evaluation AVR Future Work
27
![Page 28: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/28.jpg)
Estimating Absolute Motion
C2+VC1
Object Motion Vector
28AVR Introduction AVR Design AVR Evaluation AVR Future Work
28
![Page 29: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/29.jpg)
Isolating Dynamics from Statics
B2+V
C2+V
B1
C1
29AVR Introduction AVR Design AVR Evaluation AVR Future Work
29Static Object
Object Motion Vector
![Page 30: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/30.jpg)
Adaptive Transmission
30AVR Introduction AVR Design AVR Evaluation AVR Future Work
30
Transmitting Motion Vectors Adaptively based on Bandwidth Availability.
![Page 31: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/31.jpg)
AVR Contributions
• Accurate Positioning• Relative Positioning using 3D Feature Map
• Perspective Transformation
• Extremely Limited V2V bandwidth • Dynamic Object Extraction
• Adaptive Motion Vector Transmission
• Low Latency• Pipeline Optimization
• Motion Prediction to Hide Latency
31
31AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 32: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/32.jpg)
Pipeline Optimization
Decouple computation and reuse intermediate results to cut latency.Distribute computation evenly to 3 stages for high throughput.
32AVR Introduction AVR Design AVR Evaluation AVR Future Work
32
Stage 1: Preprocessing Stage 2: Localization Stage 3: Postprocessing
![Page 33: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/33.jpg)
AVR Evaluation
• The Benefits of AVR in ADAS and Autonomous Driving
• End-to-end Performance
• Reconstruction Accuracy
• Bandwidth / Accuracy Tradeoff
• The Impact of Delay
• Motion Estimation Accuracy
• Throughput And Latency
33
33AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 34: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/34.jpg)
AVR Evaluation
• The Benefits of AVR in ADAS and Autonomous Driving
• End-to-end Performance
• Reconstruction Accuracy
• Bandwidth / Accuracy Tradeoff
• The Impact of Delay
• Motion Estimation
• Throughput And Latency
34
34AVR Introduction AVR Design AVR Evaluation AVR Future Work
![Page 35: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/35.jpg)
AVR Setup
Alienware Laptop• I7 CPU, 4.4GHz• 16GB DDR4 RAM• Nvidia 1080p GPU, 2560 CUDA cores.
ZED Stereo Camera
TP-Link Talon 7200• 802.11 g/n/ac/ad• Wireless Distributed System (WDS)
35AVR Introduction AVR Design AVR Evaluation AVR Future Work
35
![Page 36: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/36.jpg)
AVR Setup
3636AVR Introduction AVR Design AVR Evaluation AVR Future Work
36
![Page 37: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/37.jpg)
AVR Setup
3737AVR Introduction AVR Design AVR Evaluation AVR Future Work
37
![Page 38: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/38.jpg)
AVR Setup
3838AVR Introduction AVR Design AVR Evaluation AVR Future Work
38
Building an extended vision by merging the leader’s view.
![Page 39: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/39.jpg)
Benefits in Autonomous Driving
3939AVR Introduction AVR Design AVR Evaluation AVR Future Work
39
Drivable Space (in blue)
Planned Path (in green) Follower can see oncoming vehicle (in red circle)
![Page 40: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/40.jpg)
Benefits in Autonomous Driving
AVR can double the drivable space detected.
4040AVR Introduction AVR Design AVR Evaluation AVR Future Work
40
![Page 41: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/41.jpg)
Benefits in Autonomous Driving
AVR can help path planner to avoid dangerous / wasteful lane change maneuver.
4141AVR Introduction AVR Design AVR Evaluation AVR Future Work
41
![Page 42: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/42.jpg)
End-to-end Result
• Full: sharing the point cloud of the whole scene.
• Dynamic: sharing only the dynamic part of the scene.
AVR reduces 10x of bandwidth requirement in dynamic mode. AVR adapts smoothly to available bandwidth.
MeanThroughput
(Mbps)Motion Vector
StreakMotion Vector /
Frame
Full 367.02 2.80 1.17
Dynamic 34.95 1 0.003
4242AVR Introduction AVR Design AVR Evaluation AVR Future Work
42
![Page 43: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/43.jpg)
End-to-end Result
AVR achieves 30 fps using a 3-stage pipeline AVR induces 96 ms processing delay
4343AVR Introduction AVR Design AVR Evaluation AVR Future Work
43
Preprocessing
Localization
Postprocessing
![Page 44: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/44.jpg)
Caveat: perfect calibration, more details in the paper.
Reconstruction Accuracy
Error @ 20mph Static Objects Moving Objects
Median 0.027 m 0.070 m
90th Percentile 0.045 m 0.193 m
AVR achieves high reconstruction fidelity.Major source of error: motion estimation, camera calibration.
4444AVR Introduction AVR Design AVR Evaluation AVR Future Work
44
![Page 45: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/45.jpg)
• Stereo Camera vs. LiDAR
• Accuracy
• Range
• Robustness to poor visibility
• Cooperative AVR
Limitation & Future Work
4545AVR Introduction AVR Design AVR Evaluation AVR Future Work
45
![Page 46: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/46.jpg)
AVR Demo
46
Augmented Vehicular Reality❑Breaks the LOS limit to see beyond occlusion
❑Extends 3D perception range
❑Inspires more informed driving decisions
![Page 47: Augmented Vehicular Reality - Microsoft...Augmented Vehicular Reality (AVR) Hang Qiu, Fawad Ahmad, Fan Bai, Marco Gruteser, Ramesh Govindan Reliable Autonomous Driving Human drivers](https://reader030.vdocuments.site/reader030/viewer/2022041109/5f0e73d57e708231d43f50fd/html5/thumbnails/47.jpg)
Thank You!Q&AQ&A
Q&A47
Code & Datasethttps://github.com/hangqiu/AVR16