(reading group) first results in detecting and avoiding frontal obstacles from a monocular camera...
TRANSCRIPT
![Page 1: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/1.jpg)
First Results in Detecting and Avoiding Frontal Obstacles
from a Monocular Camera for Micro Unmanned Aerial
Vehicles
WakaWaka Group : H.Kidane , I.Sadek , and
M.Elawady
8/18/2014 1
Supervisor : Prof. Yvan Petillot
Robot Project
Tomoyuki Mori and Sebastian Scherer
![Page 2: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/2.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 2
![Page 3: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/3.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 3
![Page 4: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/4.jpg)
Introduction
• Limited payload to carry
additional sensors
• Depended on monocular
camera
• Obstacles can’t be observed
directly using this camera
8/18/2014 B31XP Robotics Project 4
Problem Definition
![Page 5: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/5.jpg)
Introduction
8/18/2014 B31XP Robotics Project 5
Objective
Detecting and avoiding frontal obstacles by utilizing the size change
![Page 6: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/6.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 6
![Page 7: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/7.jpg)
Related Work
8/18/2014 B31XP Robotics Project 7
Motion Parallax
Monocular Cues
Stereovision
![Page 8: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/8.jpg)
Related Work
8/18/2014 B31XP Robotics Project 8
Motion Parallax
Monocular Cues
Stereovision
![Page 9: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/9.jpg)
Related WorkMotion Parallax
8/18/2014 B31XP Robotics Project 9
Near objects move very quickly. However , distant objects move much more slowly
http://www.garyfisk.com/anim/mparallax.swf
Motion Availability
Optical Flow/Structure from Motion- Large Computation.
- Cannot detect obstacles straight a head
![Page 10: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/10.jpg)
Related Work
8/18/2014 B31XP Robotics Project 10
Motion Parallax
Monocular Cues
Stereovision
![Page 11: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/11.jpg)
Related Work
8/18/2014 B31XP Robotics Project 11
Monocular Cues
Monocular Cues provide depth information when viewing a scene with one eye (Wiki )
Method Availability
Perspective- Long lines detection
- Well strutted environments (In door)
Relative Size
- Roughly similar objects (The larger the
object the closer to the observer)
- Available for frontal obstacle avoidance
Known Object Size- Features are required for particular objects
Texture Gradient
Depth From Focus - Not applicable for small aperture cameras
![Page 12: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/12.jpg)
Related Work
8/18/2014 B31XP Robotics Project 12
Motion Parallax
Monocular Cues
Stereovision
![Page 13: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/13.jpg)
Related Work
8/18/2014 B31XP Robotics Project 13
StereovisionStereovision is to extract 3D information from digital images (Wiki )
Method Availability
Stereoscopic
parallax- Needs a sufficient baseline
Convergence
-The convergence is the angle formed by your
eyes and the observed object
- Available only for human
![Page 14: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/14.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 14
![Page 15: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/15.jpg)
8/18/2014 B31XP Robotics Project 16
ApproachFrontal Obstacle Detection
Depth
Relative
Size
Natural Environment
![Page 16: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/16.jpg)
8/18/2014 B31XP Robotics Project 17
ApproachFrontal Obstacle Detection
After Few Frames
Detect “Relative Size”
![Page 17: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/17.jpg)
8/18/2014 B31XP Robotics Project 18
ApproachFrontal Obstacle Detection
Algorithm 1
Scale Expansion
Detector
Current
Image
Previous
Image
Algorithm 2
Template
Matching
Position of
Obstacle
![Page 18: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/18.jpg)
8/18/2014 B31XP Robotics Project 19
ApproachFrontal Obstacle Detection
Algorithm 1 : Scale Expansion DetectorSURF/SIFTComputation
Scale
Changes
![Page 19: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/19.jpg)
8/18/2014 B31XP Robotics Project 20
ApproachFrontal Obstacle Detection
Algorithm 2 : Template Matching
SURF Scale VS
Correlation Distance
![Page 20: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/20.jpg)
8/18/2014 B31XP Robotics Project 21
• Karaman and Frazzoli
– Used position distribution to model the tree location
• For this kind of forest only the closest obstacle is really relevant
This motivate use of reactive obstacle avoidance law
ApproachReactive Obstacle Avoidance
for Forest Flight
![Page 21: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/21.jpg)
8/18/2014 B31XP Robotics Project 22
J. J. Gibson : animals detect “visual collision” with looming and do not detect the distance
Likewise the vehicle avoids frontal obstacle when it detects that the time to collision is too small
ApproachReactive Obstacle Avoidance
for Forest Flight
![Page 22: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/22.jpg)
8/18/2014 B31XP Robotics Project 23
ApproachReactive Obstacle Avoidance
for Forest Flight
Frew et al: proposed a trajectory generator for a small
UAV to fly in forest
But as the Ar.Drone is small and it has to react quickly,
path planning is not used.
![Page 23: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/23.jpg)
8/18/2014 B31XP Robotics Project 24
ApproachReactive Obstacle Avoidance
for Forest Flight
• Information the Arial vehicle has to know:• Bearing of the closet tree
• Goal position
• It’s own position
• Control command to the UAV:• velocity
• Approach of control command:• Fly sideways when obstacle found in field of view
• otherwise control yaw angle to achieve the goal bearing
![Page 24: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/24.jpg)
8/18/2014 B31XP Robotics Project 25
ApproachReactive Obstacle Avoidance
for Forest Flight
![Page 25: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/25.jpg)
8/18/2014 B31XP Robotics Project 26
ApproachReactive Obstacle Avoidance
for Forest Flight
Parameters required to determine feasibility of flight:
• Response time
• Acceleration limit
• Flying speed
• Obstacle sensing distance
• Field of view
• Trunk size
• Minimum distance between trees
![Page 26: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/26.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 27
![Page 27: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/27.jpg)
8/18/2014 B31XP Robotics Project 28
Experiment setup
• Parrot Ar.Drone • Simulation software
• Intel [email protected]
running Linux (Laptop)
• Frontal Camera (FOV 92°, Res. 320x240, 10Hz)
• Sonar height sensor, Inertial measurement unit, down camera
![Page 28: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/28.jpg)
8/18/2014 B31XP Robotics Project 29
Experiment
![Page 29: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/29.jpg)
8/18/2014 B31XP Robotics Project 30
Experiment Result
• Failures are due to slow response time
![Page 30: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/30.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 31
![Page 31: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/31.jpg)
8/18/2014 B31XP Robotics Project 32
Conclusion
• They develop scale expansion detector to detect the approaching object using monocular vision
• They use SURF features to extract the expanding key points
• Obstacles have to have sufficient texture to make SURF key points
• The performance of this approach can be improved using better vehicle platform which has fast response time
![Page 32: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/32.jpg)
Outlines
• Introduction
• Related Work
• Approach–Detection
–Avoidance
• Experiments
• Conclusion
• Future Work
8/18/2014 B31XP Robotics Project 33
![Page 33: (Reading Group) First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles](https://reader031.vdocuments.site/reader031/viewer/2022032214/55a396021a28aba37a8b4641/html5/thumbnails/33.jpg)
8/18/2014 B31XP Robotics Project 34
Future Work
Combining multiple detection algorithms:
– Optical flow
– Perspective cues
Which are effective in:
– Textured natural environments
– Homogeneous urban environments