silvina rybnikov supervisors: prof. ilan shimshoni and prof. ehud rivlin homepage:...

Download Silvina Rybnikov Supervisors: Prof. Ilan Shimshoni and Prof. Ehud Rivlin HomePage:

Post on 17-Dec-2015




1 download

Embed Size (px)


  • Slide 1
  • Silvina Rybnikov Supervisors: Prof. Ilan Shimshoni and Prof. Ehud Rivlin HomePage:
  • Slide 2
  • Introduction Problem statement Existing research Thesis goals The robot Algorithm outline Navigation to a single target Single image cover Room estimation Room coverage Improving positioning Algorithm Experiment
  • Slide 3
  • AGVs can be used in: Industrial applications to move materials around a manufacturing facility or a warehouse. Storage Delivery Transport medicine in hospitals Transporting containers in ports
  • Slide 4
  • Easy navigation from an unknown position to predefined targets. No human intervention. No artificial beacons/landmarks.
  • Slide 5
  • Reliable representation of targets Minimum probability of misidentification. Robust to small changes in the environment. A good solution is an image taken from the target pose Autonomous environment exploration Find all targets. A representation of targets relative positions Support easy navigation to a required target from an unknown position.
  • Slide 6
  • SLAM (Simultaneous Localization And Map Building) Collect all the features into a map of the explored environment. Next week in pixel club : "Visual Simultaneous Localization and Mapping using Inverse Scaling Parametrization and Uncertain Projective Geometry", Dr. Davide Migliore Path Following A human operator steers the robot on a path/paths while capturing images after each small step. The robot can repeat the same path or reach a goal near the graph of paths.
  • Slide 7
  • Build a robotic system with an efficient algorithm for targets search. Find ways to improve positioning accuracy. Implement the entire robotic system which: explores the environment locates the targets builds a representation of targets relative position enables navigation from any position to any target
  • Slide 8
  • Robot equipped with a single camera The camera is stationary Robot moves only on a plane (X-Z plane)
  • Slide 9
  • Input : Images taken from target positions Stage1: Locate all the target poses Estimate the environment Cover the environment while matching the targets Stage2: Build the graph of targets By finding paths between each pair of targets With accurate positioning Application: Navigation from unknown position to one of the targets Using the graph
  • Slide 10
  • Algorithm for reaching a position described by an image Appears in Image-Based Robot Navigation in Unknown Indoor Environments by E. Rivlin, I.Shimshoni, and E. Smolyar. Algorithm has two stages called iteratively: Finding direction to the target Finding the distance to the target
  • Slide 11
  • Given two images which partially overlap we can find : Direction between the cameras centers Rotation between the cameras view directions
  • Slide 12
  • Slide 13
  • Fundamental/Essential matrix Describes the relation between two images Gives for each point on one image the line on which it lies on the other image. E is 3X3 matrix For each point correspondence E holds: p 2 T Ep 1 = 0 The points are in normalized image coordinates.
  • Slide 14
  • In planar motion there are three unknowns: t x t z The essential matrix is of the form:
  • Slide 15
  • A linear solution requires 3 point correspondences. We use the linear solution as a starting point for a non linear optimization with two unknowns : t x /t z Since there are outliers in the SIFT matching process we use PROSAC to select triplets for the Essential matrix calculation. PROSAC (Progressive Sample Consensus) is a variant of RANSAC (Random Sample Consensus)
  • Slide 16
  • Why cant we find distance from two images ? Solution: take another image at a known distance
  • Slide 17
  • The robot moves in direction t x /t z a step of length . Features are matched in the three images The distance remaining to the target can be calculated from each triplet of correspondences after removing the difference in rotations.
  • Slide 18
  • For this calculation the images are aligned, using the calculated rotations with small changes. Choose pair of rotations which minimize following error expression: The distance is estimated as
  • Slide 19
  • Introduction Problem statement Existing research Thesis goals The robot Algorithm outline Navigation to a single target Single image cover Room estimation Room coverage Improving positioning Algorithm Experiment
  • Slide 20
  • Configuration space of the robot is the set of all possible positions (x,z) and orientations . How much of the configuration space a single image covers ? The covered space is the positions and orientations in which an image taken will match the current image. We have estimated the covered space experimentally.
  • Slide 21
  • The size of the covered space depends on the distance from the viewed scene, we claim that the dependence is linear. The resulting shape can be resized according to the distance from the scene. The coverage has three scores : 1. No - very low probability of a match 2. Maybe - some probability of a match 3. Yes - very high probability of a match
  • Slide 22
  • 0 degrees20 degrees40 degrees
  • Slide 23
  • Why ? 1.The space for coverage. 2.The size of a SIC of an image taken at a specific pose. How ? 12 stereo pairs In each pair SIFT features are matched (along the epipolar line) and their 3D position recovered.
  • Slide 24
  • Slide 25
  • The features above the floor and below 2 meters are considered as obsticles. The walls are estimated from those features.
  • Slide 26
  • We cover the room while trying to match the target images. Configuration space represented by three- dimensional grid with: Spatial resolution of 10cm Angular resolution of 5 . Goal : To locate all the targets in a minimum time Approach: Greedy, with minimum fragmentation
  • Slide 27
  • Overhead of traveling grater than of taking images When a location is reached cover all directions (12 images).
  • Slide 28
  • Minimize fragmentation by encouraging overlap in the Maybe portions of the SICs. Two overlapping Maybe scores considered as a Yes score. Score for position : the additional Yes volume added by the 12 SICs.
  • Slide 29
  • Precompute SICs at different scales and orientations Greedy approach (no lookahead) Score computations only for points on a random greed. Out of the possible 6 angle offsets only two are checked: 0 and 15
  • Slide 30
  • Errors in odometry propagate and increase. We choose a reference pose during the room estimation stage. The pose is saved as an image taken from it The robot returns after each exploration to the reference pose. We use EKF (Extended Kalman Filter) module the current pose covariance Improve pose estimation with two kinds of observations
  • Slide 31
  • The Kalman filter is a recursive estimator. Only the estimated state from the previous time step and the current measurement are needed to compute the estimate for the current state. The Kalman filter has two distinct phases: Predict and Update. Predict: Make a step from the previous estimated state. Update: Use observations to update the predicted state.
  • Slide 32
  • The robot rotates radians and travels d cm. is the orientation after the rotation by . G - Jacobian matrix of the partial derivatives of the update function of P with respect to P V - Jacobian matrix of the partial derivatives of the update function of P with respect to (d, ) M- the covariance of the step
  • Slide 33
  • Wh en a target image is first matched, its matching image J is saved. P J and J are also saved. From then on the covariance is calculated relative to P J. When the target pose is reached, the pose P T is improved by finding the angle between the current image and image J.
  • Slide 34
  • The path back from the target to the reference pose is used as an observation to improve the estimate of the target pose.
  • Slide 35
  • Estimate room Locate the targets The search is driven by the goal to cover the search space with a minimum number of movements and images. After each exploration the robot returns to the reference pose. Connect all the pairs of targets by an edge In case of failure to go between two targets return to reference pose. Edge estimated from both the way forth and back Estimates are combined using EKF (observation II)
  • Slide 36
  • Navigation from unknown position Robot receives an identifier of a target to go to Robot rotates on a spot till one of the targets in the graph is matched If no target matched makes a random move and repeats the previous step Finds the path with the lowest cost on the graph from the current node to the requested node. The cost is the sum of the uncertainties on the edges. Follows the path
  • Slide 37
  • experiment1.mp4 experiment1.mp4
  • Slide 38
  • Slide 39
  • The Graph: ed