•  Vision loss impedes the independent activities of the visually impaired •  Conventional aid device, such as white cane can not provide non-reactive navigation. •  Stereo camera based navigation system for the visually impaired is proposed to extend the range of activities of the visually impaired. •  The system consists of two main parts: SLAM system and Feedback system • SLAM system: Visual odometry + Feature based SLAM + Obstacle avoidance + Path planning • Feedback System: A vest-type interface consisting of four micro-vibration motors delivers cues for real-time navigation with obstacle avoidance.

Robust estimation in mapping and navigation

Cue to the user Traversability Map

Left Camera Images Right Camera Images

Abstract

Feature Extraction

/ KLT Tracking

RANSAC Motion Estimation

Rao Blackwellized Particle Filter

Camera Pose Estimation

Update Traversability

Map

Update Direction

1

5

2

4

3

System overview of a real time navigation system for the visually impaired using RGB-D Camera

Algorithm Flow Description 1.  FAST Corner Extraction + KLT Tracking 2.  Camera Motion Estimation using 3-Point Algorithm +

RANSAC and Sparse Bundle Adjustment 3.  Map building using RBPF 4.  Obstacle Avoidance using Vertical and Horizontal Patches

Way Point Generation (Furthest point in a certain area) 5.  Shortest Path Computation using D-Star Lite Algorithm. 6.  Cue Delivery to the Vest Interface

System and Algorithm Overview

Stereo Camera Based Navigation for the Visually Impaired Young Hoon Lee, Hyunsuk Ko, Sukjin Lee

-  Future works -  More accurate 3D reconstruction algorithm is

required -  Numerical analysis on performance of algorithm

Results

Vertical patch(Red): Obstacle Horizontal patch(Green): Free space

Traversability map Red: Obstacle/ Green: Free space

Current position Way Point

Shortest Path to the way point

Four snapshots of a stereo camera based navigation system processing.

1

2 3

4

5 6

t Top row: Gray scale converted RGB images Bottom row: Corresponding traversability map

GPU Stereo Matching

1.  Determine how to allocate threads - Eliminate redundant computation minimize kernel size

2.  Each thread sums the squared differences (SSD) between pixels in the reference image and the comparison image

3.  Sum the column SSD values from the neighboring columns within kernel radius

4.  Determine if the current disparity value is the best correspondence match

Image from left camera

Image from right camera