Advanced Robotics: Autonomous Mobile Robots

Arshad Jamal,

Scientist, Intelligent Systems and Robotics Division Centre for AI & Robotics, DRDO

Bengaluru

1. Motivation 2. History and Current Scenario 3. Technologies for Autonomous Mobile

Robots 4. Capability requirements 5. Robotic Systems developed by CAIR

a. Systems with different mobility b. Semi-autonomous mobile robots c. Autonomous mobile robots d. GPS-less autonomous navigation

i. SLAM algorithms ii. Autonomous search robot

Outline

Why Autonomous Robotics Systems?

1. First response, Surveillance and Reconnaissance, Patrolling

2. IED handling, UXO handling, Mine laying and breaching

3. Communication relays, Logistics transport 4. Convoy protection, Road clearance 5. Target Identification and Tracking, Remotely

operated weapons 6. Disaster management

History of Autonomous Vehicles 1977: Tsukuba Mechanical Engineering Lab in Japan creates the first autonomous,

intelligent, vehicle. It tracked white street markers and achieved speeds up to 30 km/h.

1987-1995: The pan-European Prometheus project, also known as the EUREKA

Prometheus Project, the largest autonomous vehicle project so far, is funded by the European Commission.

1997: Demo '97 in San Diego, California, in which about 20 automated vehicles,

including cars, buses, and trucks, were demonstrated 2000s: Several DARPA challenges made significant contributions in development of

autonomous vehicles. Google starts work on its driverless car in 2009 2010s: All major automobile companies are working on driverless autonomous

cars

Major Research Programme for Autonomous Vehicle

• DARPA Grand Challenge 2005 – Autonomous driving through 150 miles of

desert terrain in less than 10 hours – Winner: Stanford Racing Team (6:54 hrs),

Stanford University

• DARPA Urban Challenge 2007 – Autonomous driving through 60 miles in an

urban environment in less than 6 hours – Winner: Tartan Racing (4:10 hrs), Carnegie

Mellon University

• DARPA Robotics Challenge 2014 – Humanoid robots in disaster management

• Enter, drive and exit a vehicle, Clear obstacles and open a door, Climb a ladder, Turn off a valve, Attach a hose

Global Scenario - Military • Light weight man-portable to heavy duty systems

– Scouting operations, intelligence, surveillance and reconnaissance, IED handling, logistics support, mine clearance

Dragon runner Viper PackBot Talon

MULE Gaurdium Big Dog Panther

Technologies for Autonomous Mobile Robots

Mobility

Power Communication

Human-Robot Interaction

Localization

Perception Planning

Navigation

Learning/ Adaptation

Enabling Technologies

Autonomous Behaviour

1. Perception a. Multi-modal environment sensing and data fusion

• Vision (EO, IR), ranging (LIDAR, RADAR, SONAR), tactile (haptic) b. Highly influenced by operational environment and platform

embodiment 2. Knowledge representation

a. Ontologies • Describes entities and their relationships

b. Enables abstraction of concepts and inference c. Provides the bridge between perception and machine reasoning

3. Reasoning a. ‘What-If’ scenario modeling - Projection of actions and consequences

into the future in a given context b. Highly dependent on embodiment c. Enables high level behaviours

• Collaboration

Key Capabilities for Unmanned Systems

4. Planning a. Decomposition of a mission specification into specific tasks

• Considers available platforms and capabilities vis-à-vis mission requirements – non-trivial!

b. Requires semantic characterization of capabilities c. Requires reasoning to infer that a composition of capabilities can

meet a mission requirement d. Scheduling, monitoring and re-planning in context e. Key enabler for effective collaboration

5. Learning a. The defining principle of intelligence! b. Enables high-level behaviours

• improvisation, adaptation, cunning, strategy 6. Self-monitoring for effective immunity from external infections

a. Guard against internal corruption in the system b. Immunization against adversarial take-over c. Identify the right set of security requirements starting from policy to

architecture to models to mechanisms; more than just encryption!

Key Capabilities of Unmanned Systems

1. Consists of three repeated steps a. Sense your

environment b. Plan what to do next

by building a world model through sensor fusion and taking all goals (both short term and long term) into account

c. Execute the plan through actuators

Key Capabilities: Sense-Plan-Act Paradigm for Autonomous Systems

Sensory Inputs

Actuation

Intelligent Robotics Systems Developed at CAIR

Technology Focus Areas Mobility

Leg & Wheel Legged Serpentine Wall Climbing Tracked

Autonomous Navigation

Robot Sentry Wheeled Vehicle Tracked Vehicle

Flapping Quad-rotor

Manipulation

Hot slug handling, NMRL

Inspection of Composites, HAL

Steam generator inspection, NPCIL

Educational Manipulators arms

Mobile Manipulator

Perception

Change Detection Fusion Tracking SLAM

1. Suspension with linkage mechanism a. Six actuated wheels b. Parallel bogies in center c. Fork suspension in front d. Step climbing capability upto

1.5 times wheel diameter

2. Suspension with Spring-Damper a. Six actuated wheels b. Lower vibration c. Suspension in both roll and

pitch

Wheeled Locomotion- Passive Suspension

1. Requirements a. Stair climbing b. Self righting c. High ground clearance

2. Multi-segment tracked robot (MiniUGV) a. Main tracks in center b. Tracked flippers in

front and rear with endless rotation

c. Camera in front and rear

d. Remote operation with 160m NLOS range

e. 3 hrs of endurance f. 50 Kg payload

capacity on flat terrain

Tracked Locomotion

1. Hexapod a. Cockroach type

i. 6 legs with 2 DoF b. Crab type

i. 6 legs with 3 DoF ii. Joint level leg control (Angles)

c. Omni-hex i. 6 legs with 3 DoF ii. Cartesian level leg control (X, Y, Z) iii. Ultrasonic Sensors for obstacle

avoidance

Legged Locomotion

1. Quadruped a. No static stability b. Lateral movement of

body can shift the CG to stable location

c. 3-DoF Leg design with extra Hip joint

d. Gaits i. Crawl: one leg at a

time with body sway ii. Trot: diagonal legs

at a time at higher speed

Legged Locomotion

1. Design a. 2-DoF Joints with alternate

horizontal and vertical joints b. Free wheels to emulate differential

friction 2. Combination of travelling waves

along horizontal and vertical plane generate

a. Lateral undulation b. Caterpillar gait c. Side-winding gait d. Rolling gait

Serpentine Locomotion

Transverse wave propagates along the body Differential Friction results in forward motion

Wave travel

Body displacement

1) Low pressure adhesion 1. Impeller to generate vacuum for

sticking to wall a. Suction force large enough

to offset the robot weight b. Suction force Small enough

to enable locomotion 2. Differential tracks for motion 3. Camera on 2-DoF arm

2) Electro-adhesion based - Adhesion due to electrostatic

force - Noiseless operation - High endurance

Wall Climbing

Semi-autonomous Mobile Robot

1. Autonomous Navigation a. with a-priori map

2. Continuous Video Feed a. via Pan-Tilt-Zoom Network

Color Camera 3. GPS and Stabilized Digital

Compass a. for Localization

4. Scanning Laser Range Finder a. for Obstacle Avoidance

5. WiFi Link a. for Command & Control

Semi-Autonomous Navigation – Sentry Robot

Current Pos/Head Desired Pos Laser range data

Autonomous Mobile Robot

1. Unstructured environment 2. No predefined map is available 3. Only coarse waypoints may be available 4. Day and night operation 5. Modular, scalable and extensible hardware and

software architecture

Autonomous Tracked Platform - Scope

Hardware Architecture

• Drive-by-wire system interface

• Vehicle speed and heading controller

Steering actuation Throttle actuation

Topological Architecture

21-03-2016 24

Goal Specification

Vehicle Drive-by-Wire

Control

GPS

INS

Odometer

Localization Data Fusion

LOCALIZATION PERCEPTION

Monocular Camera

Stereo Camera LIDAR RADAR

Perception Data Fusion

Global path-planning

Local path-planning PATH PLANNING

Localization

50 Hz

Obstacle map

Occupancy map

Perception

10 Hz 15 Hz 4 Hz

5 Hz

Goal specification

Global path

planning

Local path planning

Path planning

0.5 Hz 10 Hz

Vehicle control

Navigation 20 Hz

Autonomous Navigation System

Goal specification

Global path

planning

Local path planning

Path planning

0.5 Hz 10 Hz

Localization

50 Hz

Obstacle map

Occupancy map

Perception

10 Hz 15 Hz 4 Hz

5 Hz

Vehicle control

Navigation 20 Hz

Sensor interface for data acquisition Vehicle pose (position and attitude) Vehicle velocities

Coordinate frame transforms for localizing sensor data with respect to the vehicle

Localization

Goal specification

Global path

planning

Local path planning

Path planning

0.5 Hz 10 Hz

Localization

50 Hz

Obstacle map

Occupancy map

Perception

10 Hz 15 Hz 4 Hz

5 Hz

Vehicle control

Navigation 20 Hz

Generation of 3D point cloud from LIDAR data Generation of occupancy map from 3D point cloud Fusion of occupancy maps to create obstacle map

Perception

Range data

Intrinsic parameters

Point Cloud (in sensor frame)

Point Cloud (in vehicle frame)

Extrinsic parameters

Point Cloud (in world frame)

Vehicle pose

New occupancy map Existing occupancy map Fused occupancy map (log odds formulation)

Fused occupancy map (weighted sum formulation)

Obstacle map (thresholded occupancy map)

Occupancy map

Occupancy map

Path planning

Range data Point Cloud (in vehicle frame)

Extrinsic parameters

Point Cloud (in world frame)

Vehicle pose

New occupancy map Existing occupancy map Fused occupancy map (log odds formulation)

Nodder pose

+ + ∫

Goal specification

Global path

planning

Local path planning

Path planning

0.5 Hz 10 Hz

Localization

50 Hz

Obstacle map

Occupancy map

Perception

10 Hz 15 Hz 4 Hz

5 Hz

Vehicle control

Navigation 20 Hz

Integration of ARA* search-based lattice planner for global path planning

Integration of Trajectory Rollout for local path planning

Path Planning

Obstacle map Inflated Obstacle map Global plan

Local plan

Goal specification

Motion primitives

Newtonian mechanics model

Desired vehicle speed and turning Vehicle control

Vehicle pose

Path Planning

Obstacle

Vehicle position and heading

Occupancy map

Path Planning

Goal pose

Vehicle pose

Obstacle inflation

Path Planning

Replanning

Localization

50 Hz

Obstacle map

Occupancy map

Perception

10 Hz 15 Hz 100 Hz

5 Hz

Goal specification

Global path

planning

Local path planning

Path planning

0.5 Hz 10 Hz

Vehicle control

Navigation 20 Hz

Implementation of entire workflow using open-source framework ROS 12 nodes configurable to run

across multiple computation units

Configurability of key parameters for adaptability to different platforms Vehicle dimensions Motion model Map dimensions and resolution Obstacle definitions Sensor mounting configuration Sensor interface configuration

Software Architecture

1. Web-based a. Can be run from any laptop

connected to vehicle WiFi 2. Configuration of all software

modules and parameters 3. Selection of modules and

customization of parameters for a particular run

4. Selective power-on of systems and sensors

5. Live feed of vehicle position and image data, if available

GUI

• Autonomous navigation • With online cost-map generation • In unstructured terrain

• Drive-by-Wire conversion of a COTS tracked vehicle Terex ST50

• Localization system • High accuracy integrated GPS-INS

• Perception system comprising – 3D LIDAR for long range mapping – 2D LIDAR with nodding for short

range mapping – Color camera for feedback

• Onboard embedded computational platform comprising

– Networked box PCs • Wireless link for remote

monitoring

Autonomous Navigation – Tracked Vehicle

Autonomous Navigation – BMP

Towards GPS-less Autonomous Navigation

Localization Where am I? The problem of estimating the pose (position &

orientation) of the robot relative to a map. Mapping

What does the world look like? Mapping is the problem of integrating the information

gathered with the robot's sensors into a given representation.

Simultaneous Localization And Mapping Defined as the problem of building a map while at the

same time localizing the robot within that map.

Simultaneous Localization And Mapping

Figure: A classification of various SLAM algorithms

Classification of SLAM algorithms

1. Kalman Filter and variants – EKF, UKF based feature based SLAM

2. Particle Filter (PF) based – Rao Blackwelliesd Particle Filter based

3. Information Filter based – Sparse extended information filters

Filtering Based SLAM Approaches

• Particle Filter based Algorithm – GMapping – Rao-

Blackwellised PF based algorithm.

Filtering based SLAM: Algorithm Choice

Video: Sentry Indoor Navigation

• ORB-SLAM – Output in CAIR Outdoor campus – Output in CAIR indoor campus

• LSD-SLAM – Output in CAIR outdoor campus – Output in CAIR indoor campus

Monocular SLAM

RGBD SLAM

RGB images Frame to

frame motion estimation

Robot Location

Depth images

Sensor processing module for computation of visual odometry is aided with observations from 2D LiDAR

RGBD sensor data

To “Build Pose Graph”

2D LiDAR

RGBD SLAM

Pose Graph

Bag of Words loop closures

Spatial proximity loop closures

View Point based loop closures

Frame Matching

Loop Closures

Loop closure detection is driven by Bag-of-words approach.

Yes

No

RGBD SLAM Results

Video: RGBD SLAM in System Block, CAIR

Simulation video of Multi-robot exploration and mapping [speed-up 12x]

Simulation Video

1. Two robots interacting with each using ROS Multi-master a. Both the robot are controlled by the same

ROS console. b. Share of data between the robots and the

operator console e.g. online created maps from both robots.

c. Display of robot data on the operator console.

d. Autonomous navigation by the robots. e. Video: Demonstration of ROS Multi-master

Two robot communicating using Multi-master

Technologies

at work

Artificial Intelligence

Perception

Robotics

1. Environment: Unknown. Urban indoor terrain.

i. User asks the robot to search for a particular color or object in the environment.

ii. Robot enters the space, searches for objects and sends them back to user’s device.

iii. Map of explored areas and its current position is continuously updated on the user screen.

iv. Robot returns to its initial position after the search operation is completed.

Autonomous search robot

1. Exploration Algorithm a. Finds new frontiers as goals, monitors updated map for new

frontiers 2. Path Planning

a. Plans a minimum cost path to the goal 3. Obstacle Avoidance

a. Avoids obstacles in the path and steers robot away 4. Simultaneous Localization and Mapping

a. Creates map of environment and finds its own location 5. Object detection

a. Detects user desired objects, captures images and marks its location in the environment map

6. Visualization and User Interface a. Renders the environment and displays the detected objects

interactively

Autonomous Search Robot

Results for Autonomous Search Robot

Visual Perception Technologies

Visual Perception Technologies

Camera Calibration

Feature Detection and matching

Sparse 3D estimation

Time-to-collision

Motion estimation

Stereo Vision

Dense 3D estimation

Visual Odometry and

SLAM

Object, Place, Scene

Saliency Analysis

1. Significant advances in every aspect of the field

2. CAIR has developed many advanced robotics system

3. Currently working on Multi-Agent Robotics System to manage scale of the problem

4. Long Term Autonomy and Never Ending Learning paradigms are needed

5. Huge potential for collaboration between DRDO, Industry and Academia

Conclusion

Thank You

arshad@cair.drdo.in