AUTONOMOUS AND SEMI-AUTONOMOUS ROBOTS FOR RECUE APPLICATIONS

Prof. Dr. Karsten Berns

Robotics Research Lab

Department of Computer Science

University of Kaiserslautern

RRLAB RESEARCH

ICARUS: UNMANNED SEARCH & RESCUE 2

Wh

eele

d S

ervi

ce R

ob

ots

Autonomous Mobile Robots

Methodology and Algorithms

Hu

man

oid

Ro

bo

ts

Off

road

Ser

vice

Ro

bo

ts

OFFROAD SERVICE ROBOTS

ICARUS: UNMANNED SEARCH & RESCUE 3

Construction Machines Agricultural Vehicles Test Vehicles

PROBLEM STATEMENT

• Disasters disrupt our society and are very difficult to manage • Current Search & Rescue actions are labour-intensive and slow

Source: B-FAST

ICARUS: UNMANNED SEARCH & RESCUE 4

G. De Cubber, D. Doroftei, Y. Baudoin, D. Serrano, K. Berns, C. Armbrust, K. Chintamani, R. Sabino, S. Ourevitch Search and Rescue robots developed by the European ICARUS project. Proceedings of the I7th IARP RISE-ER 2013)

ICARUS - CONSORTIUM

Integrated Components for Assisted Rescue and Unmanned Search operations • Participants:

• 24 partners • 10 countries • 2 end-users:

• B-FAST • Portuguese Navy

• 3 large industrials • NATO / NURC

• Total Budget: 17.5 M€

Source: Wikimedia Commons

5

OBJECTIVES

8 OBJECTIVES

ICARUS: UNMANNED SEARCH & RESCUE 6

Shashank Govindaraj, et al.. The ICARUS Project - Command, Control and Intelligence (C2I). Proceedings of the 11th IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR 2013) - IEEE RAS . Oct. 21-26, 2013 - Linköping, Sweden

ICARUS – OBJECTIVE 1

ICARUS: UNMANNED SEARCH & RESCUE 7

Development of a light sensor capable of detecting humans

• Based on QCD technology • Minimal levels of weight (500 g),

dimensions (12x12x6 cm),total power consumption (5 W)

• Image and video processing algorithms for detecting human survivors with high detection performance Source:

RMA

ICARUS – OBJECTIVE 2

Development of cooperative Unmanned Aerial System (UAS) tools for unmanned SAR

• Mapping of terrain • Situation assessment (People

endangered/injured?) • People search outdoors and indoors • Kit delivery • Communication relay (base station <-> SAR

teams)

ICARUS: UNMANNED SEARCH & RESCUE 8

ICARUS – OBJECTIVE 3

Development of cooperative Unmanned Ground Vehicle (UGV) tools for unmanned SAR

• Development of a large UGV as a mobile base • Development of a small UGV for entering in

collapsed buildings searching for victims

Main Objectives • Development and realisation of mechatronics

of LUGV and SUGV • Development and realisation of control

system of LUGV and SUGV • Development and realisation of user

interfaces for LUGV and SUGV

ICARUS: UNMANNED SEARCH & RESCUE 9

ICARUS – OBJECTIVE 4

Development of cooperative Unmanned Surface Vehicle (USV) tools for unmanned SAR

• Survivors detection and tracking. • Mission planning and control

for operations with single or multiple vehicles.

• Capsule deployment system (life-rafts)

ICARUS: UNMANNED SEARCH & RESCUE 10

ICARUS – OBJECTIVE 5

Heterogeneous robot collaboration between Unmanned Search and Rescue devices

• Robot Interoperability • Coordination between robots • Heterogeneous operations UAS + UGV/USV in a SAR

context

ICARUS: UNMANNED SEARCH & RESCUE 11

ICARUS – OBJECTIVE 6

Development of a self-organising cognitive wireless communication network, ensuring network interoperability

• Ad-Hoc Network • Self-managed network with failure-resilient

protocols • Flexible security scheme • Able to encompass several data-link

technologies (WLAN, GSM)

ICARUS: UNMANNED SEARCH & RESCUE 12

ICARUS – OBJECTIVE 7

Integration of Unmanned Search and Rescue tools in the C4I systems of the Human Search And Rescue forces

• Collection of data/information • Merging of data from different sources,

including GIS information; • Monitoring and control interfaces

ICARUS: UNMANNED SEARCH & RESCUE 13

ICARUS – OBJECTIVE 8

Development of a training and support system of the developed Unmanned Search And Rescue for the Human Search And Rescue teams

• Targeting at end-users (i.e. first responders) • PC simulators • E-learning

ICARUS: UNMANNED SEARCH & RESCUE 14

ICRAUS – REQUIREMENTS AND OBJECTIVES

ICARUS: UNMANNED SEARCH & RESCUE 15

LARGE UNMANNED GROUND VEHICLE (LUGV)

• Semi-Autonomous • Rough-terrain capable • Mobile Sensor Platform • Powerful Manipulator • Transport platform for small UGV • Tasks:

• Debris removal • Route Clearing • Shoring • Situation Awareness

ICARUS: UNMANNED SEARCH & RESCUE 16

SMALL UNMANNED GROUND VEHICLE (SUGV)

• Digital Vanguard Base • Light Manipulator Arm • Sensors for HAZMAT detection • (IR) Camera for victim detection • Stair-climbing ability • Semi-autonomous capabilities • Tasks:

• Indoor victim search • Indoor scene assessment

ICARUS: UNMANNED SEARCH & RESCUE 17

TEST PLATFORM

Setup of test platforms (off-road robot, EOD platform, UAV)

Test platform RAVON (RRLab)

Test platform tEODor (RMA, Brussels, Belgium)

18 ICARUS: UNMANNED SEARCH & RESCUE

SENSOR SYSTEMS FOR UGVS

Main Sensors of LUGV 2x 2D Laser Range Finder

1x 3D Laser Range Finder

1x Inertial Measurement Unit

1x GPS Receiver

2x Stereo Vision System

Main Sensors of SUGV

1x Inertial Measurement Unit

2x ASUS Xtion PRO (or ToF sensors if

problems with sunlight occur)

1x Human Detection Sensor from WP210

3x Camera for Tele-Operation

(built-in)

ICARUS: UNMANNED SEARCH & RESCUE 19

TERRAIN ANALYSIS OF UGVS

Use of stereo camera data for terrain traversability analysis

Stereo-based Reconstruction

ToF-based Reconstruction

ICARUS: UNMANNED SEARCH & RESCUE 20

SIMULATION FOR UGVS

ICARUS: UNMANNED SEARCH & RESCUE 21

Jens Wettach, Daniel Schmidt, Karsten Berns . Simulating Vehicle Kinematics with SimVis3D and Newton. 2nd International Conference on Simulation, Modeling and Programming for Autonomous Robots . November 15-18, 2010 - Darmstadt, Germany

RAVON: FRONT SENSOR SYSTEMS

Long-Range Color Stereo System Large Scale Terrain

Traversability Estimation

Short-Range Color Stereo System Obstacle Detection

Rotating 2D Laser Scanner Obstacle Detection /

3D Local Memory

Planar 2D Laser Scanner Obstacle Detection / Safety System

Spring-Mounted Bumper Tactile Vegetation Discrimination

ICARUS: UNMANNED SEARCH & RESCUE 22

CONTROL ARCHITECTURE

• Layered, hybrid, and modular control system architecture

• Implemented using MCA2-KL (modular framework for robot control systems)

• Realised using iB2C (integrated behaviour- based control architec- ture)

ICARUS: UNMANNED SEARCH & RESCUE 23

Bernd-Helge Schäfer, Christopher Armbrust, Tobias Föhst, Karsten Berns The Application of Design Schemata in Off-Road Robotics Intelligent Transportation Systems Magazine, IEEE - 4-27 , 2013 Martin Proetzsch . Development Process for Complex Behavior-Based Robot Control Systems Verlag Dr. Hut - {ISBN}: 978-3-86853-626-3 - 2010

24

3D SENSOR PROCESSING

Main sensor: panning laser range finder • Yields position, shape, and type of hazards in 3D space, e.g.:

ICARUS: UNMANNED SEARCH & RESCUE

Solid Object Penetrable Object

• Penetration technique for evaluating an obstacle’s rigidity ( Rock or grass?)

h

d h h

d

Positive Obstacle Negative Obstacle Overhanging Obstacle Water

beneath ground

ground

near ground

far ground

sky

shortest ground

distance

Y

X

Extract ground representatives (blue)

From ground level classify:

Pts beneath ground

Associated ground pts (purple) represent the load-bearing surface

Near ground pts (surmountable)

Far ground pts (positive or overhanging obstacles)

Sky points

Evaluation of Individual Scans

25

Grid Map as Local Obstacle Memory

Robot centric, orientation-fixed local grid map

Scrolls in x- and y-direction

Serves as local obstacle memory

Robot “remembers” obstacles

Collisions with obstacles not in direct sensor range can be avoided

26

SENSOR DATA REPRESENTATION

• Data of different sensor systems is entered into grid maps

• Grid maps used as “local short-term obstacle memory” Important as sensor systems cannot cover complete area around robot

• Sector maps as abstract and uniform data representation

• Extracted from grid map

• Each sector contains information about area it covers

Cartesian Sector Maps Polar Sector Maps

Grid Maps

Extraction

ICARUS: UNMANNED SEARCH & RESCUE 27

SECTOR MAPS

ICARUS: UNMANNED SEARCH & RESCUE 28

29

BEHAVIOUR-BASED CONTROL

ICARUS: UNMANNED SEARCH & RESCUE

Behavioural Group for Fast Driving

Emergency Stop

Drive Commands

Point Access

Behavioural Group for Moderate Driving

Behavioural Group for Tactile Creep

State Switching and Behavioural Group Stimulation

Stimulation

Drive Commands

Drive Commands

Fusion

Drive Commands

Human Operator

Bernd-Helge Schäfer . Control System Design Schemata and their Application in Off-road Robotics. Verlag Dr. Hut - {ISBN}: 978-3-86853-865-6 – 2011

Tim Braun . Cost-Efficient Global Robot Navigation in Rugged Off-Road Terrain. Verlag Dr. Hut - {ISBN}: 978-3-86853-135-0 - 2009

Offroad Path Detection

• Not only look for free traversable space

• Find structures that help reaching the horizon

30

31

SENSOR SYSTEM OF ICARUS LUGV

ICARUS: UNMANNED SEARCH & RESCUE

ICARUS LUGV

32

LAND DEMONSTRATION

• Demonstration at end of project

• Scenario:

– Earthquake strikes fictitious country

– First responders equipped with ICARUS components are sent out to help

ICARUS: UNMANNED SEARCH & RESCUE

Demonstration Site at Marche-en-Famenne, Belgium

KARSTEN BERNS

Robotics Research Lab

Department of Computer Science

University of Kaiserslautern