Driver Assistance and Chassis Systems - Group Research and Advanced Engineering

Autonomous Driving to Advance Road Safety and Comfort

The S500 INTELLIGENT DRIVE

Ralf Herrtwich

Project Goals

Highly autonomous driving on the original Bertha Benz Route from Mannheim to Pforzheim 125 years after the world's first overland drive • with as few driver interventions as possible • floating within regular traffic • with no special guidance in front • with no alterations of given infrastructure

Goals: • Gain experience with autonomous driving beyond highways • Experiment with a vehicle setup with regular production or

close-to-production sensors (only more) • Cameras (mono and stereo) • Radars

• Identify technical issues and examine different solutions for future autonomous functions

• Prepare for innovation beyond 2020 • Push the envelope

Why We Wanted to Do It

Mercedes-Benz Intelligent Drive

Accident-free driving Autonomous driving

Prevent and avoid accidents altogether or at least mitigate their effects

to an amount of minimum harm and damage

Assist drivers with maneuvers if they want it, but only when and where it is technically possible

without taking imponderable risks

Safety Comfort

Key to the Mercedes-Benz Brand

Example: PRE-SAFE BRAKE

Example: DISTRONIC PLUS

Autonomous Driving in the New S- and E-Class

Mercedes-Benz Intelligent Drive Stop & Go Pilot: • Vehicle moves autonomously

at low speeds in traffic jams • Longitudinal and lateral control

incl. lane keeping in curves • Lane mark detection • Observation of vehicle ahead

(swarm mode) • Intelligent hands-on detection

• Driver is called back into the loop when needed

• Every 10-15 seconds as soon as vehicle drives 30 km/h or more

Autonomous?

αὐτο "self" + νόμος "law" Autonomy (as seen by Immanuel Kant): The capacity of an individual or entity to make an informed, un-coerced decision within principles of reason.

Rules of behaviour

Observations

Actions

Stages of Autonomy (NHTSA Logic)

Single control functions such as speed selection, braking or lane

keeping are automated

Level 1 Function-specific

automation More than one control function is automated

Driver expected to be available for control

at all times and on short notice

Level 2 Combined function

automation Vehicle takes control

most of the time

Driver expected to be available for

occasional control with comfortable transition times

Level 3 Limited self-driving

automation

Driver in charge

Level 0 No automation

Vehicle takes control all of the time

Driver not expected to be available for

control at any time

Level 4 Full self-driving

automation From here:

To here:

Complexity of Automation

Low ego velocity High ego velocity

Structured traffic

environment

Chaotic traffic

environment

Traffic Jam

Parking

Highway

Off-Highway

Step1

Step 4 Step 2

Step 3

Bertha Benz Drive

Complexity of Automation

Highway Off-highway

Lanes Wide Narrow and not necessarily exclusive

Direction Same direction for all Vehicles from and in all directions

Maneuvers Lane Change

• Passing of in-road obstacles (e.g. parked cars) incl. coordination with oncoming traffic

• Intersection navigation incl. turning • Roundabouts

Traffic signs Speed limits Traffic lights and priority rules

Road users Vehicles only Pedestrians and cyclists all around

Field of view Wide Obstructed

Where We Did It

The Historic Bertha Benz Route

In August 1888, Bertha Benz took her husband's Patentmotorwagen at dawn and, together with her two sons, drove to Pforzheim to visit her family.

She arrived shortly before midnight after what later became known as the world's first overland drive.

Route Difficulty (Original Estimation)

N Mannheim

Pforzheim

Route Difficulty (Actual Difficulty)

N Mannheim

Pforzheim

Ladenburg Passage

Pfinztal Passage

Mannheim Passage

Heidelberg Passage

Bruchsal Passage

Weingarten Passage

Nussloch Passage

How We Did It

Vehicle Base

Regular S 500 with all emergency braking systems enabled as underlying protection

Sensors

Actuators

Special software version for Electrical Power Steering

to allow for larger steering angles at higher speeds

(212 4x4 ZF EPS)

Special software version for DISTRONIC

to limit initial acceleration and braking deceleration

(222 MB DISTRONIC)

Steering Acceleration / Braking

System Structure

Feature Loc Camera

Vehicle Loc Sensors

Stereo Camera

Traffic Light Camera

360° Radars

DISTRONIC Radars

ESP Sensor Cluster

Traffic Light Map

Radar Processing

Lane Localization

Traffic Light Detection

Object Detection

Feature Localization

Feature Map

Localization

Planning Map

Planning

ESP

EPS

Visualization

Lane Map

Sensors Actuators Artificial Intelligence

RDU Emulation

What the Car Sees

Stereo Vision

Left Image Right Image Disparity (Distance) Image

Color encoded distance: close ….. far

Object Recognition: Pedestrians

Pedestrian examples Non-pedestrian examples

Image

Depth

Input Hypothesis Tracking Classification

Traffic Light Recognition

Traffic light recognition is not as easy as one may think since

• When stopping in front of the traffic light, it must be in the field of view • When approaching a traffic light on rural roads, it must be visible at large distances • At intersections, we have to find “our” traffic light

Traffic Light Recognition

Easy Medium Hard Impossible

What the Car Knows

What's in a Map?

Trajectory plan: color = confidence Vehicle Stop line

Traffic light

Map Architecture Different map layers serve different purposes.

Planning layer Localization layers Navi map (GDF)

Map Accuracy

Relative accuracy of map features of 10 cm

desirable

Curb +/- 5 cm

Lane marking +/- 5 cm Traffic light

+/- 10 cm

Camera image of road surface

Map data

Speed limit +/- 10 m

Vehicle Localization

1. Landmarks

for localization are defined.

3. A camera detects landmarks.

Those are matched with landmark maps

to compute the actual pose of the vehicle.

2. A map is created

with such localization landmarks.

[ ]

[ ]

[ ]

Landmark Robustness and Diversity Different environments require different types of landmarks.

Lane Markings

Curbs

Edge Detector

Urban: "Feature Loc" Collaboration with KIT

Rural: "Lane Loc"

Map Generation

To explore map generation for autonomous driving, Daimler has entered into a cooperation with Nokia HERE as one of the world's leading map suppliers.

Exploration areas: • Content needs (and accuracy needs)

for autonomous driving • Localization support via landmarks

in maps • Scalable and reliable data capturing

methods • Online map updating • Online map insufficiency reporting

incl. crowd sourcing concepts • "Path clearance" concepts

Source: HERE

What the Car Thinks

Maneuvering Tasks

Object classification and prediction

Behaviour generation

Trajectory planning

Prediction Path A (unlikely)

Prediction Path B (likely)

Prediction w/o lane information

Object Classification and Prediction

Hypothesis generation for movements of dynamic objects based on • Object attributes • Driving lanes

Probabilistic evaluation of hypotheses

Behaviour Generation

Hierarchical concurrent state machine to determine vehicle behaviour in individual situations

Collaboration with KIT (former AnnieWay technology from Urban Challenge)

Trajectory Planning Examples

How the Car Behaves

Overland

Few road users, unobstructed view, dedicated lanes

Inner-City

Denser traffic, obstructed view, dedicated lanes

Parked Cars

Driving around static obstacles

Cyclists

Driving around dynamic obstacles

Pedestrians

Stopping for pedestrians at crosswalks

Intersections

Stopping at traffic lights at intersections

Turning

Observing cross and oncoming traffic (possible manual confirm)

Roundabouts

Waiting for "empty slot" from a safe position

What Is Next

Complexity of Automation

Low ego velocity High ego velocity

Structured traffic

environment

Chaotic traffic

environment

Traffic Jam

Parking

Highway

Off-Highway

Step1

Step 4 Step 2

Step 3