safe machine learning solutions virtual autonomous driving ......iso/pas 21448:2019 road vehicles...

Dr. Stefan Milz

Founder & Managing Director (Head of R&D) @ Spleenlab.ai

Research Fellow @ Ilmenau University of Technology

Chair of Software Engineering for Safety-Critical Systems / Prof. Dr. Patrick Mäder

Strategy to Increase Safety of Deep Learning basedPerception for Highly Automated Driving

Virtual Autonomous Driving Meetup #2

Safe Machine Learning Solutions

http://spleenlab.ai


AGENDA - SAFETY AND DEEP LEARNINGWe climb up the ladder of automation towards levels 3, 4 and 5:

2

In this Presentation, we want to explore a strategy to build Perception systems using DL in such a way that they are robust and safe

• Understand the challenges of ML for safety critical systems• Current Safety Strategies with respect to actual reference standards• The Data problem and scalability of AI• Example Research Overview: Domain Adaptation, Self-Supervision• Safety by Design: Doer / Checker Principles in Automated Driving

*Image source Stefan Milz

https://www.regansweddingcars.com/self-driving-car-levels.html


WHAT IS AUTOMOTIVE SAFETYExtreme complex definition in the domain of Automated Driving

3

Wood et al. (White paper Safety First) → Twelve Principles!

Stefan Milz

https://www.aptiv.com/docs/default-source/white-papers/safety-first-for-automated-driving-aptiv-white-paper.pdf


WHAT IS DEEP LEARNINGDeep Learning is a subset discipline of AI (*source)

4Stefan Milz

https://towardsdatascience.com/rolling-in-the-deep-learning-basic-concepts-for-everyone-84bdb4766d18


WHAT IS DEEP LEARNINGDeep Learning is a subset discipline of AI (*source)

5Stefan Milz

https://towardsdatascience.com/rolling-in-the-deep-learning-basic-concepts-for-everyone-84bdb4766d18


WHAT IS DEEP LEARNINGIs a statistical Method and not fully deterministic (or describe-able) in its behavior!

6Stefan Milz

Example: Regression or Classification Tasks


WHAT IS DEEP LEARNINGIs a statistical Method and not fully deterministic (or describe-able) in its behavior!

7Stefan Milz

Statistical Components during Inference-Time!

Drop Out

Pruning

De-Noising

Quantization


WHAT IS DEEP LEARNINGConclusion

8Stefan Milz

Deep-Learning based System-Behavior is not describable deterministically from a functional point of view


WHAT IS DEEP LEARNINGConclusion - Question for Validation?

9Stefan Milz


How to validate a Deep Learning based Automotive System?


WHAT IS DEEP LEARNINGConclusion - Question for Validation? - Data

10Stefan Milz



Data


WHAT IS DEEP LEARNINGConclusion - Question for Validation? - Data & Safety by Design

11Stefan Milz



Data

Safety by Design


WHAT IS DEEP LEARNINGConclusion - Question for Validation? - Data & Safety by Design & New Standard

12Stefan Milz



Data

Safety by Design

New Standard


WHAT IS DEEP LEARNINGConclusion - Question for Validation? - Data & Safety by Design

13Stefan Milz



Data

Safety by Design

New Standard

Current Reference Standards(ISO26262/

SOTIF)

ResearchOngoing

Leitinitiative KIKI-Absicherung


CURRENT REFERENCE STANDARDSISO26262 - SOTIF

14Stefan Milz

ISO/PAS 21448:2019 Road Vehicles – Safety of the intended functionality (SOTIF)ISO 26262:2018 Road Vehicles – Functional safety

ISO/SAE CD 21434 Road Vehicles – Cybersecurity engineeringISO 19157:2013 Geographic information – Data qualityISO/TS 19158:2012 Geographic information – Quality assurance of data supplyISO/TS 16949:2009 Quality management systems ISO 9001:2008 for automotive production and relevant service part organizationsISO/IEC 2382-1:1993 Information technology – Vocabulary – Part 1: Fundamental termsISO/IEC/IEEE 15288:2015 Systems and software engineering – System life cycle processes


VALIDATION WITH DATA OFTEN FAILS Assuming Deterministic Functions in actual Reference Standards

15Stefan Milz

Current Standards require a proof of compliance verification at all levels of detail, down to the deepest software requirement → i.e. “Every System Condition must be describable”


VALIDATION WITH DATA OFTEN FAILS Determinism vs Deep Learning

16Stefan Milz


DL→ Statistical Function → Data Driven →Data based System Condition → No Determinism



17Stefan Milz


DL→ Statistical Function → Data Driven →Data based System Condition → No DeterminismData is the main carrier of the proof of compliance verification → Standards (SOTIF, L3) require a huge amount of data to cover all possible scenarios with a reasonable assurance for high critical functions (e.g. ASIL Level D)



18Stefan Milz


DL→ Statistical Function → Data Driven →Data based System Condition → No DeterminismData is the main carrier of the proof of compliance verification → Standards (SOTIF, L3) require a huge amount of data to cover all possible scenarios with a reasonable assurance for high critical functions (e.g. ASIL Level D)

Example: Data Amount for ASIL Level D → at least 108 driven hours needed (~11k years)


THE DATA PROBLEMRemarks

19Stefan Milz

Data Amount for ASIL Level D → 11k Years driven hours → nearly impossible amount of Data!Even ASIL B requires a huge amount of Data → already done for DL


THE DATA PROBLEMRemarks

20Stefan Milz

Data Amount for ASIL Level D → 11k Years driven hours → nearly impossible amount of Data!Even ASIL B requires a huge amount of Data → already done for DL

What about Labels?Mainly Supervised Training Regimes (Training Loss Definition) are used for Deep Learning in Automotive:Three important Paradigms for Deep Learning can be derived:

❏ Simulated Sensory Data Needed❏ Domain Adaptation Methods are inevitable❏ Self-Supervised Models are necessary and attractive

Example: Visual Semantic Segmentation


SIMULATION & DOMAIN ADAPTATIONOpen Source Engines

21Stefan Milz

Carla → source AirSIm → source

http://proceedings.mlr.press/v78/dosovitskiy17a/dosovitskiy17a.pdf

https://arxiv.org/pdf/1705.05065.pdf


SIMULATION & DOMAIN ADAPTATIONOverview

22Stefan Milz

Target Domain GeographicDifference

Target Domain Sensor

Viewpoint

Target Domain Different Weather

Target Domain → InferenceReal-Word

Source Domain → Training

e.g. Simulation

Almost eachReal World Application with

Deep Learning performs Domain Adaptation


SIMULATION & DOMAIN ADAPTATIONOverview

23Stefan Milz

Target Domain GeographicDifference

Target Domain Sensor

Viewpoint

Target Domain Different Weather

Target Domain → InferenceReal-Word

Source Domain → Training

e.g. Simulation

Almost eachReal World Application with

Deep Learning performs Domain Adaptation

Conclusion: We may do not need the highest realism in Simulation but the highest diversity, i.e. classes, structures, scenes, weather etc.


SIMULATION & DOMAIN ADAPTATIONExample: Sim2Real - ADVENT

24Stefan Milz

Visual Semantic Segmentation (Vu et al.) GTA5 2 CityScapes

Simulation Engine

Real-World

Endless Labels

No Labels




25Stefan Milz


Simulation Engine

Real-World

Endless Labels

No Labels




26Stefan Milz





27Stefan Milz

Impressive Results without having Real-world ground truth labels

Low entropy

High entropy


SELF-SUPERVISION MODELSOverview

28Stefan Milz

Self-supervised learning (or self-supervision) is a relatively recent learning technique (in machine learning) where the training data is autonomously (or automatically) labeled

→ Attractive to our Data-Problem (Mainly Geometrical Tasks)


SELF-SUPERVISION MODELSExample: FisheyeDistanceNet → Monocular Scale-Aware Depth Estimation on Fisheye

29Stefan Milz

No Depth Ground TruthVisual Data + Odometry ICRA 2020 Oral (Kumar et al.)

Quite Impossible to gather Dense Depth Labels for large FoV Cameras (e.g. Fisheye)




30Stefan Milz

No Depth Ground TruthVisual Data + Odometry ICRA 2020 Oral (Kumar et al.)

Quite Impossible to gather Dense Depth Labels for large FoV Cameras (e.g. Fisheye)




31Stefan Milz



32Stefan Milz

http://www.youtube.com/watch?v=Sgq1WzoOmXg&t=15

http://www.youtube.com/watch?v=Sgq1WzoOmXg&t=128


SELF-SUPERVISION MODELSExample: StickyPillars - Robust and Efficient Feature Matching on Point Clouds

33Stefan Milz

GT Extracted from Odometry Data (Simon et al.) (Example: KITTI Point Clouds → Δ 10 Frames)a) StickyPillarsb) ICP

Robust (realtime) Point Cloud Registration without Registration GT (SLAM)

https://arxiv.org/abs/2002.03983



34Stefan Milz



35Stefan Milz

(Example: KITTI Odometry)



36Stefan Milz

https://docs.google.com/file/d/1FRORQdZdVd6JM0-XeT9n_zReoQdKiUbd/preview

https://docs.google.com/file/d/1FRORQdZdVd6JM0-XeT9n_zReoQdKiUbd/preview


VALIDATION WITH DATASummary

37Stefan Milz

❏ High Critical Levels (ASIL D): Cannot be validated reliably due to the large amount of data required→ more than 11k years driving data needed

❏ Even for lower Levels (ASIL B) → massive Data amount needed❏ Simulation Engines should be used❏ Domain Adaptation applies in every Real-World Szenario❏ Self-Supervised Models and Domain Adaptation Models are necessary for

scalability


SAFETY BY DESIGN USING DEEP LEARNINGIntroduction

38Stefan Milz

Current Standards (Determinism) require detailed proof of compliance verification at all levels of detail for implementation errorsDeep Learning based Functions (Statistics) based algorithms currently only achieved ASIL-B Level by Validation with Data.


SAFETY BY DESIGN USING DEEP LEARNINGIntroduction

39Stefan Milz

Current Standards (Determinism) require detailed proof of compliance verification at all levels of detail for implementation errorsDeep Learning based Functions (Statistics) based algorithms currently only achieved ASIL-B Level by Validation with Data.

Safety by DesignParadigms

Sensor Analysis(Fusion)

Sensor Redundancy

Separation of Processing chains

ASIL Decomposition


SAFETY BY DESIGN USING DEEP LEARNINGSensor Analysis, Fusion and Sensor Redundancy

40Stefan Milz

Different Sensors have different strengths and disadvantages that requires Fusion

https://docs.google.com/file/d/1uymEkGAZf8cYgv15weeVV8pu-IfS2JLH/preview

https://docs.google.com/file/d/126EA8DXQU-hRiNUn2NR5Zf0qT2Jgn0tj/preview


SAFETY BY DESIGN USING DEEP LEARNINGSensor Analysis, Fusion and Sensor Redundancy

41Stefan Milz

Different Sensors have different strengths and disadvantages that requires FusionOver-Engineering of Sensor Modalities using Deep-Learning is not useful from a functional point of view (e.g. Dynamics on Camera, Classification on Lidar)


ASIL DECOMPOSITIONFunctional Division

42Stefan Milz


ASIL DECOMPOSITIONFunctional Division

43Stefan Milz


ASIL DECOMPOSITION + SENSOR-REDUNDANCYDeep Learning based System

44Stefan Milz

Output

(ASIL D)(ASIL C)

Camera

Lidar

Radar

Ultrasonics

DeterministicModule

(ASIL B)

DeterministicModule

Sensor Fusion

Sensor Fusion

Deep-Learning Module

DeterministicModule

Deep-Learning Module

Sensor Fusion

...

...

+ ++


ASIL DECOMPOSITION + SENSOR-REDUNDANCYDeep Learning based Perception for Highly Automated Driving

45Stefan Milz

Camera

Lidar

Sensor Raw Data

AI-based Module

Deterministic Module

Semantics

Freespace

Redundancy

(ASIL B)

(ASIL C)

(ASIL C)+

+Fusion:

Freespace observes Semantic

Sensor Raw Data


ASIL DECOMPOSITION + SENSOR-REDUNDANCYDeep Learning based Perception for Highly Automated Driving

46Stefan Milz

https://docs.google.com/file/d/145ft_4qzhgOZXF9d8XMeTXcsIjgBeb_h/preview


SAFETY BY DESIGN USING DEEP LEARNINGSummary

47Stefan Milz

❏ Sensor Redundancy❏ Use the best sensor a specific Task❏ Separate Processing Chains, as many as possible!❏ ASIL-Decomposition is a necessary tool

❏ AI based Systems (Modularized) are meanwhile certifiable with the current standards


NEW VALIDATION PARADIGMSOutline

48Stefan Milz

❏ Research Ongoing → VV - Methoden, “Leitinitiative AI” - KI Absicherung❏ Explainable AI, Unsupervised Learning, Uncertainty Prediction, Teacher Networks


Dr. Stefan MilzManaging Director / Head of R&D

M: [email protected]: +49 172 64 240 55

Thanks for your Attention

safe machine learning solutions virtual autonomous driving ......iso/pas 21448:2019 road vehicles...

Documents