sensor fusion and tracking for autonomous systems

1© 2015 The MathWorks, Inc.

Sensor Fusion and Tracking for

Autonomous Systems

Gerald [email protected]

mailto:[email protected]

2

Capabilities of an Autonomous System

Sense

3


Sense

Perceive

4


Learning Algorithms

Optimization

Sense

Perceive

Decide

& Plan

5


Control Algorithms

Sense

Perceive

Decide

& Plan

Act

6

Agenda

▪ Introduction

▪ Technology overview of perception

▪ Algorithm development for sensor fusion and tracking

▪ Q&A

▪ Resources for further exploration

7

Sensor Fusion and

Tracking

Self- awareness Situational awareness

Accelerometer, Magnetometer,

Gyro, GPS…

Radar, Camera, IR, Sonar, Lidar,

…

Signal and Image

ProcessingControl

Sensor fusion and tracking is…

8

Timeline of Technology Advances

Military Commercial Ubiquitous

TodayTimeline

Multi-sensor Fusion

for Autonomous Systems

Computer Vision

for Transportation

Multi-object

tracking

Localization

Air Traffic Control

9

Fusion Combines the Strengths of Each Sensor

Vision measurement

at time step k

Radar measurement

at time step k

Fused

estimate at

time step k

Fused estimate

at time step k-1

Predicted estimate

at time step k

Cross range

Down range

Vision

Measurement

Radar

measurement

Track (fused

estimate)

Ellipse

represents

uncertainty

Legend

10

What is Localization?

PositionInertial Sensor Attitude

11

Fusing Sensor Data Improves Localization

Error MeasurementsSensorsGround truth vs. Estimate

12

Fuse IMU & GPS for Self-Localization of a UAV

Sense

Perceive

Decide

& Plan

Act

Locate

Self

Track

Obstacles

13

Scenario Definition and Sensor Simulation

Flexible Workflows Ease Adoption

Ownship

Trajectory

Generation

INS Sensor

Simulation

Recorded

Sensor Data

Visualization

&

Metrics

Algorithms

gnnTrackergnnTrackerINS Filter,

Tracker, etc..

Actors/

PlatformsRadar, IR,

& Sonar

Sensor

Simulation

Documented

Interface

for detections

Documented

Interface

for tracks

14

Detections

Multitarget Tracker

TracksTracking

Filter

Track

Association

and

Management

From various sensors at

various update rates

▪ Assigns detections to tracks

▪ Creates new tracks

▪ Updates existing tracks

▪ Removes old tracks

A Multi-object Tracker is More than a Kalman Filter

▪ Fuses measurements with

the track state

15

1° Azimuth

Resolution

+ 270m

at 30km

Two targets seen

as one by the radar

Did the trajectories cross?

Performing What-If Analysis

16

+ 175m

at 10km

+ 9m

at 1km


tracker = trackerGNN( ...

'FilterInitializationFcn',@initCVFilter,...

'MaxNumTracks', numTracks, ...

'MaxNumSensors', 1, ...

'AssignmentThreshold',gate, ...

'TrackLogic', 'Score', ...

'DetectionProbability', pd, ...

'FalseAlarmRate', far, ...

'Volume', vol, 'Beta', beta);

tracker = trackerGNN( ...

'FilterInitializationFcn',@initIMMFilter,...



'AssignmentThreshold',gate, ...




'Volume', vol, 'Beta', beta);

17


tracker = trackerTOMHT( ...

'FilterInitializationFcn',@initIMMFilter,...



'AssignmentThreshold’,[0.2,1,1]*gate, ...




'Volume', vol, 'Beta', beta, ...

'MaxNumHistoryScans', 10, ...

'MaxNumTrackBranches', 5,...

'NScanPruning', 'Hypothesis', ...

'OutputRepresentation', 'Tracks');

18

Comparing Trackers and Tracking FiltersFalse track

Dropped track

SlowerFaster

19

Point object vs. Extended object

▪ Point object

– Distant object represented as a single point

– One detection per object per scan

▪ Extended object

– High resolution sensors generate

multiple detections per object per scan

20

Extended Object Tracking

Sense

Perceive

Decide

& Plan

Act

Locate

Self

Track

Obstacles

21

Tracking with Lidar

Sense

Perceive

Decide

& Plan

Act

Locate

Self

Track

Obstacles

22

Connect to Decision Making / Control: ACC Demo

Sense

Perceive

Decide

& Plan

Act

Locate

Self

Track

Obstacles

23

Connect to Decision Making / Control: ACC Demo

Sense

Perceive

Decide

& Plan

Act

Locate

Self

Track

Obstacles

24

Sensor Fusion and Tracking …

Leverages Sensor StrengthsIs Ubiquitous

Sense

Perceive

Decide

& Plan

Act

Sensor Fusion

and Tracking

Signal and Image

ProcessingControl

Enables Autonomy

25

Summary

Improve your development and test workflows for localization and tracking. We provide not

only algorithms but sensor models, scenario generation and metrics. Integrate with home

grown tools and fill in gaps.

Our configurable tracking and fusion algorithms enable re-use and sharing across your

organization and industry which will also improve overall efficiencies

Perform quick algorithm and system trade-offs eliminate errors early in the development cycle.

Direct path to move from tracker and fusion models to C-code for prototyping and initial

deployment.

26

Learn More

▪ Please visit our Technology Showcase for more details on the workflows

https://www.mathworks.com/products/sensor-fusion-and-tracking.html

https://www.mathworks.com/products/sensor-fusion-and-tracking.html

sensor fusion and tracking for autonomous systems

Documents