multi sensor data fusion andres navarro

8/2/2019 Multi Sensor Data Fusion Andres Navarro

1/52

Multisensor Data Fusion

Andrs Navarro


2/52

Contents

Introduction Definition

Sensors and levels

Fusion models Fusion techniques

Distributed Data Fusion

Scenarios

Conclusions


3/52

Introduction

Motivation Master Thesis

First approach to sensor fusion

Overview of state-of-the-art

Guide for people who do not know about sensorfusion to get introduced to the issue


4/52

Introduction

Why is sensor fusion needed? Research questions

What is multisensor data fusion? Is there an unanimousdefinition?

What models for multisensor data fusion exist in theliterature? Do they have common descriptions? Do theycontradict each other?

What techniques or methods can be used in multisensordata fusion?

Centralized or decentralized data fusion?

What scenarios can multisensor data fusion be applied?


5/52

Definition

JDL Data Fusion Lexicom

A. Steinberg and C. Bowman.

Wald

A process dealing with the association, correlation, and combination of dataand information from single and multiple sources to achieve refined positionand identity estimates, and complete and timely assessments of situationsand threats, and their significance. The process is characterized bycontinuous refinements of its estimates and assessments, and theevaluation of the need for additional sources, or modification of the processitself, to achieve improved results.

Data fusion is the process of combining data or information to estimateor predict entity states.

Data fusion is a formal framework in which are expressed means and toolsfor the alliance of data originating from different sources. It aims atobtaining information of greater quality; the exact definition of greaterquality will depend upon the application.


6/52

Sensors and levels

Commensurate

multisensor data

Non commensurate

multisensor data

Direct Data Fusion

Feature-level Fusion

High-level FusionInformation extraction


7/52

Fusion models

JDL data fusion model

Dasarathy's functional model

Waterfall fusion process model

Boyd Loop Thomopoulos' Fusion Model

Durrant-Whyte architecture

The Omnibus process model Endsley's Situation Awareness

General Data Fusion Architecture


8/52

Fusion models

JDL data fusion model Sources

Sourcespreprocessing

Level 1

Level 2

Level 3

Level 4 Database management system

HCI


9/52

Fusion models

JDL model revisions Drawbacks

Different revisions

New definitions


10/52

Fusion models

Dasarathy's functional model

Levels of abstraction

Data

Feature

Decision

Categorization of data fusionfunctions in terms of the type ofdata level at input/output.


11/52

Fusion models

Waterfall fusion process model Fusion process

in stages

Omission of

feedback dataflow is themajor limitation.


12/52

Fusion models

Boyd Loop OODA cycle:


13/52

Fusion models

Thomopoulos' Fusion Model


14/52

Fusion models

Durrant-Whyte achitecture


15/52

Fusion models

The Omnibus process model

F i d l


16/52

Fusion models

Endsley's Situation Awareness

F i d l


17/52

Fusion models

General Data FusionArchitecture

Network based

Levels described asclasses with attributesand functions.

F i d l


18/52

Fusion models

Models classification

Elmenreich

Abstract Generic Rigid

Durrant-Whyte and Henderson

Architecture: Meta, Algorithmic, Conceptual, Logical andExecution

Centralized - Decentralized

Local Global

Modular Monolithic Heterarchical - Hierarchical

T h i


19/52

Techniques

Classification Overview

Kalman Filter

Probabilistic Inference

Artificial Neural Networks

Fuzzy Logic

Support Algorithms

Selection

T h i


20/52

Techniques

Classified by JDL levels

T h i


21/52

Techniques


Type of method

Fusion problems

Techniques


22/52

Techniques


Type of method

Techniques


23/52

Techniques


Type of method

Fusion problems

Techniques


24/52

Techniques

Classified by

JDL levels

Type of method

Fusion problems

Data Association

Estimation

Identity declaration

Decision-level identity fusion

Techniques


25/52

Techniques

Kalman Filter

Extended Kalman FilterDiscrete Kalman Filter

Assumed noise

Techniques


26/52

Techniques

Kalman Filter with INS

Inertia System:

Good high frequency information

Drift at a slow rate

Other Position System Good data at low frequency, on the average

High frequency noise

The Kalman Filter approach is instead to use the statistical characteristics of the errorsin both the external information and the inertial components to determine this optimalcombination of information. Actually, the filter statistically minimizes the errors in theestimates of the navigation parameters: on an ensemble average basis, no other meansof combining the data will outperform it, assuming the internal model in the filter isadequate.

Techniques


27/52

Techniques

Kalman Filter with INS

Direct implementation

Indirect feedforward

Indirect feedback

Filter fails System Fails

High sample rate CPU load

Erros in the inertial mustremain of small magnitude

Techniques


28/52

Techniques


Bayesian Inference

It can be used to discriminate between conflictinghypotheses

Initial beliefs are needed before any evidence is evercollected

Sensorfusion:

Techniques


29/52

Techniques


Bayesian Networks Probabilistic graphical model that represents a set

of variables and their probabilisticinterdependencies

Algorithms to perform inference and learning

Dynamic Bayesian Networks

Extension of Bayesian networks thatallows the representation of temporalinformation Signals

Hidden Markov models Model for Markov process: a stochastic process in which the probability

distribution of the current state is conditionally independent of the path ofpast states

Techniques


30/52

Techniques


Dempster-Shafer Theory Generalization of Bayesian Theory: Instead of requiring

probabilities for each question, belief functions are used.

Two ideas:

Sensor1 Sensor2

m1(u

0) m

2(u

0)

m(u0)=m

1(u

0)m

2(u

0)

Obtain degrees of belief for one question fromsubjective probabilities fora related question.

Use Dempster's rule for combining these degrees ofbelief:

Techniques


31/52

Techniques




Two ideas:

Sensor1 Sensor2

m1(u

0) m

2(u

2)

m(u0)=m

1(u

0)m

2(u

2)

Obtain degrees of belief for one question from

subjective probabilities fora related question. Use Dempster's rule for combining these degrees of

belief:

Techniques


32/52

Techniques




Two ideas:

Sensor1 Sensor2

m1(u

0) m

2(u

1)

Dempster's rule

Obtain degrees of belief for one question from

subjective probabilities fora related question. Use Dempster's rule for combining these degrees of

belief:

Techniques


33/52

Techniques


Generalized evidence processing theory

Unifies the Bayesian theory with the D-S, combining theiradvantages and avoiding their disadvantages

Each sensor collects evidence and assigns the evidencevia probability masses; unlike D-S, GEP assigns andcombines probability masses based on the a prioriconditional probability of the hypotheses.

Techniques


34/52

Techniques


Computational model of biologicalneural networks:

Densely interconnected set of artificial neurons:simple units as perceptron

Feed-forward / recurrent

Non linear statistical data modelling: They can learn a complex relationship between

inputs and outputs, normally established by theunit weights

Learning:

Units' weights Backpropagation algorithm

Network structure

Techniques


35/52

Techniques


Neurons can be trained to represent sensoryinformation and, through associative recall,complex combinations of the neurons can beactivated in response to different sensory stimuli.

The main advantage of neural networks formultisensor fusion is that there is no need of amodel for the sensors or the uncertainties

Techniques


36/52

Techniques

Fuzzy Logic

Fuzzy set theory:

Elements have degrees of membership to thedifferent sets, differing from classical set theory,where elements belong or do not belong to a certain

set.

Rules: IF antecedent THEN consequence

Operators: OR, AND and NOT

Steps: Fuzzyfcation

Rule evaluation

Aggregation

Defuzzyfication

Techniques


37/52

Techniques

Fuzzy Logic

Uncertainty in multisensor fusion can be directlyrepresented in the inference process by allowing eachproposition to be assigned a degree of truth.

Fuzzy Fusion Network:

Input data

Feature Extraction

Feature Level Fusion

Decision Level Fusion

Techniques


38/52

q s

Support Algorithms

Required functions for the fusion system

Library of basic numerical methods

Database management

Man-Machine Interaction Sensor Management

Techniques


39/52

q

Techniques Selection

Goals: Maximum effectiveness: Algorithms should make

inferences with maximum specificity in the presence ofuncertain and missing data, dealing with minimal or noavailable a priori information.

Operational constraints and time constraints must beconsidered.

Resource efficiency in CPU and communications load.

Operational flexibility to account for operational needs

with changing a priori data.

Functional growth.

Techniques


40/52

q

Techniques Selection

Steps:1.Identifying categories of data-processing techniques or algorithms.

2.Surveying existing prototype and fielded data fusion systems.

3.Analyzing system requirements.

4.Analyzing and defining operational concepts for manual and

automatic processes.5.Identifying preliminary algorithms

6.Performing trade-off analyses of algorithm effectiveness versusrequired system resources.

7.Preparing detailed designs and prototypes of selected algorithms.

8.Refining and tuning the algorithms.

Distributed Data Fusion


41/52

A collection of processing nodes, connected bycommunication links, in which none of the nodes hasknowledge about the overall network topology.

Requirements

Dynamic Topology Management

Information- Sharing Strategies Algorithms

Scenarios


42/52

Image Data Fusion

Advantages: Reduction of overall uncertainty and increase of accuracy.

New features in a scene can be perceived

More timely information is available

Scenarios


43/52

Robot Navigation

Scenarios


44/52

Ambient Intelligence

Electronic environments that are sensitive and responsive tothe presence of people.

Characteristics:

Embedded

Context-aware

Personalized

Adaptive

Anticipatory

Scenarios


45/52

Augmented Reality

Integration of virtual content in a real environment in real time.

Alignment between virtualcontent and real world

Fusion of tracking systems:

INS

GPS Ultrasound

Vision-based

Accurate position

Scenarios


46/52

Augmented Reality

Interaction with otherperceptual systems:

Orienting, Auditory,Haptic, Taste, Smell

Feature extraction

Human Behaviour Sensors: Position, orientation,

body gestures, speech, vitalsigns, eyetracking...

Human behaviour and

experience models andsimulations

Conclusions


47/52

What is multisensor data fusion? Is there an

unanimous definition?

Most of definitions are restrictive to a certainterminology and applications

A broader definition is needed to cover such awide diversity of sensor fusion applications

Wald's definition is chosen

Discussion will continue

Conclusions


48/52

What models for multisensor data fusion exist in the

literature? Do they have common descriptions? Dothey contradict each other?

A common point in most of them is the need of divide thefusion process in levels of data abstraction.

More disagreement is found in the idea of a cyclingprocessing.

Relationship between specification and usability

Combine the underlying ideas for the final design.

Conclusions


49/52

What techniques or methods can be used in

multisensor data fusion?

Data fusion at different levels of abstraction impliesthe use of multiple techniques.

A layout or scheme for the implementation of anykind of sensor fusion application is not feasible.

The design of the fusion algorithms is a lengthy taskwhere multiple fusion techniques can be combined.

Conclusions


50/52

Centralized or decentralized data fusion?

The decentralized fashion has some advantagesand some disadvantages comparing to thecentralized one.

The implementation of Distributed Data Fusion

requires:

The use of certain fusion models that allowdecentralization.

Specific algorithms

Conclusions


51/52

Final conclusion

Multisensor Data Fusion is a broad issue due to thewide range of scenarios that it can be applied to.Therefore, to find a definition, a model or an algorithmscheme that is explicit, meaning that it can be followedto implement a real system, and at the same time

usable for any kind of application, is an unfeasibletask. Hence, a view of the different approaches,theories and implementations in the issue of sensorfusion can be presented, intending to be useful as acollection of different ideas that should be combined inthe implementation of a real fusion system.


52/52

Multisensor Data Fusion

Andrs Navarro

multi sensor data fusion andres navarro

Documents