data fusion- a review. layout 1-benefits of multisensor devices 2-typical sensors used in data...

Data Fusion- A review

Data Fusion- A reviewLayout

1-Benefits of multisensor devices2-Typical sensors used in data fusion3-Sensor performance4-Data fusion models5-Decision fusion in parallel sensor suite6-Comparison of mathematical tools in data fusion


Benefits of Multiple sensor devices:

-Reduction in measurement time-A downtime reduction and an increase in reliability-Redundant and complementary information-A higher signal-to-noise ratio-A reduction in measured uncertainty-A more complete picture of the environment


Sensor Output format Applications

Optical sensor Image Mobile robot guidance

Radar Pulse signal Target detection and target tracking

Infrared sensor Image Object identification

Satellite Image Surveillance and pattern recognition

Ultrasonic sensor Pulse signal Mobile robot guidance

NDT sensor Voltage Materials examination

Sonar Pulse echo Obstacle detection

Laser Image Pattern recognition

X-ray Image Medical

Survey of typical sensors used in Data fusion

Data Fusion- A reviewSensor performance

POD : Probability Of Detection

PFC: Probability of False Call

ROC: Receiver Operating Characteristic(POD versus PFC)

Sensor performance can be statistically represented using:

Major advantage of ROC curves compared to POD curves: In ROC curves false calls are taken into accountBut…In practice, ROC curves are difficult to realise.


The performance and potential of each used sensor needs to be established in order to assign weight of evidence,for example, in sensor data fusion.

The most common sources of uncertainty-little or no knowledge about measurement-incomplete measurement (when data are approximated rather than waiting for complete data which may be time consuming and costly)-limitations of the system


Common types of errors

-ambiguous

-incomplete

-incorrect

-measurement

-random

-systematic

-reasoning

- practicality (environment)-Human error-Equipment malfunction-False negative-False positive

-Incorrect output-Unreliable-No output

-Calibration error-Precision-Accuracy

-Inductive error-Deductive error

Data Fusion- A reviewData Fusion Models

-Multisensor data integration and fusion center-Three-level fusion paradigm-Centralized signal detection system-Distributed (decentralized) signal detection system

[X.E. Gros, NDT Data Fusion, 1997]


Data Processing

Assignment of Bayes or Dempster-

Shafer Rule of

Combination

Sensor Data

Selection

Y1

Y2

Yn

Z1

Z2

Zn

OptimumEstimationDecisionLevelFused

Data

Integration Fusion Integration

X1

X2

Xn

Raw Data

Sensor 1

Sensor 1

Sensor 1

ProcessedData

Multisensor data integration and fusion center

Measurements from n sensors are integrated, data is then processed withevidental reasoning, probabilistic and belief theories, the results are classifiedand selected before a decision on the optimum fused information is made.


Three-level fusion paradigm

Level of evidence

Level of Dynamics

Signal Level

Database

Sensors

DataFusion

DecisionFusion

Features fusion


Distributed (decentralized) signal detection system

Fusion Center

GlobalDecision

Level

Measurement

Measurement

Measurement

Sensor 1

Sensor 2

Sensor N

Feature Extraction

Level

Local Decision

Local Decision

Local Decision

Fuse identity declarations using Bayesian theory, the Dempster-Shafer paradigm or Thomopoulos generalized evidence processing (GEP).The output from each sensor is a decision which forms the inputs to a fusion center where association is performed.


Centralized signal detection system

Fusion Center

DecisionLevel

Measurement

Measurement

Measurement

Sensor 1

Sensor 2

Sensor N

More suitable for fusion of raw data but the association phase can be difficult .


Four major sensor network types

-Serial-Parallel-Parallel-Serial-Serial-Parallel

Sensor 1

Sensor 2

Sensor n


Sensor 1

Sensor 2

Sensor j

Sensor t

Local

processor 1

Local

processor 2

Local

processor j

Local

processor t

Data set Z1

Data set Z2

Data set Zj

Data set Zt

Decision

Fusion

Processor

OutputO1

OutputO2

OutputOj

OutputOt

Consistentdecision acrossthe suite?

Yes

No

A recursive processing structure for enhanced performance with aparallel sensor suite. B.V. Dasarathy, 1991, IEEE

Decision fusion in parallel sensor suite


Sensor output can be regarded as a decision arrayof n decisions. The efficiency of each sensor, , is the probabilityof correctness of the decision Dj from sensor j, a measure of the effectiveness of a sensor.

Cj and Wj : the belief that the decisions from sensorj are correct and wrong (based on the Dempster-ShaferTheory)Uj: the ignorance (uncertainty) of a measurand

j

1

1 ( )

j j

j j j

C W

U C W


ck , wk , uk : incremental probabilities of the joint correct, incorrect decisions and nondecisions respectively

1 1 1

2 2 2 1

1

1 1

1

1

k k k k

k k

i i ki i

c w u

c w u u

c w u u

c w u

1k k kC W u

1

k

k ii

C c

1

k

k ii

W w


( 1)1 1

1

( 1)1 1

1

ki

ki

ki

ki

C c u

W w u

( 1) 11

1 1

1

1

kki

i

uu

u

Data Fusion- A reviewDecision fusion in parallel sensor suite

1 1max max max max

1 1 1 1

1( ) ( )k k k k

c wC W C W

c w c w

1 11 1 1

1 1

(1 ) (1 )0 1

(1 ) (1 )

k k

k k

u uC c W w u

u u


Second sensor

Decision

First sensor Decision

Target Nontarget Undecided

Target Target Undecided Undecided

Nontarget Undecided Nontarget Undecided

Undecided Undecided Undecided Undecided

A simple decision fusion rules matrix



1 1 2

1 1 2

1 1 2 1 2

1

(1 )(1 ) 1

( ) 2 1

: '

c

w

u

sensor s correct decision rate

1 1 2

1 1 2

min( , )

min((1 ), (1 ))

c

w

sensor1

sensor2

Local processor 1

Local processor 2

Decision Fusion

processor



Perfect Sensor Case: ηi = 1 ( i = 1,2 )

1

1

1

max

max

1

0

1

1

0

k

k

c

w

u

C

W

η : the efficiency of the imperfect sensor

The fused decision approaches the correct decision asymptotically even though one of the sensors may remain imperfect and the user does not know which one it is.



Bad Sensor Case: ηi = 0 ( i = 1,2 )

1

1

1

max

max

0

1

0

1

k

c

w

u

C

W

Fusion leads to complete failure of the system.Therefore no totally faulty sensor can beallowed to operate indefinitely in a two-sensor fusion system of this type.



Equally Imperfect Sensor Case: ηi = η ( i = 1,2 )2

1

21

1

2 2

max2 2max

1

(1 ) 1

2 (1 ) 1

1 2

1 2 2 1 2 2k

c

w

u

C W

)1(2ln

/)12)(1(ln

k

Minimum number of recursions needed for the fused decision to bebetter than the decision derived by the individual sensor:



Fused correct decision rate (Ck ) vs. sensor efficiency ( η) at different numbers of recursions (k)

Fused correct decision rate (Ck ) vs. recursion number (k) at different sensor efficiencies (η)


Ck

η=0.1 η=0.2η=0.3η=0.4η=0.5η=0.6η=0.7η=0.8η=0.9

k

Ck

η


General Case

1 2 1 2

21

1

21

2

2max

max( , ) min( , ) 0 1

1

(1 )(1 ) 1

[(1 ) 2 ] 1

2 ( 1) 1k

c

w

u

C

Asymptotic fused correct decisionrate( Ck|max ) vs. sensor efficiency( η ) at different sensor performanceratios (α )


η

Ck |max

η1 and η2 are related by α


1 1 1 1 11 1

1 (1 ) 1 1 ( )t t

j jj j

c w u c w


Asymptotic fused decision efficiency ( Ck | max )vs. number of sensors ( t) for different sensorefficiencies (η )

t

Ck |max

η=0.1 η=0.2η=0.3η=0.4η=0.5η=0.6η=0.7η=0.8η=0.9


Equally Imperfect Sensor Case: ηi = η ( i = 1,2,…,t )

tt

t

ktt

t

k

tt

t

t

QP

r

q

p

)1(

)1(

)1(

1)1(1

1)1(

1

maxmax

1

1

1

tt

t

k)1(1ln

}/)1{(1)1(ln )1(

Minimum number of recursions needed for the fused decision to be better than the decision derived by the individual sensor:



11

1t

jj

c

11 1

1

11

t

jj

tw

m

1 1 11 1u c w


m: the number of hypothesis


The minimum number of sensors for the correct fused decision rate to exceed the incorrect fused decision rate:

1/1ln

1lnmin m

mt

The asymptotic values of the fused decision rates:

,

, 1k Max t

f mC

f m

1

, 1

t

k Max tW

f m

11, tmmf


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

w1 :majority c1:majority

w1 :consensus

c1 consensus

Initial fused decision rates vs. sensor efficiency with three sensors(comparison of the consensus and majority based fusion methods)

c1, w1

η

Binary Decision making



0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

w1 :majority c1 majority

w1 :consensus c1 consensus

Initial fused decision rates vs. sensor efficiency with three sensors(comparison of the consensus and majority based fusion methods)

c1, w1

η

Multihypothesis Decision making (m=3)


Data Fusion- A reviewFusion Methodology

The most common data fusion and integration methods

Fusion method ApplicationsPixel level fusion Image processing, image segmentation

Bayesian theory Decision making between multiple hypotheses

Demspter-Shafer theory of evidence Decision making, Beliefs intervals

Neural Network Signal interpretation

Neyman-Pearson criteria Decision making

Fuzzy Logic Handle vagueness

Knowledge based system Pattern recognition

Markov random field Image processing


Classical Inference

The most common inference approaches based on an observed sample of data for acceptance or rejection of a hypothesis:

-Maximum a posteriori-Likelihood ratio criterion-Neyman-Pearson test-Bayes criteria


Classical Inference

Maximum a posteriori

Compares two probabilities assigned to two hypothesis and favors either one or the other depending only on their chance of occurrence.

)|()|( 10 yHpyHp

y is an observation from a sensor and Hi a hypothesis i


Classical Inference

Likelihood ratio criterion

A test to decide between hypothesis H0 or its alternative Hi . If Λ(u)>t , H0 is true otherwise, H1 is true..

tHup

Hupu i

n

ii

)|(

)|()(

10

0

1

H0 and H1 are hypothesis 0 and 1, n the number of sensors, ui random observed sample data and t, the threshold (significance level) determined from experiment.

Likelihood ratio =(level of sufficiency)

Λ(u) : the degree to which the observation of evidence u influences the Prior probability of H


Classical Inference

Neyman-Pearson hypothesis test

A general theory used to make a decision between two hypothesis. Hypothesis H0 is rejected if the following equation is verified:

tHu

Hu

)|(

)|(

1

0

The threshold t is chosen depending on the risk the user is prepared to take to accept or reject H. the smaller the value of t, the lower the risk.


Classical Inference

Bayes criteria

A cost function based on false alarm and probability of detection is used to select between two hypotheses H0 and H1. P0 and P1 are a priori probabilities which govern the decision output.

The cost function C for each decision outcome:-C00 : the cost function assigned to the decision 0 when the true outcome is 0 P(H0|H0) :the probability associated with this decision- C01 : the cost function assigned to the decision 0 when the true outcome is 1 P(H0|H1) :the probability associated with this decision- C10 : the cost function assigned to the decision 1 when the true outcome is 0 P(H1|H0) :the probability associated with this decision- C11 : the cost function assigned to the decision 1 when the true outcome is 1 P(H1|H1) :the probability associated with this decision


Classical Inference

Bayes criteria

The expected values of the cost as the risk R is defined as :

00 0 0 0 01 0 0 1 10 1 1 0 11 1 1 1( | ) ( | ) ( | ) ( | )R C P P H H C P P H H C PP H H C PP H H

The decision intervals are defined as:

00 1 01 11

1 0 10 00

( | ) ( )( )

( | ) ( )( )

Hp y H p H C C

p y H p H C C

Where the right hand side is the threshold of the test and should be such that the cost is as small as possible.


Bayesian Inference

Used to estimate the degree of certainty of multiple sensors providing information about a measurand.

Uses a priori probability of a hypothesis to produce an a posteriori Probability of this hypothesis.

Suppose there are n mutually exclusive and exhaustive hypotheses H0…Hn that an event E will occur.The conditional probability p(E|Hi) states the probability of an event E that Hi is true and is given by:

( | ) ( )( | )

( )i i

i

p E H p Hp H E

p E

p(Hi) : a priori probability of the hypothesis Hi

p(Hi|E): a posteriori probability of having E given that Hi is true


Bayesian Inference

If multiple sensors are used…

0 10 1

( | ) ( | )... ( | ) ( )( | ... )

( )n

nj

j

p H E p H E p H E p Ep E H H H

p H


Bayesian Fusion

iii APABP

APABPBAP

)()|(

)()|()|(

•Target location and tracking •Search for formation of targets in a region•…

Fusion Methodology


Example: Two sensor data fusion

x: to be identified (e.g. aircraft)

2,1101 iyYY iii

Latest data set Old data set Current measurement

ionnormalisatYxPYxP

YYxPYxPYxPYYxP

)|()|(

)|()|()|()|(

20

10

20

10

21

112

11

1

Fusion Methodology


Bayesian Inference

Some limitations:

-no representation of ignorance is possible-prior probability may be difficult to define-result depends on choice of prior probability-it assumes coherent sources of information-adequate for human assessment (more difficult for machine-driven decision making)-complex with large number of hypotheses-poor performance with non-informative prior probability (relies on experimental data only)


Dempster-Shafer Evidental reasoning

Fusion Methodology

Often described as an extension of the probability theory or a Generalization of the Bayesian inference method.

Frame of discernment Θ={X0, X1 , …Xn}Mass probability (basic probability assignment (bpa)) : m(X)

0 ( ) 1

( ) 1

( ) 0

i

X

m X

m X

m



Fusion Methodology

: 2 [0,1]

( ) ( )Y X

Bel

Bel X m Y for each X

Bel(X) : the degree of support

Properties of the belief function:

( ) 1

( ) 0

0 ( ) 1

( ) ( )

( ) ( ) 1

Bel

Bel X if X

Bel X if X and X

Bel X m X for each X containing only one element

Bel X Bel X


Dempster-Shafer Evidential reasoning

Fusion Methodology

Dempster rule of combination:

1 2

1 2 3 1 1 2 2( ) ( ) ( ) ( )X X Z

m m Z m Z K m X m X


…


Geometrical representation of Dempster rule of combination

m1,2 (Xi,Xj)

m2(Xn)

m2(Xj)

m2(X1)

1

0

0 1

m1(X1) m1(Xi) m1(Xn)

Fusion Methodology



0 Belief Disbelief 1

Plausibility

Incertitude

[Bel(X),Pls(X)] Decision

[0,1] Total ignorance, no belief in support of X

[1,1] Proposition X is completely true

[0,0] Proposition X is completely false

[0.4,1] Partial belief, tends to support X

[0,0.7] Partial disbelief, tends to refute X

[0.3,0.5] Both support and refute X


Dempster-Shafer FusionGives a rule for calculating the confidence measure of each state,based on data from both new and old evidence.Assigns its masses to all of the subsets of the entities thatcomprise a system

Fusion Methodology

•Mobile robot map building (e.g. occupancy grid)

)()()()(1

)()()()()()()(

OmEmEmOm

OmUmUmOmOmOmOm

osos

ososos

{occupied, empty, unknown} :{O,E,U}m:confidence in each elementms:confidence from sensorsmo:confidence from old existing evidence



Some features:

-An overestimation of the final assessment can occur

-Small changes in input can cause important changes in output

-High efficiency with bodies of evidence in pseudo-agreement

-Lower efficiency with bodies of evidence in conflict


Bayesian Fusion vs. Dempster-Shafer Fusion

Bayes: •Works with probabilities, numbers that reflect how often an event will occur•Less calculations.

Dempster-Shafer:•Considers a space of elements that each reflect not what Nature chooses, but rather the state of our knowledge after making a measurement.•Calculations tend to be longer.•Allows more explicitly for an undecided state of our knowledge.(in military it is sometimes far safer to be undecided than to decide wrongly)•Sometimes fails to give an acceptable solution.

Fusion Methodology


Fuzzy Logic Inference Technique

Is very flexible and there is no universal rule of formalism which can be associated with it.

Fuzzy logic evaluates qualitatively a signal from a sensor and fuzzy sets associate a grade (numerical value) to each element.

Element Associated value Associated reliability

Signal high [1.0,0.7] Certain

Signal medium [0.7,0.3] Uncertain

Signal low [0.3,0.0] Incorrect

Typical associated values for different elements in fuzzy logic



A multilevel system to handle vagueness:

-sensor level-data fusion level-reasoning level

Produce information

Integrate information

Generates a decision making use of artificialintelligent systems

Fuzzy logic methods can be very useful to represent uncertainty from multiple sensors and to handle vagueness.



Combining information from multiple images to improve classification accuracy of a scene where images are processed at the pixel level using segmentation algorithm .

Can be performed for… image processing and image smoothing image segmentation to combine information perceived by visual sensors.

Fusion center


Artificial Intelligence

AI techniques developed for data association make use of expertsystems and neural networks

Artificial Neural networks (NNs) are software simulated processing units or nodes, which are trained in order to solve problems.

NNs can be very useful to solve problems in applications where it is difficult to specify an algorithm.They are composed of interconnected nodes that act as independent processing units



0

n

i ii

y f w x

node

Input xi

Weights wi

A two-layer neural network, Perceptron

Output signal



Some NN applications in data fusion

-For sensor data fusion for detection and correct classification of space object maneuvers observed by radar of different frequencies and resolution.

-Used in decision systems for target tracking, object detection, recognition and classification in defence applications

-In image processing operations such as filtering and segmentation

-To select matching pixel based fusion from sensors for robotics application. To perform pixel-to-pixel image association for object identification

-Applied to non-destructive examination for eddy current signal classification and automatic tube inspection,defect characterisation, classification of weld defects and signal interpretation.

Data Fusion- A reviewA Review on Decision Fusion Strategies

Acknowledgements:

• This powerpoint presentation was prepared by Miss Mahdavi and Miss Bahari former M.Sc. Students at School of ECE , University of Tehran in Dec. 2005 where here is highly appreciated.

data fusion- a review. layout 1-benefits of multisensor devices 2-typical sensors used in data...

Documents