Tracking in High Target Densities Using aFirst-Order Multitarget Moment Density

Ronald Mahler, Ph.D.Lockheed Martin NE&SS Tactical Systems

Eagan, Minnesota, USA651-456-4819 / ronald.p.mahler@lmco.com

IMA Industrial Seminar Series, University of MinnesotaOctober 4, 2002

Problem

• It is not always feasible—or necessary—to detect, track, and identify individual targets with accuracy

– large-formation tracking: track density is so large that only knowledge of overall geometrical target distribution is feasible

– group tracking: detection and tracking of force-level objects (brigades, battalions, etc.) is of greater interest than detection and tracking of their individual targets

– cluster tracking: a few Targets of Interest (ToI's) are obscured by a “multi-target background” of low-priority targets, which is of interest only because it might contain a ToI

Approach: “Bulk Tracking”

• Conventional: detect & track all targets

– most feasible: where density is lowest (outskirts of a formation)

– least feasible: where inter-track confusion is greatest because density is highest (i.e., where ToI's are most likely to be found)

• Bulk tracking: detect & track bulk target groupings first, then sort out individual targets as data permits

– estimate what is knowable given current data quantity / quality (at

first, only bulk multitarget behavior) instead of attempting to estimate what cannot be known until sufficient, and sufficiently good, data has been collected (i.e., individual target behavior)

– resolve individual targets out of the "multitarget background" as

separate identifiable tracks only as additional information about them is accumulated over time

Approach (Ctd.): 1st-Order Multitarget Moment Filter

single-sensor, single-target Bayes filter

Bayes-optimalapproach

Bayes problemformulation

computationalstrategy

multi-sensor, multi-target Bayes filter

state of system = random

state vector

state of system = random state-set(point process)

first-moment filter(e.g. -- filter)

first-moment filter(“PHD” filter)

multi-sensor,single-target

multi-sensor,multi-target

Topics

1.    First-order moment filtering2. Multitarget first-order moments: the “PHD”3. Multitarget first-order moment filtering4. Simulations

xk|k xk+1|k+1^^-- filter

fk|k(xk|Zk) fk+1|k(xk+1|Zk)optimal Bayes filter

randomobservations z

produced by target

randomstate-

vector, x

target motion

state space

observation space

1st- and 2nd-Order Moment Filters

fk+1|k+1(xk+1|Zk+1)

xk+1|k^

time-updatestep

data-updatestep

Xk+1k+1

Xk|k

zk

zk+1

how can we extend this reasoning to multitarget systems?Pk|k Pk+1|k Pk+1|k+1

^ ^ ^Kalman filter

Xk+1|k

Multi-Sensor/Target Problem: Point Process Formulation

sensors

targets

random

obser-vation-

set

diverse observations

“meta-sensor”

“meta- target”

reformulate multi-object problem as generalized single-object problem

“meta-observation”

random state-set

multitarget state

X = {x1,…,xn}

multisensor state

X*k = {x*1,…,x*s}

multisensor-multitargetobservation

Z = {z 1,…,z m}

random object-set

random density random countingmeasure

S

(x) = y(x) N(S) = | S|

Geometric Point Processes (= Random Finite Sets)

three equivalent formulations of a (multidimensional) simple point process

preferred by mathematicians

preferred byphysicists

“engineering-friendly”(multi-object systems are modeled

as visualizable random images)

y

sum the Dirac deltas concentrated at the

elements of

Integral and Derivative for Simple Point Processes

][][lim][

0

hFghFh

g

F

nn

n ddffXXf xxxx 11

1 }),...,({)()(

Set integral:

Functional (Gateaux) derivative:

][][lim][

0

hFhFh

F

x

x

Dirac delta function

physics:“functionalderivative”

Probability Law of a Geometric Point Process

][E][ |1

|1kkhhG kk

)Pr(][)( |1|1|1 SGS kkSkkkk 1

)Pr()(1)( |1|1|1 SSS kkc

kkkk

probability generatingfunctional (p.g.fl.)

beliefmeasure

plausibilitymeasure

(= Choquet functional)

X

X hhx

x)(

)"Pr(" ]0[)( |1|1

|1

1

XG

Xf kkkk

n

kk

n

xx multitarget

posteriordistribution

(= Janossy densities)

discrete-space notation used only for claritication

X = {x1,…,xn}

probability that all targets are in S

probability that some target is in S

Frechét functionalderivative

h = bounded real-valued test function

xx

gg

GG

fk|k(X|Z(k))multitarget posterior

multisensor-multitarget measurements: Zk = { zzm(k)}

individual measurementscollected at time k

multitarget state

Z(k)Z ,...,Zk

fk|k(X|Z(k))X = 1normality condition

fk|k(|Z(k)) (no targets)

fk|k(x|Z(k) (one target)

fk|k(xx2|Z(k)) ( two targets)

…fk|k(xxn|Z(k)) (n targets)

measurement-stream

Multitarget Posterior Density Functions

fk|k(X|Z(k),U(k-1)) fk+1|k+1(X|Z(k+1),U(k))

randomobservation-

sets producedby targets

multi-target motion

state space

observation space

fk+1|k(X|Z(k),U(k-1))

The Multi-Sensor/Target Recursive Bayes Filter

time-update data-update

k|k k+1|k+1k+1|kevolving random state-set

Zk+1

X*k+1

future observation-set (unknowable)

new sensor state-set (to be determined)

fk+1|k(Y|X) fk+1(Zk+1|X, X*k+1)multitarget Markov

transition density multisensor-multitarget likelihood function

(target-generated observations & clutter)

p1D(x,x*),…, ps

D(x,x*)sensor FoVs

recursive Bayes filter

What is a Multitarget First-Order Moment?

Naïve concepts of multitarget expected value fail

So: we must resort to “indirect” multitarget moments

X = {x1 ,…, xn} (X) = ({x1 ,…, xn})

vector spacesubset space well-behaved function

(X Y) = (X) + (Y) if X Y = (disjoint unions aretransformed into sums)

“Indirect” multitarget expectation:

E[()] expected value of random vector ()corresponding to random set

() = y(x) = (x) random state-set

random density

random counting measure

S

Two Possible Choices for

() = | S|y

sum the Dirac deltas concentrated at the

points of

three different notations for a simple point process

first-moment density

“probability hypothesisdensity” (PHD)

first-momentmeasure

S

Indirect Expected Values of a Random State-Set

D(x) = E[(x)] M(S) = E[| S|]

PHD for a Discrete State Space (Picture)

state space (discrete)x0

three-state instantiations

{x, x2, x3} of

two-state instantiations

{x, x2} of

one-state instantiations

{x1} of

four-state instantiations

{x, x2, x3, x4} of

D(x0) = Pr(x0 ) = p1 + p6 + p9 + p11 + p16

p1

p2

p3

p4

p5

p6

p7

p8

p9

p11

p12

p13

p14

p15

p17

p10

p16

probability of the hypothesis: “the multitarget system

contains a target with state x0“

The PHD (Ctd.)

x0state space

D(x0) = expected target

density at x0

PHD magnitude

S

D(x)dx = expected number

of targets in SS

If state space is discrete then the PHD is a fuzzy membership function (fuzzy subset of target states)

five peaks (largest target densities)correspond to locations of seven

partially resolved, closely spaced targets

X

target density,

D(x)

Example of a PHD on 2-D Euclidean Space

cluster

fk|k(X|Z(k)) fk+1|k+1(X|Z(k+1))

randomobservation-

sets Z producedby targets

randomstate-set multi-target motion

state space

observation space

fk+1|k(X|Z(k))

five targets three targets

First-Order Multarget Bayes Filtering

time-updatestep

data-updatestep

k|k k+1|k+1k+1|k

random

state-set

Dk+1|k(x|Z(k)) Dk+1|k+1(x|Z(k+1))Dk|k(x|Z(k))

multitargetBayes filter

1st-moment“PHD” filter

PHD Functional Derivative Formula

]1[)(x

x

G

D

]1[)( |1|1

x

x

kk

kk

GD

]1[)( 1|11|1

x

x

kk

kk

GD

PHD Filter Assumptions: Motion Model

time-step k time-step k+1

fk+1|k(y|x)probability that target will have statey if it had state x

Xbk+1|k(X|x)

x y

1 dk+1|k(x)

x

probability that target will vanish if it had state x

xprobability that target will spawn atarget-set X if it had state x

X

probability that a target-set X will appear in scene

bk+1|k(X)

all target motions are assumed statistically independent

death

creation

spawn

motion

PHD Filter Assumptions: Sensor Model

fk(z|x)

x z

probability that target will generateobservation z if it has state x

1. observations and clutter are statistically independent2. multitarget posteriors are approximately Poisson (need high SNR):

x

pD probability that target will not generate an observation (assumed state-independent)

ck+1|k(Z) probability that a set Z = {zzm}

of clutter observations will be generated; Poisson false alarms:

ck|k(Z) = e- k|kn ck|k(z1) ck|k(zm)

likelihood

misdetection

clutter

fk|k(X|Z(k)) e-N Nk|k sk|k(x1) sk|k(xn)

state space observation space

X = {x1,…,xn}

Dk+1|k(y|Z(k)) = bk+1|k(y) +Dk+1|k(y|x) Dk|k(x|Z(k))dx

PHD Time-Update Step

PHD from previous

time-step

term for spontaneoustarget births

= PHD of bk+1|k(X)

time-updated

PHD

Nk+1|k = dk+1|k(x) + nk+1|k (x) Dk|k(x|Z(k)) dx

expected number of targets spawned by x

Dk+1|k(y|x) = dk+1|k(x) fk+1|k(y|x) + bk+1|k(y|x)

Markovtransition

PHD

probabilityof targetsurvival

term for targets spawned

by existing targets= PHD of bk+1|k(X|x)

Markov transi-tion density

Nk+1|k = Dk+1|k(y|Z(k))dy

predicted expected number of targets

nk+1|k (x) = bk+1|k(y|x)dy

Dk+1|k+1(x|Z(k+1)) zZk+1

Given new scan of data, Zk+1 = {zzm}

k+1ck+1(z)+pDDk+1(z)pDDk+1(z)

Dk+1(x|z) + (1-pD) Dk+1|k(x|Z(k+1))

Dk+1(z) = fk+1(z|x) Dk+1|k(x|Z(k+1))dx

Dk+1(x|z) = f(z|x) Dk+1|k(x|Z(k+1))

Dk+1(z) Nk+1|k+1= Dk+1|k+1(x|Z(k+1))dx

expected number of targets after new scan

Nk+1|k+1 zZk+1

k+1ck+1(z)+pDDk+1(z)pDDk+1(z)

+ (1-pD)Nk+1|k

Bayes-updated PHD

single-observation Bayes update of PHD

predicted PHD (fromprevious time-step)

averageno. of

false alarms

distributionof false alarms

predicted expected number of targets(from previous time-step)

sensor likelihood function

PHD Filter Bayes Update Step

Proof, I: Transform PHD into p.g.fl. Form

1. Data-updated multitarget posterior:

YZYfYZf

ZXfXZfZXf

kkkkk

kkkkk

kk )|()|(

)|()|()|(

)(|111

)(|11)1(

1|1

YZYfYZf

XZXfXZfhZhG

kkkkk

kkkk

X

kkk

)|()|(

)|()|(]|[

)(|111

)(|11)1(

1|1

YZYfYZf

XZXfXZfh

ZDk

kkkk

kkkk

X

kkk

)|()|(

)|()|(

)|()(

|111

)(|11

)1(1|1

xx

2. p.g.fl. of multitarget posterior:

3. PHD of multitarget posterior:

Proof, II: Transform PHD into p.g.fl. Form

4. Define Bayes characterizing functional (B.c.fl.):

]1,0[

]1,0[

)|(

1

1)1(1|1

m

m

gg

mg

hgg

hmg

kkk F

F

ZD

zz

xzzx

5. PHD in B.c.fl. form:

XZXfXgGh

ZXZXfXZfghhgF

kkkk

X

kkkk

ZX

)|(]|[

)|()|(],[

)(|11

)(|11

p.g.fl. of multitarget measurement density

Proof, III: Choice of Likelihood Function

6. Multitarget likelihood function:

11 ][1 ))()()(1(]|[

kk g

XgDDk epppXgG

x

xxx

7. Simplified B.c.fl.:

)]1([

)|()1(

)|(]|[],[

|1][

)(|1

][

)(|11

11

11

gDDkkg

kkk

XgDD

Xg

kkkk

X

ppphGe

XZXfppphe

XZXfXgGhhgF

kk

kk

zxzzx dfgpg )|()()( zzz dcgg )()(][

each target generates at most one observation with probability pD

Poisson false alarm process w/ spatialdistribution c(z)

Proof, IV: Simplification

8. Assume predicted multitarget posterior is Poisson:

9. Final simplified B.c.fl.:

][

|1 ][ gpkk

sehG

][)]1([][exp],[ 11 gDsDskk phppphpghgF

10. Inductively determine denominator and numerator of B.c.fl. form of PHD using functional derivatives

xxx dshhps )()(][

Dk+1|k(y|Z(k)) = fk+1|k(y|x) Dk|k(x|Z(k))dx

Special Case

Nk+1|k = Nk|k

Dk+1|k+1(x|Z(k+1)) zZk+1

Dk+1(x|z)

Nk+1|k+1 = |Zk+1|

time-prediction

data update

assuming:no target deaths or births

assuming:no misdetections or false alarms

Following example is based on above assumptions:Example: bulk tracking of two clusters of three more separated targets

Example: Six Targets, Two Clusters

= actual target locations = noisy observations = estimated target locations

x

y

Targets are further apart. Increased data quality (relativeto filter resolution) allows filter to resolve targets as wellas track the two formations.

Two Clusters: PHD at 3rd Observation

direction ofmotion

PHDvalue

x y

Since the PHD filter estimatesthe number N of targets, a peakextraction algorithm is used tolook for the N tallest peaks

Two Clusters : PHD at 9th Observation

Two Clusters: PHD at 17th Observation

Two Clusters: PHD at 27th Observation

Two Clusters: PHD at 31st Observation

Implementation Based on Particle-System Filters

- Non-restrictive with respect to measurement models

- Very general continuous-state Markov models– e.g. heavy-tail models, non-smooth models

- Very strong, general guaranteed-convergence properties

for every observation sequence, particle distribution converges a.s. to posterior

- Computational order: O(pd) (low-SNR detection), O(p) (low-SNR tracking)p = no. particles, d = dimensionality, N = pd = no. of unknowns

- LMTS is co-developing these filters with U. Alberta (Prof. M. Kouritzin)

posterior, time k posterior, time k+1

“particles”= samples

Deltafunctions

Branching Particle-System Filtersfast and rapidly convergent

least-probable particles“die” in next time-step

most-probable particles “spawn” new particles

in next time-step

Simple Simulation: Multitarget Tracking in Clutter

May Jun Jul Aug Sept Oct Nov Dec

2002Jan Feb Mar Apr May

2003

200 scans collected

average of 120 clutter observationsper scan

targets cross at 40th scan

26% increase in RMS localizationaccuracy overtarget-generatedobservations

velocity alsotracked successfully

first target second target

Summary / Conclusions

• First-order moment filtering provides a potential means of tracking clusters of emitters until enough information has been accumulated to begin extracting RVs

• Computational power can be shifted from low-interest regions of the PHD to regions that may contain targets of interest, to allow “peaking up” of those targets

• Efficient computational implementation requires particle-systems methods