a bayesian em algorithm for optimal tracking of a ...vikramk/lkh02.pdfa bayesian em algorithm for...

Signal Processing 82 (2002) 473–490www.elsevier.com/locate/sigpro

A Bayesian EM algorithm for optimal tracking of amaneuvering target in clutter

Andrew Logothetisa, Vikram Krishnamurthyb; !, Jan HolstcaEricsson, Stockholm, Sweden

bDepartment of Electrical and Electronic Engineering, University of Melbourne, Parkville, Vic. 3052, AustraliacDepartment of Mathematical Statistics, Lund Institute of Technology, Box 118, S-221 00 Lund, Sweden

Received 9 June 2000

Abstract

The di!culty in tracking a maneuvering target in the presence of false measurements arises from the uncertain originof the measurements (as a result of the observation=detection process) and the uncertainty in the maneuvering commanddriving the state of the target. Conditional mean estimates of the target state require a computational cost which isexponential with the number of observations and the levels of the maneuver command.In this paper, we propose an alternative optimal state estimation algorithm. Unlike the conditional mean estimator,

which require computational cost exponential in the data length, the proposed iterative algorithm is linear in the datalength (per iteration). The proposed iterative o"-line algorithm optimally combines a hidden Markov model and a Kalmansmoother—the optimality is demonstrated via the expectation maximization algorithm—to yield the maximum a posterioritrajectory estimate of the target state.The algorithm proposed in this paper, uses probabilistic multi-hypothesis (PMHT) techniques for tracking a single

maneuvering target in clutter. The extension of our algorithm to multiple maneuvering target tracking is straightforwardand details are omitted. Previous applications of the PMHT technique (IEEE Trans. Automat. Control, submitted) haveaddressed the problem of tracking multiple non-maneuvering targets. These techniques are extended to address the problemof optimal tracking of a maneuvering target in a cluttered environment. ? 2002 Elsevier Science B.V. All rights reserved.

Keywords: Expectation maximization algorithm; Maneuvering target tracking; PMHT; Clutter

1. Introduction

The problem of estimating the state (position, velocity and heading) of a moving target (or an object) fromnoisy observations is known as the target tracking problem [2,4]. The target tracking problem is of signi#cantimportance to surveillance and has been extensively studied in numerous books and papers (see [2,4] andreferences therein). The aim of a tracking system is to ascertain information about multiple targets usingmeasurements or readings from a number of sensors. The sensors provide noisy observations of the targets’

! Corresponding author. Tel.: ++61-3-9344-6702; fax: ++61-3-9344-6678.E-mail address: [email protected] (V. Krishnamurthy).

0165-1684/02/$ - see front matter ? 2002 Elsevier Science B.V. All rights reserved.PII: S 0165 -1684(01)00198 -0

474 A. Logothetis et al. / Signal Processing 82 (2002) 473–490

states. From a history of these noisy observations, it is desired to obtain information pertaining to the motionof the moving target, such as velocity, position, heading and other parameters of interest.The di!culty in tracking a maneuvering target in the presence of false measurements [2,4] arises from the

uncertain origin of the measurements (as a result of the observation=detection process) and the uncertaintyin the maneuvering command driving the state of the target. Computing conditional mean estimates of thetarget state require a cost which is exponential in the number of observations and the levels of the maneuvercommand. In the target tracking literature, several mixture reduction techniques have been developed whichresult in numerically e!cient trackers.

1.1. Summary of contributions

In this paper, we propose an alternative optimal state estimation algorithm to conditional mean estima-tion. Our proposed scheme is an iterative expectation maximization (EM) algorithm. This algorithm optimallycombines a hidden Markov model (HMM) Smoother and a Kalman smoother (KS), to yield the modaltrajectory (or equivalently the maximum a posteriori (MAP) estimate [10]) of the target state given a se-quence of measurements. The EM algorithm presented in this paper has a computational cost of O(s2 K)per iteration, where K denotes the total number of scans and s is the product of the number of observa-tions per scan times the quantized levels of the input control driving the maneuvering target. Note that thealgorithm presented in this paper is o!-line. Such o"-line algorithms are practical for slowly moving tar-gets such as merged and submerged objects (e.g. ships and submarines), see [13] for other o"-line trackingalgorithms.The EM algorithm has widely been used in the engineering and statistical literature as an iterative numerical

procedure for computing maximum likelihood (or MAP) parameter estimates of partially observed models[6,19]. The EM has the appealing property of ensuring that successive iterations always result in a parameterestimate with increased likelihood (or posterior).The main contribution of this paper is to use the EM algorithm to compute MAP state estimates, i.e. to

compute the MAP trajectory estimates of the maneuvering target. Similar EM algorithms have recently beenproposed for state estimation. For example in [14], an EM-based approach is proposed to estimate discreterandom signals for multi-user detection in CDMA communication systems. The multi-user receivers treat thesignals of the interfering users as hidden data. The advantage of this approach is to signi#cantly reduce thecomputational cost from exponential (via conventional ML approach) to linear in the number of informationbearing users. In [9], the EM algorithm is applied to the problem of sequence detection when synchronization isnot present. The synchronization (or the timing error) is treated as the missing data. The E-step is implementedusing a Kalman #lter, which yields estimates of the timing error. The M-step is implemented using the Viterbialgorithm, which yields the maximum likelihood sequence estimates of the transmitted discrete signals. TheE and M steps are then repeated until convergence [9].

1.2. Computational requirements

Let m be the number of measurements per scan and u be the input control which belong to a #niteset Su , {1; 2; : : : ; Nu}. Let p be the dimensions of the transition matrix A. Each iteration (or pass) ofour EM-based state estimator requires O(m2N 2u K) + O(p3K) computations for K scans; O(m2N 2u K) is thecomputational cost of the HMM smoother (see [16]), while the computational cost of the Kalman smoother isO(p3K) (see [1]). Recall that computing conditional mean state estimates requires O((Nmu)K) computations.

A. Logothetis et al. / Signal Processing 82 (2002) 473–490 475

1.3. Proof of convergence

The appealing property of EM is that the posterior density increases within each iteration. The proof ofconvergence of the our EM algorithm is very similar to that in [12] and hence omitted. As in [20] the keystep is to use Zangwill’s global convergence theorem.

1.4. Related works

Target tracking papers techniques based on EM algorithms are presented in [11,13,15,17], see also [4,Section 7:7:3] where batch maximum likelihood estimation methods are presented. In [15], the EM algo-rithm is used to compute the MAP sequence estimates of the measurement-to-model assignments for a singlenon-maneuvering target observed in clutter when multiple observation processes are used. At each measurementscan, the origin of each measurement (whether it is a false detection, or belonging to a particular observationprocess) is unclear. Hard measurement-to-model assignments are computed using the EM algorithm and asa by-product of the EM conditional mean state estimates (conditioned on the measurements and the hardassignments) are computed. In [11], the problem of maneuvering target tracking in a clutter free environmentwith unity detection probability, is addressed. The EM algorithm is used to compute hard maneuver com-mand assignments. As a by-product of EM, target state estimates are computed based on the measurementsand the hard input control sequence estimates. As one may expect, such an approach (i.e. making statisticaldecisions) will yield larger state root mean square (RMS) estimation errors [18]. Thus, it is imperative toconsider an alternative optimization method, which will yield optimal MAP state estimates. Such an approachis presented in [18], where the EM algorithm is applied to the problem of multi-target tracking, when thedata origin is unclear, i.e. it is not known which measurement originated from which target (this problem isknown as the data-association problem). The missing data are the associations—discrete random variables.The E-step computes (estimates) the a posteriori association probabilities, while the M-step, implemented viaa KS, computes the MAP state sequence estimates of the target trajectories.Our proposed algorithm borrows many ideas from the algorithm presented in [18], known as the proba-

bilistic multi-hypothesis tracking (PMHT) algorithm. Previous applications of the PMHT technique [18] haveaddressed the problem of tracking multiple non-maneuvering targets. The algorithm presented in this papercan be considered as an extension to the PMHT addressing the problem of maneuvering targets. While, inthis paper we address the problem of a single maneuvering target in clutter, the extension to multi-targetmaneuvering tracking is straightforward and details are omitted (in the multi-target case a larger dimensionstate space is required).Finally, [12] presents an iterative EM algorithm for computing the MAP sequence estimate for a general

class of jump Markov linear systems. The technique in [12] is extended here to deal with measurements ofuncertain origin (i.e. the measurements are not labeled—it is not known which of the data are clutter (falsemeasurements) and which (if any) are the true measurements).

1.5. Summary of our proposed algorithm

We use the EM algorithm to yield the MAP target state sequence estimates. The EM algorithm convergesto a local stationary point in the state likelihood surface f(Xk |Zk), see (16). The E-step computes the aposteriori association probabilities and the a posteriori control input probabilities. The M-step, implementedvia a KS on a modi#ed state-space model, computes the MAP sequence of the target states. The E and Msteps are then repeated until convergence.


1.6. Limitations of our algorithms

1. O"-line algorithm: The proposed EM target state estimator is implemented o"-line. That is, all measure-ments must be collected and then processed to yield state estimates. It is possible to implement an onlineversion of the algorithm by replacing the Kalman and HMM smoother with #lters. This is closely related tothe interacting multiple model probabilistic data association (IMM-PDA) estimator proposed by Bar-Shalom[5]. This is the subject of future work.2. Local maxima: It is well known that the EM algorithm converges to the nearest stationary point of the

likelihood function (or the posterior density). One way of overcoming this is to consider several initial values.From the computed #nal (or converged) estimates one should discard the estimates that yield the smallestlikelihood. In recent work [7,8] we have derived Markov Chain Monte Carlo (MCMC) and particle #lteringmethods for MAP state estimation of Jump Markov linear systems—such methods can be extended to themaneuvering target case. The MCMC algorithms in [7] are globally convergent.The paper is organized as follows: In the following section we formally present the estimation objectives

and outline the procedure on meeting these objectives. In Section 3 we give the details of our expectationstate sequence maximization algorithm for estimating the state of maneuvering target in clutter. Finally, inSection 4 we present illustrative examples of our proposed algorithm.

2. Problem formulation

We present the signal model for the dynamics of the maneuvering target and the associated measurementsystem. The signal model is similar to that considered in [2,4]. In Section 2.1 we present the dynamics of amaneuvering target and in Section 2.2 the observation process is given. The observations at each time instantconsist of true and false (clutter) measurements. In Section 2.3 the estimation objective is outlined and Section2.4 presents the procedure for meeting the estimation objective.

2.1. Maneuvering target dynamics

Let k !K, {0; 1; 2; : : : ; K} denote discrete time up to time index K .

Assumption 2.1. A single target is present for all k !K.

Thus, the probability of target existence (whether the target is present or not) is always equal to one duringtime k !K.The state of the target at time k is denoted as x(k) and evolves according to the following jump Markov

linear system (the unknown control input u(k) is modeled as a #nite-state, #rst-order Markov chain)

x(k + 1) = Ax(k) + Bu(k + 1) + v(k); k = 0; 1; : : : ; K; (1)

where1. x(k) , (rx(k); rx(k); ry(k); ry(k))! is the target state vector at time kT and T is the sampling interval.

x(k) represents the target’s position rx(k) and velocity rx(k) in the x direction and the target’s position ry(k)and velocity ry(k) in the y direction. (·)! denotes the matrix transpose operator.2. A is the state matrix of the jump linear system (1) and is given by

A=

!

"

"

"

#

1 T 0 00 1 0 00 0 1 T0 0 0 1

$

%

%

%

&

: (2)


3. v(k) is the process noise, assumed to be a zero mean white Gaussian sequence, such that

E{v(k)}= 0 for all k; (3)

E{v(k)v!(l)}= !klQ with Q¿ 0; (4)

where E{·} is the expectation operator and !kl is the Kronecker delta function (!kl = 1, if k = l and zerootherwise). The positive de#nite matrix Q denotes the noise covariance matrix at time kT .4. u(k) , (ux(k); uy(k))! is the maneuver command, an additional unknown input control to the target

dynamics described by (1). The random process u(k) is modeled as a discrete-time, time-homogeneous,Nu-state (with states u(k)! {qi; i!Su , {1; 2; : : : ; Nu}}), #rst-order Markov chain with known transitionprobabilities:

pu(i; j), Pr{u(k + 1) = qj|u(k) = qi} (5)

" i; j!Su = {1; 2; : : : ; Nu}. Denote the initial probability distribution as pu(i), Pr{u(0) = qi}; " i!Su. Thetransition probability matrix [pu(i; j)], is a Nu#Nu matrix, with elements satisfying 06pu(i; j)6 1, "i; j!Su,and

'Nuj=1 pu(i; j) = 1, for each i!Su.

5. The matrix B has the following structure:

B=

!

"

"

"

#

T 2=2 0T 0

0 T 2=20 T

$

%

%

%

&

: (6)

6. We assume the initial target state is Gaussian distributed with known mean and variance, i.e.

E{x(0)}= x0; (7)

E{(x(0)$ x0)(x(0)$ x0)!}= P0: (8)

Remark 2.1. The motion of the target; as described in (1); is subject to small random perturbations v(k)occurring every T seconds. These perturbations remain constant between the sampling interval; but changein magnitude and direction randomly from one interval to the next. In addition to v(k) the target is alsosubject to an unknown input acceleration u(k); which is modeled as a #rst order #nite-state Markov chain.u(k) denotes the unknown (to be estimated) control input to the target state of Eq. (1).

2.2. Observation process

In the presence of random interference, the output of an observation system consists of a set of measurementswith uncertain origin. The measurements are outputs of a detection process and consists of [2,4]:1. True measurements, if detected from the detection process.2. False measurements, which are due to random false alarms in the detection process.The true and false measurements are assumed to have independent statistical (spatial and temporal) propertieswhich must be exploited to di"erentiate between them.

Assumption 2.2. The observation process can yield at most one true measurement at each time.

Assumption 2:2 is relaxed in [18]. This yields simpler EM algorithms when tracking multiple targets inclutter.The detection process may not always detect the target.


De!nition 2.1. Let the binary random process d(k)! {0; 1} denote the detection of the target at time k. Heredk = 1 (=0) denotes the target detected (not detected) at time k.

d(k) is a sample from an independent and identically distributed (iid) binary random process. Let Pd denotethe detection probability, that is

Pd = Pr{target detected by the measurement system} (9)

with 0¡Pd6 1.True measurements: If the target is detected, i.e. d(k) = 1, then the “true” measurement or observation is

given by the following linear model:

ytrue(k) = C x(k) + w(k) if d(k) = 1; (10)

where ytrue(k)!Rny is the observation at time k, w(k) % N (0; R)!Rnw is a zero-mean white Gaussianmeasurement noise sequence with positive de#nite covariance matrix R and C is a matrix of appropriatedimensions.False measurements: The distribution and the number of false measurements are given by the following

two assumptions which are extensively used to model clutter or false measurements in a radar or sonar targettracking systems [2,4].

Assumption 2.3. The false measurements at time k are uniformly and independently distributed in the volumeV & Rny of the measurement space. Furthermore; the false measurements are independently distributed fromone time to the next and also mutually independent of the true measurements.

Assumption 2.4. The number of false measurements at each time k is Poisson distributed and independentfrom one time to the next. The probability mass function "F(m) for detecting m false measurements at timek is given by

"F(m) = Pr{m{false measurements detected}= e"#V (#V )m

m!; (11)

where # is the spatial density of false measurements and denotes the average number of false measurementsper unit volume. Thus; #V is the expected number of false measurements in the measurement volume V .

De!nition 2.2. Let m(k) denote the number of measurements at time k and Z(k) = {zi(k)}m(k)i=1 denote the setof m(k) measurements at time k.

De!nition 2.3. Let a(k)! {0; 1; : : : ; m(k)} denote the measurement-to-target association at k.

The measurement-to-target associations are de#ned as follows: if a(k) = i for some i in {1; 2; : : : ; m(k)},then zi(k) = ytrue(k), i.e. the ith measurement from Z(k) is the true measurement (that is, the target wasdetected, and consequently d(k)=1). On the other hand, if a(k)=0, then all measurements in Z(k) are falsemeasurements (thus d(k)=0). The events (a(k)¿ 0 and d(k)=0) or (a(k)=0 and d(k)=1) are not feasible.Clearly, the detection d(k) of the target at time k and the measurement-to-target association a(k) are relatedas follows:

d(k) =

(

0 if a(k) = 0;1 if a(k)! {1; 2; : : : ; m(k)}:

(12)


The associations a(k) are iid discrete random variables, independent from one time to the next, with (see[2] p. 168)

Pr{a(k) = i}=(

cPd"F(m(k)$ 1)=m(k) for i! {1; 2; : : : ; m(k)};c(1$ Pd)"F(m(k)) for i = 0;

(13)

where c, [Pd"F(m(k)$1)+(1$Pd)"F(m(k))]"1 is a normalizing constant. "F(m(k)$d(k)) is the probabilitymass function of having m(k)$ d(k) false measurements.

Assumption 2.5. The process noise; the observation noise; the initial target state and the false measurementsare uncorrelated; thus

E{x(0)v!(k)}= 0; E{x(0)u!(k)}= 0;

E{x(0)w!(k)}= 0; E{v(l)w!(k)}= 0;

E{zi(k)x!(l)}= 0 for k; l! {0; 1; : : :}; and i! {False measurements}: (14)

Assumption 2.6. The model parameters $ de#ned below are assumed known

$, {pu(i); pu(i; j); A; B; x0; P0; Q; C; R; #; V; Pd; i; j!Su}: (15)

If these parameters are not known, they can be estimated via a similar EM-type algorithm. However, issuesregarding the identi#ability of such models are beyond the scope of this paper.

Notation. Denote the sequence of observations {Z(1); : : : ; Z(K)} as ZK . Also; let XK ; UK ; AK denote the statesequence {x(1); : : : ; x(K)}; the control sequence {u(1); : : : ; u(K)} and the sequence of measurement to targetassociations {a(1); : : : ; a(K)}; respectively.

2.3. Estimation objectives

Assuming the model parameters $ in (15) are exactly known, and given the measurement sequence ZKgenerated by the signal model described in Sections 2.1 and 2.2, in this paper we compute optimal (in aMAP sense) estimates of XK by maximizing the probability density function of XK conditioned on ZK , i.e.

XMAP = argmaxXKf(XK |ZK); (16)

where XMAP denotes the modal state trajectory estimate of XK . In Section 3, the EM algorithm will be usedto compute XMAP.

2.4. Expectation maximization algorithm for target tracking

The EM algorithm [6,19] is normally used as an iterative parameter estimation scheme for extractingthe mode of the likelihood function or extracting the mode of the posterior distribution for incomplete datamodels. In this section we establish our notation for using the EM algorithm to compute MAP target trajectoryestimates.


Let X K denote the target state sequence estimate given the observation sequence ZK . Let M , (Mobs; Mmis)denote the complete data from time 1 up to time K , where Mobs and Mmis are the observed and missing data,respectively. That is, Mobs denotes the observed measurements (i.e. Mobs , ZK) and Mmis , (Ak; UK) denotesthe sequence of associations and controls up to K .The objective is to compute the MAP estimate of XK given the observed data, i.e. XMAPK , argmaxXK

f(XK |Mobs), which is equivalent to XMAPK = argmaxXK f(XK ;Mobs). Here f(XK |Mobs) is the posterior densityof XK conditioned on the observations Mobs. Maximization of f(XK |Mobs) (as a function of XK) is extremelydi!cult, since integrating out the missing data in f(XK ;Mmis|Mobs) has a computational cost which is expo-nential in the data length.The EM algorithm relies on the simplicity of expressing and maximizing f(XK ;M). The EM algorithm

generates the missing data Mmis using the observed data Mobs and the current best estimate of the desiredstate sequence XK . This estimate is then updated by maximizing a function of the complete data density andthe cycle is then repeated.The EM algorithm computes XMAPK , by generating from an initial state sequence estimate X (0)K a sequence

of state estimates {X (l)K }. Each iteration or “pass” of the EM algorithm consists of the following double step:On the l+ 1th iterationE step: Evaluate the conditional expectation of the complete log augmented density:

Q(XK ; X(l)K ) = E{lnf(XK ;M)|Mobs; X

(l)K }: (17)

Here X (l)K is the state sequence estimate at the lth iteration.M step: Compute X (l+1)K as

X (l+1)K = argmaxXK

Q(XK ; X(l)K ): (18)

The appealing property of the EM algorithm is that the posterior density increases monotonically, i.e.,f(X (l+1)K |Mobs)¿f(X (l)K |Mobs) with equality holding at the stationary points (local minima, maxima and saddlepoints) of the posterior distribution [20].The convergence of X (l)K to a stationary point, whether it is a local maximum, local minimum or a saddle

point, depends on the choice of the starting point X (0)K . Thus, it is recommended that several EM iterationsbe tried with di"erent starting points [20].

3. MAP state trajectory estimation of the target state sequence XK

In this section we present the EM algorithm for obtaining the MAP state trajectory estimate (see Eq. (16)for de#nition) of XK . The EM algorithm is used to obtain XMAPK . The terms modal state trajectory estimateand MAP state sequence estimate are used interchangeably.

3.1. Summary of the MAP state trajectory estimate algorithm for tracking a maneuvering target in clutter

The EM algorithm for computing XMAPK is summarized in Fig. 1 and schematically shown in Fig. 2. Asshown in Fig. 2, the implementation of the MAP algorithm for tracking a maneuvering target in clutter isexactly implemented by iteratively cross-coupling a HMM smoother with a #xed-interval KS. The KS operateson an averaged state space system—averaged over the states of the Markov chain (which is used to modelthe input control sequence) and the measurements at each time instant.


Fig. 1. Summary of the MAP state trajectory estimation algorithm for tracking a maneuvering target in clutter.

The proposed algorithm has a similar structure to the IMM-PDA estimator proposed by Bar-Shalom [3].The IMM-PDA can be viewed as an on-line version of the proposed algorithm with several Kalman #ltersoperating in parallel—each Kalman #lter operates on a di"erent maneuvering model. The state estimates ofthe parallel Kalman #lters are combined and used to initialize the #lters at the next iteration. Note however,unlike the proposed algorithm, IMM-PDA is an approximate (sub-optimal) algorithm.

3.2. Expectation step

In this subsection the aim is to evaluate Q(XK ; X(l)K ) de#ned in (17).


Fig. 2. Proposed EM MAP state trajectory estimation algorithm for computing the state sequence XK of a maneuvering target in clutter.

The density function for the joint complete data ZK ; XK , the control input sequence UK and the associationsequence AK can be expressed as

f(ZK ; XK ; UK ; AK) =K)

k=1

f(Z(k)|x(k); a(k))K)

k=1

f(x(k)|x(k $ 1); u(k))f(x(0))

#K)

k=1

Pr{a(k)}K)

k=2

Pr{u(k)|u(k $ 1)}Pr{u(1)} (19)

which follows directly from the state space model and the statistical assumptions on the random variablespresented in Section 2. In particular, the #rst and second terms on the right-hand side follow from theindependence assumption of the measurement noise w(k) and state noise v(k), respectively. The densityf(Z(k)|x(k); a(k)) is given by

f(Z(k)|x(k); a(k) = i) =(

V"m(k) for i = 0;

V"m(k)+1f(zi(k)|x(k)) for i! {1; 2; : : : ; m(k)}:(20)

Expanding (19) and taking the natural logarithm we have

lnf(ZK ; XK ; UK ; AK)

=$ m(k)K*

k=1

ln(V ) +K*

k=1

ln(V )I{a(k) '=0}$ 12

K*

k=1

ln(|2%R|)I{a(k) '=0}

$ 12

K*

k=1

(za(k)(k)$ Cx(k))!R"1(za(k)(k)$ Cx(k))I{a(k) '=0}$ 12

K*

k=1

ln(|2%Q|)

$ 12

K*

k=1

(x(k)$ Ax(k $ 1)$ Bu(k))!Q"1(x(k)$ Ax(k $ 1)$ Bu(k))


$ 12ln(|2%P0|)$

12(x(0)$ x0)!P"1

0 (x(0)$ x0) +K*

k=1

ln(Pr{a(k)})

+K*

k=2

ln(pu(u(k $ 1); u(k))) + ln(pu(1)); (21)

where I{·} denotes the indicator function, which is equal to unity if the event (·) is true, zero otherwise.From the de#nition of Q(XK ; X

(l)K ) and (21) we have

Q(XK ; X(l)K ) =$m(k)

K*

k=1

ln(V ) +K*

k=1

(1$ Pr{a(k) = 0|ZK ; X (l)K })+

ln(V )$ 12ln(|2%R|)

,

$ 12

K*

k=1

m(k)*

ia=1

(zia(k)$ Cx(k))!R"1(zia(k)$ Cx(k))Pr{a(k) = ia|ZK ; X(l)K }

$12

K*

k=1

ln(|2%Q|)$ 12

K*

k=1

Nu*

i=1

(x(k)$Ax(k$1)$Bqi)!Q"1(x(k)$Ax(k$1)$Bqi)Pr{u(k)

= qi|ZK ; X (l)K }$ 12ln(|2%P0|)$

12(x(0)$ x0)!P"1

0 (x(0)$ x0)

+K*

k=1

m(k)*

i&=1

ln(Pr{a(k) = ia})Pr{a(k) = ia|ZK ; X (l)K }

+K*

k=2

Nu*

i=1

Nu*

j=1

ln(pu(i; j))Pr{u(k $ 1) = qi; u(k) = qj|ZK ; X (l)K }

+Nu*

i=1

ln(pu(i))Pr{u(1) = qi|ZK ; X (l)K }: (22)

Let us now consider the task of computing '(l)k (ia; iu), Pr{a(k)= ia; u(k)= qiu ; |ZK ; X(l)K } for ia ! {0; 1; : : : ;

m(k)}, iu ! {1; : : : ; Nu}. We de#ne the forward variable &(l)k (ia; iu) and the backward variable ((l)k (ia; iu) asfollows:

&(l)k (ia; iu), f(Zk ; X(l)k ; a(k) = ia; u(k) = qiu); (23)

((l)k (ia; iu),f(ZKk+1; XK (l)k+1 |a(k) = ia; u(k) = qiu ; Zk ; X

(l)k ); (24)

where ZKk+1 denotes the sequence of measurements from k + 1 up to time K , that is ZKk+1 = (Z(k + 1);Z(k + 2); : : : ; Z(K)).De#ne the observation merit function b(l)ia;iu(Z(k)), f(Z(k); x(l)(k)|Zk"1; X (l)k"1; a(k) = ia; u(k) = qiu). Then

we have

b(l)ia;iu(Z(k)) =f(Z(k)|x(l)(k); a(k) = ia)f(x(l)(k)|x(l)(k $ 1); u(k) = qiu); (25)


where f(Z(k)|x(l)(k); a(k) = ia) is given by (20) and

f(x(l)(k)|x(l)(k $ 1); u(k) = qiu)

= |2%Q|"1=2 exp+

$12(x(l)(k)$ Ax(l)(k $ 1)$ Bqiu)!Q"1(x(l)(k)$ Ax(l)(k $ 1)$ Bqiu)

,

: (26)

In analogy to standard HMM [16], it can be shown that &t and (t are computed via the following forwardand backward recursions, respectively:

&(l)k+1(ja; ju) =m(k)*

ia=0

Nu*

iu=1

b(l)ja;ju(Z(k + 1))Pr{a(k + 1) = ja}pu(iu; ju)&(l)k (ia; iu); (27)

&(l)1 (ja; ju) = b(l)ja;ju(Z(1))Pr{a(1) = ja}pu(ju); (28)

((l)k (ia; iu) =m(k)*

ja=0

Nu*

ju=1

((l)k+1(ja; ju)Pr{a(k) = ia}pu(iu; ju)b(l)ja;ju(Z(k + 1)); (29)

((l)K (ia; iu) = 1; (30)

'(l)k (ia; iu), Pr{a(k) = ia; u(k) = qiu |ZK ; X(l)K }=

&(l)k (ja; ju)((l)k (ja; ju)

'm(k)ja=0

'Nuju=1 &

(l)k (ja; ju)(

(l)k (ja; ju)

(31)

for all ja ! {0; 1; : : : ; m(k)} and ju ! {1; : : : ; Nu}. With some abuse of notation, we de#ne the a posteriorimeasurement-to-target association probabilities and the a posteriori control input probabilities, respectively, asfollows:

'(l)k (ia), Pr{a(k) = ia|ZK ; X (l)K }=Nu*

iu=1

'(l)k (ia; iu); (32)

'(l)k (iu), Pr{u(k) = qiu |ZK ; X(l)K }=

m(k)*

ia=0

'(l)k (ia; iu): (33)

3.3. Maximization step

Having evaluated Q(XK ; X(l)K ) in Section 3.1, our aim in this section is to #nd the MAP state estimate of

XK from argmaxXK Q(XK ; X(l)K ). X

(l+1)K is thus computed as follows:

From Q(XK ; X(l)K ) in (22), ignoring terms independent of XK which are irrelevant to the maximization, we

have

X (l+1)K = argmaxXK

Q(XK ; X(l)K )

= argminXK

-

K*

k=1

m(k)*

ia=1

(zia(k)$ Cx(k))! R"1(zia(k)$ Cx(k))'(l)k (ia)

+ (x(0)$ x0)!P"10 (x(0)$ x0)

K*

k=1

Nu*

iu=1

(x(k)$ Ax(k $ 1)$ Bqiu)!

#Q"1(x(k)$ Ax(k $ 1)$ Bqiu)'(l)k (iu)

.

: (34)


There are several ways of maximizing Q(XK ; X(l)K ). One method involves direct di"erentiation and setting the

resulting expression to zero. This requires solving a symmetric block tridiagonal linear system of equationsand is not pursued in this paper.An alternative method based on the Kalman smoother is presented here. By completing the square in x(k),

and after some simple algebraic manipulations the following is obtained:

Q(XK ; X(l)K ) =$1

2

K*

k=1

(z(l)(k)$ Cxk)! R(l)!1

(z(l)(k)$ Cxk)

$ 12

K*

k=1

(x(k)$ Ax(k $ 1)$ Bu (l)(k))!Q"1(x(k)$ A x(k $ 1)$ Bu (l)(k))

$ 12(x(0)$ x0)!P"1

0 (x(0)$ x0) + [terms independent of x(k)]; (35)

where

z(l)(k) =1

1$ '(l)k (ia = 0)

m(k)*

ia=1

zia(k)'(l)k (ia); (36)

R(l)!1

= R"1(1$ '(l)k (ia = 0)); (37)

u (l)(k) =Nu*

iu=1

qiu'(l)k (iu): (38)

The following theorem shows that the maximization of Q(XK ; X(l)K ) as a function of XK—given the previous

MAP estimate X (l)K —can be achieved by a KS operating on an averaged state space system (averaged overthe states of the Markov chain and the measurements at each time instant).

Theorem 3.1. Consider the following linear Gaussian state space model:

)(k) = A)(k $ 1) + Bu (l)(k) + v(k); (39)

y (l)(k) = C)(k) + wk ; (40)

where )(0) % N (x0; P0); w(k) % N (0; R(l)) and v(k) % N (0; Q) are mutually independent white Gaussian

noise processes. The averaged variables R(l); z(l)(k); u (l)(k) are given by Eqs. (36)–(38). Then the MAP

state trajectory estimate of XK on the l+1th pass is computed by a "xed-interval KS on the average statespace model (39) and (40) as

X (l+1)K = ()1|K ; : : : ; )K|K); (41)

where )k|K , E{)(k)|Y (l)K }; k = 1; 2; : : : ; K .

Proof. Due to the Gaussianity and independence of )(0); w(k) and v(k); the following expression for the jointdensity of )K and Y

(l)K can be easily derived:

lnf(Y(l)K ; )K = XK) = Q(XK ; X

(l)K ) + [term independent of x(k)]; (42)

where Q(XK ; X(l)K ) is de#ned in (35).


It is well known (see [10] pp. 215–218) that the MAP state sequence estimate of any linear Gaussiansystem e.g. (39) and (40) is given by a #xed-interval KS, that is

(x1|T ; : : : ; xT |T ) = argmaxXKf(Y

(l)K ; XK): (43)

We know that the EM algorithm computes

X (l+1)K = argmaxXK

Q(XK ; X(l)K ): (44)

Also from (42) we have argmaxXKf(Y(l)K ; XK) = argmaxXKQ(XK ; X

(l)K ). From (44), Eq. (41) follows.

The E-Step in Section 3.1 and the M-Step in Section 3.2 are repeated until the distance (Euclidean or someother metric) between X (l)K and X (l"1)K is less than some predetermined value.

4. Simulation studies

In this section we present an illustrative example of our proposed Bayesian expectation maximization algo-rithm for optimal tracking of a maneuvering target in clutter. We also compare our algorithm with IMM-PDAestimator proposed by Bar-Shalom [5].We recommend using an algorithm such as IMM-PDA to compute an initial target trajectory estimate X (0)K

for our EM algorithm. From X (0)K , the maximum a posteriori target state trajectory is then computed by theproposed algorithm as summarized in Fig. 1.We now present some computer simulated experiments. For convenience, we compute x(0)(0) using the #rst

noisy true measurement. The initial target trajectory X (0)K is then computed based on x(0)(0) and assumingthat the target (even though it may maneuver) is moving on a perfectly straight line.Table 1 shows the parameters used in our simulation studies. Fig. 3 shows the simulated target trajectory,

which consists of the following #ve legs: (1) the target moves on a straight line up, (2) makes a right-handturn, (3) moves straight, (4) makes a left-hand turn and #nally (5) moves straight ahead.

Table 1Target and measurement parameters used

Sampling interval T = 7 sNumber of measurements K = 50Initial target position (!500;!500)" mInitial target velocity (0:0; 5:0)" m=s

Transition probability matrix pu(i; j) =

(

0:9 if i = j0:05 otherwise

Maneuver commands (three)Bu= (0 0 0 0)" (straight)Bu= (!1:225 ! 0:35 1:225 0:35)" (left turn)Bu= (1:225 0:35 ! 1:225 ! 0:35)" (right turn)

Observation matrix C = I4#4Process noise Q = (0:1)2I4#4Measurement noise R= diag(20:02; 1:02; 20:02; 1:02)Measurement volume V [! 1000; 1000] m in x and y position

[! 10:0; 10:0] m=s in x and y velocity


1000 800 600 400 200 0 200 400 600 800 1000 1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

_

_

_

_

__ _ _ _ _ 1000 800 600 400 200 0 200 400 600 800 1000

1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

1

0

_

_

_

_

__ _ _ _ _

Fig. 3. Target trajectory. Fig. 4. MAP state trajectory estimate of a maneuvering target forPd = 1:0 (perfect detection) and #V = 0 (no clutter). · · · denotemeasurements and l=0; 1; : : : denote the target state estimate X (l)Kon the lth iteration.

1000 800 600 400 200 0 200 400 600 800 1000 1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

0 1 2 3

4

5

6

_

_

_

_

__ _ _ _ _ 1000 800 600 400 200 0 200 400 600 800 1000

1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

_

_

_

_

__ _ _ _ _

Fig. 5. MAP state trajectory estimate of a maneuvering target forPd =1:0 (perfect detection) and #V =5. · · · denote measurementsand l = 0; 1; : : : denote the target state estimate X (l)K on the lthiteration.

Fig. 6. MAP state trajectory estimate of a maneuvering target forPd=1:0 (perfect detection) and #V=50. · · · denote measurements.

Depending on the probability of detection Pd and the clutter density #, we consider the following scenarios:• Pd =1:0 and #V =0. This scenario corresponds to a perfect detection process with no clutter. Fig. 4 showsa typical measurement realization and the sequence of target estimates. The estimate converges after oneiteration.

• Pd = 1:0 and #V =5. Fig. 5 shows a typical measurement realization and the sequence of target estimates.In most cases the estimates converged in less than #ve iterations. Fig. 5 clearly shows how the target statetrajectory estimates are improved from one iteration to the next.

• Pd = 1:0 and #V = 50. As the average number of clutter points increases, the number of iterations of theproposed EM algorithm also increase (for example see Fig. 6).


1000 800 600 400 200 0 200 400 600 800 1000 1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

0

1

2

34

_

_

_

_

__ _ _ _ _ 1000 800 600 400 200 0 200 400 600 800 1000

1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

4 3

2

10

_

_

_

_

__ _ _ _ _

Fig. 7. MAP state trajectory estimate of a maneuvering target forPd = 0:65 and #V = 0 (no clutter). · · · denote measurements and" " " denote missed detections. l=0; 1; : : : denote the target stateestimate X (l)K on the lth iteration.

Fig. 8. MAP state trajectory estimate of a maneuvering target forPd =0:65 and #V =5. · · · denote measurements and " " " denotemissed detections. l=0; 1; : : : denote the target state estimate X (l)Kon the lth iteration.

1000 800 600 400 200 0 200 400 600 800 1000 1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

_

_

_

_

__ _ _ _ _ 1000 800 600 400 200 0 200 400 600 800 1000

1000

800

600

400

200

0

200

400

600

800

1000

X (m)

Y (m

)

_

_

_

_

__ _ _ _ _

Fig. 9. MAP state trajectory estimate of a maneuvering target forPd =0:65 and #V =50. · · · denote measurements and """ denotemissed detections.

Fig. 10. MAP state trajectory estimate of a maneuvering targetfor Pd = 0:65 and #V = 50. · · · denote measurements and " " "denote missed detections. Lost track with RMS position error of143.612.

• Pd = 0:65 and #V = 0. This scenario corresponds to an imperfect detection process with no clutter.Fig. 7 shows a typical measurement realization and the sequence of target estimates.

• Pd =0:65 and #V =5. Fig. 8 shows a typical measurement realization and the sequence of target estimates.• Pd = 0:65 and #V = 50. Finally, in Figs. 9 and 10 two di"erent MAP state trajectory estimates are shown(corresponding to two di"erent measurement realizations). The latter #gure corresponds to a lost track.The above simulations were then repeated for 100 di"erent measurement realizations for a single target

moving on the same path (the trajectory of Fig. 3). Table 2 summarizes the performance of the proposed


Table 2RMS target position error

EM IMM-PDA

Pd = 1:0 and #V = 0 10.3193 16.4832Pd = 1:0 and #V = 5 10.9071 16.8327Pd = 1:0 and #V = 50 16.9294 16.9332Pd = 0:65 and #V = 0 12.6561 18.0927Pd = 0:65 and #V = 5 15.4537 18.3527

algorithm. The performance measure is the RMS position error, computed as follows:

RMS =

/

0

0

1

1NK

N*

n=1

K*

k=1

[(rx(k)$ rMAPx (k; n))2 + (ry(k)$ rMAPy (k; n))2]; (45)

where N is the number of Monte Carlo simulations and rMAPx (k; n) is the MAP target position estimate (inthe x direction) at time k of the nth Monte Carlo simulation.Also presented in Table 2 are the RMS errors for the IMM-PDA algorithm. The results show that the EM

algorithm outperforms IMM-PDA for di"erent values of Pd and #V .From Table 2 we can make the following intuitive remarks: (1) the performance of the algorithm decreases

as the probability of detection Pd decreases, and (2) the performance of the algorithm decreases as the numberof average clutter points increases. This is clearly seen for the case Pd=0:65 and #V=50. The high RMS errorvalue is due to lost tracks. That is, portions of the target trajectory (usually the latter stages) are incorrectlyestimated (for example see Fig. 10).

5. Conclusions

In this paper, we presented an iterative o"-line optimal state estimation algorithm, which yields the maximuma posteriori state trajectory estimate of a target maneuvering in clutter. The problem is formulated as a jumpMarkov linear system and the expectation maximization algorithm is used to compute the state sequenceestimates.The algorithm proposed in this paper, uses probabilistic multi-hypothesis (PMHT) techniques for tracking

a single maneuvering target in clutter. Previous applications of the PMHT technique [18] have addressed theproblem of tracking multiple non-maneuvering targets. The extension of our algorithm to multiple maneuveringtarget tracking is straightforward and details are omitted.

References

[1] B.D.O. Anderson, J.B. Moore, Optimal Filtering, Prentice-Hall, Englewood Cli"s, NJ, 1979.[2] Y. Bar-Shalom, T.E. Fortmann, Tracking and Data Association, Academic Press, New York, 1988.[3] Y. Bar-Shalom, X.R. Li, Multitarget-Multisensor Tracking: Principles and Techniques, YBS, 1995, Storrs, CT, USA.[4] S. Blackman, R. Popoli, Design and Analysis of Modern Tracking Systems, Artech House, 1999, Boston, MA, USA.[5] H.A.P. Blom, Y. Bar-Shalom, The interacting multiple model algorithm for systems for systems with Markovian switching

coe!cients, IEEE Trans. Automat. Control 33 (August 1988) 780–783.[6] A.P. Dempster, N.M. Laird, D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc, Ser.

B 39 (1) (1977) 1–38.[7] A. Doucet, A. Logothetis, V. Krishnamurthy, Stochastic sampling algorithms for state estimation of jump Markov linear systems,

IEEE Trans. Automat. Control 45 (2) (February 2000) 188–202.


[8] A. Doucet, N. Gordon, V. Krishnamurthy, Particle #lters for state estimation of jump Markov linear systems, IEEE Trans. SignalProcess 49, No 3, 613–624 March (2001), to appear, also technical report, University of Cambridge, CUED-F-INFENG TR.359,1999.

[9] C.N. Georghiades, D.L. Snyder, The expectation-maximization algorithm for symbol unsynchronized sequence detection, Trans.Commun. 39 (1) (January 1991) 54–61.

[10] A.H. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, New York, 1970.[11] B.F. La Scala, G.W. Pulford, Manoeuvring target tracking using the expectation-maximisation algorithm, The Fourth International

Conference on Control, Automation, Robotics and Vision, ICARCV ’96, Singapore, 1996, pp. 2340–2344.[12] A. Logothetis, V. Krishnamurthy, Expectation maximization algorithms for MAP estimation of jump Markov linear systems, IEEE

Signal Process. 1999, to appear.[13] K.J. Molnar, J.W. Modestino, Applications of the EM algorithm for multitarget=multisensor tracking problems, IEEE Trans. Signal

Process. 46 (1) (January 1998) 115–129.[14] L.B. Nelson, Multiuser Detection for Radio Frequency and Optical CDMA Communications, PhD thesis, Department of Electrical

Engg., Princeton University, 1995.[15] G.W. Pulford, A. Logothetis, An expectation-maximisation tracker for multiple observations of a single target in clutter, submitted

at the 35th Conference on Decision and Control, Kobe, Japan, 1996.[16] L.R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Proc. IEEE 77 (2) (1989)

257–285.[17] R.L. Streit, Studies in Probabilistic Multi-Hypothesis Tracking and Related Topics, Vol. SES-98-01. Naval Undersea Warfare Center

Division, Newport, Rhode Island, February, 1998.[18] R.L. Streit, T.E. Luginbuhl, Probabilistic multi-hypothesis tracking, NUWC–NPT, Technical Report 10428, Naval Underseas Warfare

Center, Newport, Rhode Island, 1995.[19] M.A. Tanner, Tools for Statistical Inference, Springer, Berlin, 1993.[20] C.F.J. Wu, On the convergence properties of the EM algorithm, Ann. Statist. 11 (1) (1983) 95–103.

a bayesian em algorithm for optimal tracking of a ...vikramk/lkh02.pdfa bayesian em algorithm for...

Documents