on computational efficiency of linear filtering algorithms

Automatica, Vol. 7, pp. 299-314. Pergamon Press, 1971. Printed in Great Britain.

On Computational Efficiency of Linear Filtering Algorithms*

Sur l'efficacitd du calcul des algorithmes de filtrage lindaire

(Jber die Effektivit it der Berechnung linearer Filteralgorithmen

0 6 3qbqbeKTHBHOCTH BbIqric.rIeHH~I a . r I r o p r t T M O a YIHHe~HOFO qba.r IbTprlpoBaI-nOt

I. A. G U R A t and A. B. B I E R M A N t

The computational requirements for several popular linear estimation algorithms are compared via a systematic approach.

Summary--Although considerable attention has been given to various aspects of the filtering problem, the question of computer requirements has not been analyzed in detail. This paper presents an attempt at a systematic analysis of this problem. Formulae for the arithmetic operation count and storage requirements are derived for several popular filtering algorithms within the framework of a general estimation problem. Parameters included in the study are the dimensions of the state vector, the frequency and number of observations, and the frequency of state updates. The results for each algorithm are discussed from a practical viewpoint and comparisons are made for certain interesting cases. Several common misconceptions are exposed.

1. INTRODUCTION

IN RECENT years, the attention lavished on the Kalman filter [1] has uncovered a vulnerability to numerical instabilities arising from computational errors. As a result, several alternative estimation algorithms [2-9] are receiving notable attention. Although it has been shown that the numerical behavior of these methods is superior to the tradi- tional Kalman mechanization, little attention has been given to the computer time and storage requirements associated with each algorithm. In some isolated cases where computer requirements have been considered, the application was highly specialized. In other instances, comparisons of candidate schemes have been performed with a "stacked deck" in which the selected operational sequence has been more favorable to one of the candidates. As an example, a published comparison of the Kalman filter and the least squares estimation algorithms [10] assumed that a state estimate update was to be performed at every data

* Received 22 May 1970; revised 16 October 1970. The original version of this paper was not presented at any IFAC meeting. It was recommended for publication in revised form by associate editor H. A. Spang.

t The Aerospace Corporation, E1 Segundo, California 90245.

point along a trajectory, even though there are many cases where updates are not needed at the same frequency that data is gathered. In addition, this comparison completely ignores the existence of a sequential form of the least squares estimator [8, 9]. More common and similarly misleading are judgments based on the number and order of matrix inversions without regard to the accom- panying matrix multiplication operations.

While the selection of a particular estimator depends upon the demands of the specific application, e.g. accuracy, update frequency, computer capability, etc., any fair comparison should involve a complete analysis of the total number of arithmetic operations and the total storage locations necessary to produce a required sequence of estimates.

A general study should include as parameters the size of the state vector, the frequency and number of observations, and the frequency of state updates.

This paper presents an attempt at such a systematic analysis. Several popular algorithms will be presented within the framework of a general estimation problem. Formulae for the count of arithmetic operations and for storage ~equirements are then derived for each method. The results are applied to each estimation algorithm and the corresponding computer time and storage requirements are discussed from a practical viewpoint.

While an extensive analysis of computer time and storage requirements is presented here, only part of the general problem of comparing filter algorithms has been addressed. The companion area of relative accuracy has not been consideled. Its analysis would involve new complexities such as those associated with model approximations, computer word length, system software, etc. A

299

300 1. A. GURA and A. B. BIERMAN

complete treatment of this question would clearly be beyond the scope of a single paper. It is hoped, however, that this work will provide a basis for a systematic analysis of the accuracy question.

2. NOTATIONAL CONVENTIONS

(a) Matrices are upper case letters. (b) Vectors are lower case italic letters. (c) Scalars are Greek letters. (d) Asterisks (*) denote matrix transposition. (e) Vectors are assumed to be columns unless

otherwise denoted by an asterisk. (f) Exceptions to these rules are i, j, k , / , m, n and

q which are integers, and t which denotes time. (g) The symbols 2u J, P u i refer to a state vector

estimate and its associated eovariance matrix at time t~ based on data taken through tj.

3. BASIC SYSTEM MODEL

A general linear system model encountered in numerous estimation problems is

Xk + 1 = ~ ( k + 1, k ) x k

Yk+ 1 = n k + 1Xk+ 1 -1- Dk+ 1 (1)

where Xk+ 1 is an n-dimensional state vector, Yk+ 1 is an m-dimensional data vector, ~ ( k + l , k) and Hk+~ are known appropriately dimensioned matrices, and Vk+ ~ is a random m-vector such that ~(vk+l)=O and o~(Vk+lV~+l)=R. It is usually assumed that an a priori estimate of the state vector, ~o, with an associated covariance P o = e ( x o - ~ o )

(x o - 20)* is known. While this model is certainly general, admitting

data at arbitrary times, its direct use could result in an excessive number of computations for a number of practical problems. Specifically, there are many engineering situations having a repeat- ing data pattern in which several independent sets of data are received in rapid succession, followed by a relatively long interval with no data reception. If, in such cases, the system dynamics change slowly over the high data rate periods, the transition matrix, O(k+ 1, k), could be essentially equal to the identity matrix over that part of the data span. Where this effect is expected to occur, it is practical to avoid the O(k+ I, k) computation implied in equation (1). That is, if the data times t~, t 2 , . . . , tv are close to each other compared to dominant system time constants, while t~ and t,+ are relatively far apart, and this pattern repeats for each v data sets, then equation (1) becomes

Xk + 1 = ~(k- t - 1, k)x~ for k = j r , j = O, 1, 2 . . . . , 1

Xk+ 1 =X k for all other k

Yk+l=Hk+lXk+l +Vk+x. (2)

The parameter v, introduced here, is the number ol uncorrelated sets of m-dimensional data vectoi"s received in an interval between system dynamic updates. Thus v relates the effects of dynamics, or, perhaps, lack of dynamics, to computational requirements. The parameter I is defined as the total number of dynamic updates in the entire data span. This quantity will prove useful in establish- ing a uniform basis for comparing various filter algorithms. The relationships among these parameters are, illustrated in Fig. 1.

EACH MEASUREMENT IS AN m-VECTOR

• . . . . I l l , , , , . . . . t o tl t z t3 " " t v iv4-1 t2v t(~-Ilv+l tz'v

TIME

Flo. l . Relationships among model parameters.

It may have been observed that the models (1) and (2) lack some generality since measurement noise has been assumed stationary, and dynamic noise has been omitted completely. These assump- tions were made so that uniform, yet practical, bases of comparison can be established. In applications, the need for time varying measurement statistics is not very common. Similarly, an actual model requirement for dynamic forcing noise does not often arise. However, the formalism of dynamic model noise is commonly used as an engineering adjustment for gross model errors. The desired effect is to "increase" the covariance so that the estimates will not become prematurely insensitive to the data. Since such adjustments are peculiar to the problem at hand and are generally made only when necessary, it is ditiieult to deter- mine their significance in an abstract comparison of estimation algorithms.

Although the effects of time varying measurement noise and forcing noise are omitted from the basic comparisons, the modifications required for their inclusion are outlined in a special summary.

4. ALGORITHM DESCRIPTIONS

While the algorithms below have all been described in the literature, they are presented here to establish the necessary notational continuity con- sistent with the model (2).

Three of the methods discussed are based on propagation of the square root of the covariance matrix. For the purposes of this paper, a matrix B (or A ¢) is said to be the square root of a square symmetric matrix A if

A = B B * . (3)

For positive definite A, a suitable B can always be found by the Cholesky factorization [1 I].

On computational efficiency of linear filtering algorithms 301

(i) Standard Kalman filter [1--4] This form of the filter is probably the most

popular and is essentially the one derived by Kalman in 1960. Note that the covariance matrix, Pk+ ~/k+ a, can fail to be positive definite if roundoff errors become serious or if this matrix approaches zero.

Xk+ 1/k =~( k'J¢- 1, k).~k/a ~for k =jv, Pk+ a/k=O(k+ 1, k)Pk/kt~*(k+ l, k ) J j =0, 1, 2 . . . . . l

Xa+ a/k :Xk/k ~for all other k. Pa + 1/k -~ Pk/k

K = Pk+ alaHk+ l(Hk+ lPk+ l /aHk+ 1 "l" R ) - a

Pk + a /k + a = Pk + a/k-- KHk + l Pk + a/k

:~k+ a/a+ a = :~k+ ~/~ + K ( y a + a - - Ha+ a:ca+ a/k)

(ii) Stabilized Kalman filter [2-4] where Although*this algorithm has appeared in various

applications, no definite primary reference can be given.

From an accuracy viewpoint, it is superior to the Standard Kalman filter since it is less sensitive to numerical roundoff. This method also generates correct covariances even if the filter "gain", K, is computed in an arbitrary manner. The previous algorithm only yields correct covariance matrices for the optimum gain sequence.

~k + 1/k : f~(k -~- 1, k)2k/k ~for k =jr, Pk+a/k =@(k+ 1, k)Pk/kdP*(k+ 1, k ) J j = 0, 1, 2 . . . . . l

} k + I[k -~" "Xk/k for all other k. Pk + 1 ]k = Pk/k

K = e k + l /kH*+ l(Hk+ xPk+ 1/kH*+ l + R ) -a

Pk + 1/k + 1 = (I - KHa + a )Pk + a/,(I - K H , + 1 )*

+ K R K *

i k + ~ /k+, =:~+ a/k+K(Yk+a--H~+a:ck+a/k).

(iii) Sequential least squares filter [8, 9] This algorithm is a modern version of the

classical method introduced by Gauss over 150 years ago.

1--1 --1 --1 Po/t, = Po + ~ ¢P*(jv + 1, O)

j=o

~=j~+ a Let

P~/~,--- ~ ( ( l - 1)v + 1, O)Po/t,~*((/- 1)v + 1, O)

2~/ i~=~D(( l -1)v+l , O) 2o+Po¢~ O * ( j v + l , O)

[ : ~ ; : ~ H * R - I ( Y t - H / I ~ ( j v + I , O ) : ~ o ) ] } •

Note that these relationships can be used to filter each Iv set of measurements if ~o and Po are appropriately identified with the results of the previous fit span.

(iv) Potter square root filter [5] The notion of square root filtering was intro-

duced by Potter in a special algorithm for processing measurements with uneorrelated noise. The algorithm presented here is a generalization of that approach, valid for data vectors with correlated components.

-~k+ x/k=O(k+ 1, k)~a/k~. k)p~/k~or k =jv, j = 0 , 1, 2 . . . . . l

P~+ l /k ~ f~( k -~ - 1,

"r~+ X/a=:~k/k ~for all other k. p~ - p½ k+a[k-- k/k )

The data

G-1Hk+a=(ba, b2, . . . , b~)*

G- lyk+a=(Ot , 0 2 , . . . , Om)*

RAGG*

are processed sequentially according to the following scheme

Yq=~k+l/k, WI=P~+I/a

sl= W~bi

~i= s*s~ + 1

Wi+ x = W i - W :i(s*/(~i + J ~i))

i = 1 , 2 . . . . . m

R = GG*

B = (P~+ a/k)*Hk + l ' G - 1.

Find the m non-zero eigenvalues 2 . 2z, . . . . 2m and corresponding normalized eigenvectors Sa, s 2 , . . . , s m of B B * .

Let

S~(sa, s2 .... , s,D

D A ( I . + A ) - ~ - I , ~

~i+ l =.~i + W isi((Oi- b *~)/ai)

P~k+ l/k+ l = Wra+ l

"~k+ I/k+ 1 ='~m+l •

(v) Bellantoni and Dodge square root fi l ter [6] This method resulted from an attempt to genera-

lize Potter's work for vector measurements. A major drawback is the necessity for eigenvalue- eigenvector computations at each stage in the recursion process.

xk + 1/k = ~(k + 1, k):ek/~), P~k+i/k=tl>(k+ I, k)P~/kj ~l°r k=jv, j=O, I, 2 ..... I

.~k+ I/k=2k/k ~,for all other k. eL a /~ = e~/a J


where

Then

A_

21

22

,.

~r Pk + 1/k + 1 = P~k + 1/k(ln -t- SDS*)

Xk+ l]k+ l = '~k+ l/k

-t- P~+ a/kB(I" + B ' B ) - 1 G - 1 (Yk + I -- HR + I Xk + 1/k).

(vi) Andrews square root filter [7] The algorithm presented here was originally

suggested by Andrews for filter covarianee analysis as an alternative to both the Potter and the BeUantoni and Dodge methods.

"fk+l/k=~(k+l' k)2k/k]'for k=jv, j=O, 1 2, l P~k+ 1/k =(~)(k.-{- l , k)P~/k~ . . . . .

P~ + l /k = P~/k ~f°r all other k.

Let R=GG*

Z =(P~+ m)*H*k+

U U * = R + Z * Z .

Then,

_ ½ ½ , - 1 P~+I/R+I--PR+I/R--Pk+1/RZ( U ) (U-t-G)-IZ *

.~k+ Z/k+ t = Xk+ 1/k

+ P~+ ~/kZ(UU*)- ~(Yk+ ~ -- Hk+ X2k+ ~/k).

5. COMPARISON OF ALGORITHMS--COUNTS OF OPERATIONS AND STORAGE

In attempting to compare the various algorithms of the previous section, it becomes obvious that there is a fundamental difference between the Least Squares filter and all the other algorithms. Whereas the other methods will always generate a new state estimate after processing each m-vector of data, the times at which the Least Squares method updates state estimates can be arbitrarily determined by the user. Since in practical situations, estimates are generally needed only at specific times, it is reasonable to require the Least Squares filter to generate a new state estimate only at these times, even though the other methods yield estimates more often. In the following analysis, this policy is implemented by choosing the time interval for comparison so that the Least Squares algorithm gives a single state estimate at the end of the data span.

Under the above assumption, the count of operations and storage locations required for each of the algorithms given above is derived in Tables 1-6 and summarized in Table 7. The basic assump- tions made and conventions used ill developing these tables are given here:

(1) Square brackets are used in the tables to indicate the specific matrix operation being considered.

(2) Operations performed only once for each algorithm are ignored.

(3) Only multiplications, divisions, and square roots, see 8, below, are expqcitly counted. If an addition count is desired, an excellent approximation is the multiplication/division count.

(4) No distinction is made between multiplication and division operations. The' latter are relatively small in number and arise primarily in the matrix inversion process.

(5) In Tables 1, 2 and 3 operation and storage counts are given both with matrix symmetry considered and with such symmetry ignored. Table entries are presented in the form no symmetry/symmetry whenever the counts differ. In Tables 4, 5 and 6 calculations are based on symmetry considerations only.

(6) All matrix inversions are assumed to be performed via the efficient Cholesky factorization [11]. It can be readily shown that the inversion process fo~ an n x n matrix only requires (n3/2)+;in z+nq operations and n(n+l)/2 storage locations. Since these locations can be the same as those assigned to the given matrix, no additional locations are allocated for the inversion process.

(7) In general, operation counts for eigenvalue- eigenvector calculations are difficult to specify because of the iterative nature of the problem. Thus, in Tables 5 and 7 this number is para- meterized as p to allow for reasonable varia- tions.

(8) Since square root extraction is also an iterative procedure, a parameter q is used to represent the numbm of multiplications required.

(9) No provisions have been made for computation of the state transition ((I)) or measurement sensitivity (H) matrices. The number of operations necessary for computing these quantities is clearly heavily dependent on the application at hand. In all cases, however, the resultant operation count increase would be the same for all algorithms. Although much consideration has been given to choosing the best form for each algorithm for effÉciency of computation, an optimum is in no way implied.

(10)

On computational efficiency of linear filtering algorithms

TABLS 1. STANDARD KALMAN FILT-I~

303

Computation sequence Operations Storage

1 ~k/k ?l

2 Pk/k n 2 f

~ - - ~ ~ n ( n ÷ 1)/2

3 R m 2

4 Yk+ 1 m

5 ~(k+ 1, k)

6 [~(k+ 1, k)][Pk/k]

max(n~, ran)

n s max(n2, mn)

Pk +t/k = [O(k + I, k)Pk/k] [O*(k + I, k)] Store in 2

8 ~,+,/,=[m(k+1, k)][~,/,] /l 2 B

9 Hk+ 1 Store in 5

10 yk+1-EH~+dE~+i/d mn Store in 4

U Es~+1IEe~+i/~] n2m Store in 6

12

13

En~+le~+,:dEn~+,]

CH~+ * tPk+ 1/kHk+ 1 "4"R] - I

m2

m3+3m2+mq Store in 11 2 2

1 4 K = [ H k + 1 P k + * * - 1 l/k] [Hk+ tPk+ 1/kHk+ 1 + R] m2n Store in 5

mn Store in 1

16 P~+i/k+1 =P~+t/t-[KICHI,+IP~+I/kl Store in 2

304 1. A. GUR~ and A. B. BIERMAN

TABLE 2. STABILIZED KALMAN FILTER

Computation sequence Operatons Storage

2

8

9

10

11

12

13

14

15

16

17

18

19

20

2k/k

Pk/k

R

Yk+ 1

@(k+ 1, k)

[(I)(k + 1, k)] [P,/k]

Pk+ 1/k = [O(k + 1, k)Pk/k][@*(k + 1, k)]

#k+ ,/k = [O(k + 1, k)][#k/k]

nk+ i

[Hk+,][Pk+ i/k]

[Hk+ 1Pk+,/k][H*+ i ]

[Hk + I Pk + x/k H* + R]- i

K = [Hk+ IPk+ Uk]*[Hk+ IPk+ i/kH~+, + R ] - I

Yk+1 - [Ilk+ 1][#k+,/k]

& +, ~, + , = & +1/~ + [ K ] [y, +, -- H~ +, & +, ~ ]

t -- [ K ] [ n ~ + , ]

[ l -- KHk+ ,][Pk+ x/k]

[ ( I - K H k + 1)Pk + ,/k][ I - KHk + 1]*

[K][R]

[ KR] [K* ] + ( I - KH k + 1 )Pk +1/k( I - KHk + 1)*

FI 3

n 2

FI

m

max(n 2, mn)

max(n z, mn)

n2m

Store in 2

n

Store in 5

Store in 6

rtl2n ~ m 2

~ m + 1 ) / 2 ~ m + 1 ) / 2

m 3 3m 2 + + mq Store in 11

2 2

m2n mn

mn Store in 4

mn Store in 1

n2m Store in 6

n 3 Store in 5

Store in 2

m2n Store in 5

n2 m ~ ~ / m n ( ~ ' n + 1)/2 Store in 2


T&U! 3. LEWTSQUARESmLTW


1 $2,

2 PO

3 C&l -I

4 R-l

5 Yis Yzs * * * 9 Ylv

6 Hi

7 Q(jv+ 1,O)

8 C@(iv+L o>lc&l

9 yr - Ml[Wv + 1, WJ

10 [H:I[R- ‘I

11 [H:R-‘][y,-Hi@(jV+ 1, O)fo]

12 [H:R- ‘l[H,]

13

14

15

[@*(jv+l, O)][~H:R-‘(Y,-H,~(jv+l. OR,)]

[@*(jv+ 1, W[~H:R-‘&I

[@*(jv+I, O)zH,*R-‘H,][@((jvfl, 0)] I

16 P o,l”=[P~l

17

18

+ C@*(jvfl, O)~H:R~lHj@(jv+l, O)]” i i

[@*(iv + L O)3[Po~J

[Q*(jv+L O)P,,J[~@*(jv+ 1, O)~H$-‘(Yi i

- H,@(jv + 1, 0)&J + Q( jv + LO)&,

19 P Ivllv = [WV+ 1, WJ,~d~*(jv f L 011

n3 32

Y-T +nq

n

Store in 2

s Wi

mn

n2

n2 n

mn m

m2n max(n*, mn)

&&

n2 n

n3 Store in 10

s Store in 2

n3 3*

If? 4nq Store in 2

n3 Store in 10

n2 Store in 1

Store in 2


TABLE 4. POTTER SQUARE ROOT FILTER


1 :Cklk

D1]2 2 ~k/k

max(n, m)

max(n 2, ran)

3 G - I = R -1/2 m(m + 1)/2

4 Yk+l max(n, m)

5 (I)(k + 1, k) max(n z, mn)

b'~ol]2 Pak/+ZX/k=CD(k + 1, '~:--k/* (= W,)

:~k + ,/* = [O(k + 1, k)]EYCk/k] ( = #~)

[a-'][y,+,]=(o, . . . . . o.)*

rl 3

/12

m(m+l)/2

/,/2

Store in 1

9 H k + t Store in 2

I0 [ G - ' ] [ H k + , ] = b = ( b , , b2 . . . . . bin)* nm(m+l)/2 Store in 5

i i s ,= [ W J * [ b , ] /12 Store in 2

13 ~/a~

14 Ew,]E~,] n 2 Store in 4

15 Es;*-[/(~, + ~/~,) Store in 2

16 (0 , - [b,]*E#J)/~, n + l Store in 1

17 ~c, + ~ = .~, + [ W ,s,] [(0, - b ? ~,) loq ] Store in 7

(in 1 when i=m)

18 = w , - + n 2 Store in 6

(in 2 when i = m)


TABLE 5. BELLANTONI AND DODGE SQUARE ROOT FILTER

307


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

"~k/k max(n, m)

p1/2 max(n 2, ran) k/k

G - 1 = R - 1/2 m(m + 1)/2

Yk+ 1 m

@(k+ 1, k) max(nZ ' m(2_+ 1))

p1/2 = E(I)(k + 1, k)]EP~/2] n 3 n 2 k + 1/k

= E (g + 1, k)] n n

Hk+ 1 mn

[G- 1][Hk+ 1] nm(m + 1)/2 Store in 2

Yk+ l -- [Hk+ 1][~k+ l/k] mn Store in 4

B,=EG-t a/2 Hk+ 1]l-Pk+ l/k] n2m Store in 8

I,. + [B*]l-B] nm(m + 1)/2 Store in 5

l-Im+B*B] -1 m S + 3 m 2 + m q Store in 5 2 2

l-G- 1]['yk+ 1 --Hk+ l~k+ 1/k] m(m + 1)/2 Store in 1

l-(Im + B*B)] - x l-G-l(yk + 1 -- Hk + 1 ~ + 1/k)] m 2 Store in 4

l'B] [(Im + B'B) - 1 G - 1 (Yk + I -- Hk + 1.~k + 1/k)] m n Store in 2

[p~/+2/k] l-B(Im-F B * B ) - I G - l ( y k +1 - H k +1Xk +1/k)] n 2 S to re in 1

+~k+t/k

[B]l-B*] mn(n + 1)/2 Store in 5

S Store in 2

P A Store in 4

D = I-I,. + A ] - 1/2 -1,,. mq Store in 4

[D][S*] mn Store in 8

[S][DS*] -I-I m mn(n + 1)/2 Store in 5

p112 n a k + ilk + 1 = [P~/+2/k][I, , , + S D S * ] Store in 2

308 I. A. GURA and A. B. BIERMAN

TABLE 6. ANDREWS SQUARE ROOT FILTER


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

1.7

18

19

20

"~k/k

~k

G=R ~

R

Yk+ 1

¢P(k + 1, k)

2k+ ,lk = [~(k + 1, k)][2k/k]

Pt+ ,/k = [O(k + 1, k)][Ptlk]

Hk+ 1

Z = ½" * [Pk +,/k] [nk +, ]

u u* = R + [z*] [z]

[u] , [ u - ' ]

Yk+ , - [Hk+ ,] [2~ + ,/k]

[m-'] [y~+, -H ,+ :~+ ,/Q

[ z ] [ ( v * ) - ' ]

[ u + c ] - '

[P t+ , / k ] [Z (V* ) - ' ]

[(u+a)-'][z*]

Xk+ 1/k+ 1 =2k+ 1/k

+ [PL~/kZ(t: *)- ' ] [ C- '(y~ +~ --HR +, 2k +,/k)]

Pk+ i/k+ l = Pk+ llk

+[P~+.~z(vu*)-'][(u+~)-'z*]

n 2

n 3

n2m

nm(m + 1)/2

m 3 _ m + - - + m 2 mq 3 3

m t l

m(m + 1)/2

nm(m + 1) /2

m3+,.2+._, 6 2 3

n 2 m

nm(m + 1)/2

m n

n 2 m

max(n, m)

max(n 2, ran)

m(m + l )/2

m(m + I)/2

m

max(n 2, m 2)

n

/,12

m 8

Store in 2

Store in 6

Store in 6

Store in 1

Store in 5

Store in 9

m(m + 1)/2

Store in 6

Store in 9

Store in 1

Store in 2


TABLE 7a. S UMMARYOFOPERATIONTOTAIS

Algorithm Totals

Standard filter

Stabilized filter

Least squares filter

No symmetry

Symmetry

(( $+3n+ m2 3 T+Zm+q+$m mv+$‘+$’

) ) 1.

No symmetry

cc 2!!f+m2 m

T+$+q+3n2+3nm+2n)mv+2n3+n2)l.

Symmetry

cc 3 n3 n2 ’ 3 ---+-+m+_m+q++2++m+3n mv+$‘++z2 1. 2m 2m 2 2 ) )

No symmetry

((mn+n2+2n)mv+2n3+2n2)Z+3n3+4n2+2nq.

Symmetry

((mn+~+~n)mv+$3++2)~+$3+$2+2nq.

Potter square root filter

Bellantoui and Dodge square root filter

~+~+2n2+mn+$+3m+5n+29+lt+l mv+n”+n’ m m m ii)

Andrew square root filter

310 I. A. GURA and A. B. BIERMAN

TABLE 7b. SUMMARY OF STORAGE TOTALS

Algorithm Totals

Standard filter No symmetry

n2 + 2n+ 2m2 + m + 2max(n 2, ran).

Symmetry

kn2 +~n+m2 + 2m+ 2max(n 2, ran).

Stabilized filter No symmetry

n2 + 2n + 2m2 + mn + m + 2 max(n 2, mn) .

Symmetry

½n 2 +~n + m 2 +mn +2m +2 max(n 2, ran).

Least squares filter No symmetry

3n 2 + 4n + m 2 + m + mn + mvl + max(n 2, ran).

Symmetry

2n 2 + 5n + m2 + 3m + mn + mvl + max(n 2, ran). 2 2

Potter square root filter n 2 + n + m2 + m + 2 max(n 2, ran) + 2 max(n, m). 2 2

Bellantoni and Dodge square root filter

n 2 +n+ m~+ ~m+mn+max(n2 ' mn)+max(n2 ' re(m+ 1)/2)+max(n, m).

Andrews square root filter n2+n+½m2+]m+mn+max(n 2, mn)+max(n 2, m2)+max(n, m).

6. EFFECTS OF PROCESS NOISE A N D NON- STATIONARY M E A S U R E M E N T STATISTICS

Table 8 below gives the adjustments required for making Tables 1-7 applicable to the system:

x~+ x =O(k+ 1, k)x~+u k for k=jv , j=O, 1, 2 . . . . . 1

Xk + 1 = Xk for all other k

yk+l=H~+xXk+t+Vk+l for all k.

where

g(vk + i ) = 0 , ¢ (vk + 1 v~, + 1) = Rk + 1

g( uD = O, ¢( ukuD = 9~

g(vAuD=0.

All algorithms presented can include the effects of the time varying measurement statistics by directly replacing R by Rk +1.

The standard and stabilized filter algorithms can accommodate forcing noise by using the covariance extrapolation equation:

Pk+ l/k=f~(k + 1, k)Pk/k~*(k + 1, k)+ Qk

for k=jv , . /= O, 1, 2 . . . . . l

while the square root filters must use the relation- ship:

~+,/k = [ ( ~ ( k + 1, k)Pt/O(,~(k + 1, k)P~:O* + Qd +

where Qk is defined above. The effects of forcing noise are not considered

in the Least Squares filter because of its general impracticality in such applications,


TABLE 8. ADDITIONAL REQUIREMENTS WITH NONSTATIONARY MEASUREMENT AND PROCESS NOISE

311

Algorithm Nonstationary measurement noise:

additional operations* (Mult/Div)

Process noise: additional operations*

(Mult/Div)

Standard Kalman filter None None

Stabilized Kalman filter None None

Least squares filter m3 + 3m2 -Jr mq)vl 5 - 5

(Computing R k 1)

Impractical

Potter square root filter

Bellantoni and Dodge square root filter

Andrews square root filter

(m~--~+m2--3+mq)vl

(Computing G~ 1)

(m--~ ÷ m2 --3÷ mq)vl

(Computing G~ "1)

m 3 + rrt 2 2m + m "~ T-5 q)

(Computing Gk x)

(~n3+m2--~n-l 'nq)

Computing vl/2 ~ ~t k+ 1/k/

(~na+m2--~n+nq)

Computing ~,x/2 k+ l/k/

2n3 + m 2-2n + nq) 3 3

(Computing or/2 ~k + I/k!

* Storage increase depends on specific problem.

7. CONCLUSIONS

The derived analytic expressions for operations counts were programmed for a digital computer and a series of parametric studies were performed to investigate the efficiency of each algorithm over the spectrum of operating conditions described below.

Para- meter

n

m

Description number of state variables

number of measurements received at one time

number of data vectors received between system dynamic updates

number of dynamic updates in the interval of comparison

Range in

study 1-100

1-24

1-24

1-96

Although it was difficult to study all possible combinations of the parameters involved, enough data was gathered so that important basic trends could be established. Over a wide parametric range, the storage required for the various algorithms generally differs by less than 25 per cent. Since the formulae for the storage requirements are relatively simple, expressions describing their asymptotic behavior as m and n vary were derived. The results, as presented in Table 9, are in complete agreement with the data.

Typical data for the operations counts are summarized in Figs. 2, 3 and 4. In Figs. 2 and 3, the number of operations is plotted against the dimensions of the state vector while in Fig. 4, the number of operations is plotted against the variable my. In all cases it is clear that the Bellantoni and Dodge and the Stabilized Kalman formulations are not nearly as efficient as the other algorithms. Indeed,


TABLE 9. ASYMPTOTIC BEHAVIOR Or STORAGE REQUIREMENTS

n>~m /l ,~, m

n ~ l n>>l m>> n

Standard Kalman 2"5n 2 3.5n(n + 1.3)

Stabilized Kalman 2.5n z 4.5n(n + 1)

Sequential least squares 3n z + mvl 4-5n(n + 1.4) + mvl

Potter square root 3n z 3.5n(n + 1)

Bellantoni and Dodge 3n 2 4.5n(n+0-8) square root

Andrews square root 3n 2 5"5n(n +0"8)

Standard Kalman 3n z 5n(n +0-6) (no symmetry)

Stabilized Kalman 3n z 6n(n +0.5) (no symmetry)

Sequential least squares 4n 2 + mvl 6n(n + 0.8) + mvl

(no symmetry)

3.5n a m 2

4.5n z rn 2

4.5n2 + mvl 0.5m2 + mvl

3.5n 2 0.5m 2

4.5n 2 m 2

5.5n 2 2.5m 2

5/,/2 2m 2

6n 2 2m 2

6nZ + mvl m2 + rnvl

the results for the Bellantoni and Dodge algorithm are actually overly optimistic since the eigenvalue- eigenvector operations count, It, was taken as 1/3 when in reality it is probably much larger [12]. Since this algorithm is extremely time consuming and does not offer any advantage in accuracy, it does not merit further consideration for practical

i

! - :2

FIG. 2.

[0 5 _ ~ , I

BELLANTONI AND DODGE - - - STABILIZED

10 4̀ _ __ KALMAN ....... ~,

ANDREWS

<

~ IO 3 -

~D

CD

k J

IO 2 __ ALL FORMULAE i EXPLOITMATRIX r SYMMETRY i

• q=7 I

. . . . SEQUENTIAL [ EAST SQUARES

" STANDARD KALMAN

" POTTER

! i

!o Io 2 DIMENSION OF STATE VECTOR,n

Operations count vs. dimension of state vector for m=l, v=24, 1=10.

estimation problems. From the figures, it is also clear that the Potter, Andrews and Standard Kal- man formulations are similar in behavior and quite efficient over a wide range of operating conditions. However, because of the advantages in accuracy, the Potter or Andrews algorithms are recommended over the Standard Kalman filter.

, r I 7 - ~

r ~ i

~2

i0 3

C ) c ~

k

~.n Io 2 E L ,

STABILIZEE: KALMAN-

BELLANTONI AND DODGE-

ANDREWS-

~STANDARDKALMAN

SEQUENTIAL LEAST SQUARES

o ~ Y' ALL fORMULAE EXPLOIT [ [ MAFRiX SYMMETRY , , r l : 2

]

L ,. L i i ! i I l l I I O 10 2

C',MENSION Oi: SIA FF VECibF,,~

Flo. 3. Operations count vs. dimension of state vector for m=24, v=l, I=10.


I000

500

,,? I0

g m0

0

~ 5o c ~ (D

I0

r/I I / 5 t P~-BELLANTONI / /

DODGE / / S AB,LIZED/ / AL Ay

/ / / \ STANDARD '\ \ KALMAN \ POTTER, / \ ANDREWS ~ SEQUENTIAL

\ I LEAST SIQllARES

/I~BELLANTONI ~ / / / k D°DGE / i j l / STABILIZED/" ~ _? _

/ / ; / / MATRIX SYMMETRY

/ [ ] [ I [ I

0 I0 40 70 I00 rnt/

FIG. 4. Operations count vs. mv for m = l , n--20.

The behavior of the Sequential Least Squares filter differs considerably from the other algorithms. While it is not as efficient as the other algorithms for many situations, it is unquestionably the most efficient when my is large. Note from the description of the basic system model in Section 3 that large m y occurs when the data rate is high compared to the rates of change of the state variables. The Sequential Least Squares algorithm is also attrac- tive for low n, high 1 cases, i.e. when the state vector is low dimensional and when there is no need for frequent state updates.

As indicated on the figures, all curves presented were based on the operation count formulae which consider matrix symmetry where possible. Counts based on formulae which ignore matrix symmetry are generally about 25 per cent higher. This, however, does not necessarily mean that observing matrix symmetry is always more efficient, since the additional indexing required for implementation of the resulting algorithms may more than offset the slight advantage gained in the operation count.

An interesting question which often arises in estimation problems is whether given my data points should be processed one point at a time (m= 1) or as a vector (v = 1), if both alternatives are possible. The solution to this problem becomes evident upon inspection of the operations totals in Table 7a when it is noted that my is constant. Clearly, for the Standard Kalman, the Least Squares, the Potter, and the Andrews methods, processing the data point by point (m-- 1) is always most efficient. However, both the Stabilized Kalman and the Bellantoni and Dodge Filters exhibit a crossover point as n increases where some value of m > 1 is

more desirable. This phenomenon is illustrated in the specific cases shown in Figs. 2 and 3.

To this point all considerations have been based on the assumption that the linear model (1) is perfect. When model errors occur, periodic corrections to the covariance matrix, i.e. deweighting of "old" data, must be made. Use of the "forcing noise" formalism results in modifications of the operation count formulae as described in Table 8. The effects of such changes degrade the efficiency of the square root algorithms, but leave the Standard and Stabilized Kalman filters essentially unaffected. Under such conditions the Standard Kalman filter becomes the most efficient algorithm; when frequent covariance corrections are required, the Stabilized Kalman filter could well surpass the efficiency of the square root methods.

The situation becomes even more complicated when nonlinear systems must be considered. The usual practical procedure in such cases is to itera- tively apply the formulae of Section 4 to linearized approximations to the given non-linear equations. If, however, filtering is performed over relatively short data intervals, iteration can be omitted with the last filtered estimate serving as a new lineariza- tion nominal for the next span of data. This sim- plification is clearly not applicable to the Sequential Least Squares filter since the essence of that algorithm is to process a large amount of data before updating a state estimate. Thus, the relative efficiency of this algorithm can be considerably degraded in a non-linear environment.

REFERENCES

[1] R. E. I ~ L M ~ : A new approach to linear filtering and prediction problems. Trans. ASME, J. bas. Engng. Ser. D 82) 35-45 (1960).

[2] A. E. BRYSON and Y. C. Ho: Applied Optimal Control. Blaisdell, Waltham, Mass. (1969).

[3] J. S. M~DrrCH: Stochastic Optimal Linear Estimation and Control. McGraw-Hill, New York (1969).

[4] H. W. SORENSON: Kalman filtering techniques. Advances in Control Systems, Vol. 3 (Ed. C. T. LEONDES). Academic Press, New York (1966).

[5] R. H. BAaqTN: Astronautical Guidance~ pp. 338-339. McGraw-Hill, New York (1964).

[6] J. F. B~LLANTONI and K. W. DODGE: A square root formulation of the Kalman-Schmidt filter. AIAA Journal 5, 1309-1314 (1967).

[7] A. ANDREWS: A square root formulation of the Kalman covariance equations. AIAA J. 6, 1165-1166 (1968).

[8] M. A o ~ : Optimization o f Stochastic Systems. Aca- demic Press, New York (1967).

[9] I. A. GtmA: Extension of linear estimation techniques to nonqinear problems. J. Astron. Sci. 15, 194-206 (1968).

[10] R. S. Buoy and P. D. JOSEPH: Filtering for Stochastic Processes with Applications to Guidance, pp. 141-142. Interscience, New York (1968).

[11] L. Fox: An Introduction to Numerical Linear Algebra. Clarendon Press, London (1964).

[12] E. BODEWIG: Matrix Calculus. North-Holland, Am- sterdam (1956).


R~sum6--Malgr6 le fait q'une grande attention ait et6 accord6e aux divers aspects du probl6me de filtrage, la question des exigences du calculateur n'a pas 6t6 analys6e d'une mani6re d6taillge. Le pr6sent article constitue une tentative d'analyser ce probl~me systgmatiquement. Des formules sont 6tablies pour les exigences des operations arithmetiques, du comptage et de la m6morisation avec plusieurs algorithmes de filtrage pr6f6r6s dans le cadre d'une probl6me d'estimation g6n6ral. Les param&tres inclus dans l'6tude sont les dimensions du vecteur d'&at, la fr6quence et le hombre d'observations et la fr6quence des remises h jour de l'6tat. Les resultats pour chaque algorithme sont diseutes d'un point de vue pratique et des comparaisons sont effec- tu6es pour certains cas int6ressants. Certaines conceptions erron6es communes sont expos6es.

Zusammenfassung--Obgleich den verschiedenen Aspekten des Filterproblems erhebliche Aufmerksamkeit geschenkt wurde, ist die Frage nach den Anforderungen an den Rechner noch nicht im Einzelnen analysiert worden. Hier wird eine systematische Analyse dieses Problems versucht. Formeln ftir die arithmetische Ztthlung der Operationen und die Anforderungen an die Speicher werden far mehrere tibliche Filteralgorithmen im Rahmen eines allgemeinen

Schhtzproblems abgeleitet. In die Studie eingeschlossene Parameter sind die Dimensionen des Zustandsvektors, die Frequenz und die Zahl der Beobachtungen und die Frequenz der Zustandsermittlung. Die Ergebnisse werden far jeden Algorithmus vom Standpunkt der Praxis diskutiert und fiir bestimmte interessierende Ffille Vergleiche angestellt. Mehrere allgemeine Mil3versttindnisse werden aufgedeckt.

Pe3IOMe---HCCMOTpA Ha TO qTo MHOFO BHHMaHH~ 61~LqO y~eJleHO pa331HqHblM acileKTaM npo6YlgMbl ~HJIbTpHpo- BaHH~/, BOnpOC Tpe6oBal~I~l IIH~poBo~ BblqHCnHTe3~bHO~I ManlIIHbl Re 6b13~ iipoaHa31H311poBan B BeTaa.qx. HacTo~lula~I CTaTb~I COCTaBJIfleT IIOHbITKy CllCT~MaTI4~KOFO aaann3a 9ro~ npoOneMbL Bbme~eHb~ c~opMy~bt ~In~ Tpe6oBannfi apnqbMeTnqecKnx onepaun~, c~eTa n 3anoMnnann~ c Hecro- JlbKI4MFI tIRCTO-MCHOYlB3yeMbIMH a~ropHTMaMH dpUm,TpHpo- BaHHa B paMKax o6~e~ npoO~eMbi ouertKu. ]71apaMeTp~l BKJIIOHeHHble B O63Op COCTOflT H3 pa3MepOB BeKTOpa COCTO- ~[HIJ~, 143 qaCTOTbI IR ancna Ha6nlO~eHH~ ~ }~3 qaCTOTbl I, I3MeHeHHfi nph'~rlOCHMblX COCTO~IHHIO no Bp-~MeUI4. Pe3ynb- TaTbt K a ~ o r o aYwopnTMa 06cyac~atoTcn c HpaKTHqecKo~ TOqKR 3peHH~l rt HpOBO~lflTC~I cpaBneni~Dl ~.nfl tteKoTOpblX tlHTepeCHblX cny•aeD. YKa3blgalOTC~l HeKOTOpble 06Utile OmH6OqHble KOHt~enTbl.

on computational efficiency of linear filtering algorithms

Documents