on kalman filtering, posterior mode estimation and fisher scoring.pdf

7/28/2019 On Kalman Filtering, Posterior Mode Estimation and Fisher Scoring.pdf

1/24

M etr ika (1991) 38 :37 - 60

On Kalman Fi l te r ing , Pos ter ior Mode Est imat ion andFisher Scor ing in D ynam ic E xpon ent ia l Fami lyRegression

B y L . F a h r m e i r I a n d H . K a u f m a n n l ' 2

S u m m a r y :Dynamic exponent ia l fami ly regress ion provides a f ramework for nonl inear regress ionana lys i s w i th t ime dependen t pa rame te r sflo , f l l , . . . , f i t , . . . , d i m f l t= P. In add i t ion to the fam i l ia rcondi t iona l ly Gauss ian model , i t covers eg . models for ca tegor ica l o r counted responses . Parametercan be e s t ima ted by e~ t ended Ka lman f i lt e ri ng and smoo th ing . I n t h i s pape r, f u r the r a lgo r i t hms a r ep re sen ted . They a r e de r ived f rom pos t e r i o r mo de e s t ima t ion o f t he w ho le pa ram e te r vec to r (fl~) . . . f l' t) by G auss-N ewton resp. F isher sco r ing i te ra tions . Fac tor iz ing the in form at ion m at r ix in to b lockbid iagonal mat r ices , a lgor i thms can be g iven in a forward-backward recurs ive form where on ly inve r se s o f " sma l l " p p-m a t r i c e s occur. Approx im a te e r ro r cova r i ance ma t r i ce s a re ob t a ined by an i nve r s ion fo rmu la fo r t he i n fo rma t ion ma t r i x , wh ich i s exp l i c i t up t o pp-ma t r i ce s .

1 I n t r o d u c t i o n

L e t a r e g r e s s i o n r e l a t i o n s h i p b e t w e e n a r e s p o n s e v a r i a b l e Y t a n d c o v a r i a t e sx t b eo b s e r v e d s e q u e n t i a l l y i n t im e . T h e n i t i s o f t e n a p l a u s i b l e a s s u m p t i o n t h a t t h e u n -k n o w n r e g r e s s io n p a r a m e t e r s a r e a l s o t i m e d e p e n d e n t :

y t = x ' t f l t + e t , t = 1 , 2 . . . . . ( 1 . 1 )

S u p p o s i n g a l i n e a r t r a n s i t i o n e q u a t i o n

~ t = T t 1 ~ t - I q - u t, t = 1 , 2 . . . . ( 1 . 2 )

1 Ludwig Fah rme i r an d He inz Kau fm ann , Univers it~ it Regensburg , Lehrs tuh l fiar S ta t i s tik ,Universit~itsstraBe 31, I)-840 0 Re gen sbu rg.2 He inz Leo Kau fman n , my f r i end and coa u tho r fo r m any yea rs , d i ed i n a t rag ica l r ock c limb inacc iden t in Augus t 1989 . Th is paper i s ded ica ted to h i s memory.

0 0 26 -1 33 5 /9 1 / 1 / 37 -6 0 $2 .50 1991 Physica-Verlag, H e ide lbe rg


2/24

38 L. Fahrmeir and H. Kaufman n

f o r th e r e g r es s io n p a r a m e t e r s a n d m a k i n g c e r ta i n a s s u m p t i o n s o n t h e n o i se p r o -cesses [ e t ] , [ v t ] ,eq ua t ion s (1 .1 ) an d (1 .2 ) con s t i tu te a l inear s t a te space m od e l (e.g .

Sage an d M elsa 1971, ch . 7 , A nd ers on an d M oo re 1979, ch. 2). Given the observa-t io n s y~ . . . . Y t, e s t i m a t i o n o f f it ( " f i l te r i n g " ) a n d o f fl0 . . . . fit- 1 ( " s m o o t h i n g " ) ,t oge the r w i th co r r e s po nd i ng e r ro r co v a r i an ce m a t r i c e s , i s o f p r im a ry i n t e r e s t .These t a sk s a r e s o lv ed r ecu r siv e ly by t he l i ne a r K a l m an f i lt e r and t he f i x ed i n t e rva ls m o o t h e r.

The obse r v a t i on equ a t i on ( 1 .1 ) i s app ro p r i a t e f o r m e t r i c r e s ponse s . Gene ra l -i z ed l i n e a r m o d e l s ( Ne l d e r an d W ed d e rbu rn 1 972) p rov i de a f r a me w ork fo r r eg re s-s i on a na ly s i s wh e re t he d i s t r i b u t i on o f t he r e spo ns e va r i a b l e Yt be longs t o ana tu r a l expo ne n t i a l f ami ly. Un i va r i a t e ex am p le s a r e t he no rma l , b inomia l ,

P o i s s o n a n d g a m m a d i s t r ib u t i o n , m u l t iv a r i a te ex a m p l e s t h e m u l t i n o r m a l a n d t h emu l t i nomia l d i s t r i b u t i o n . T hus , be s i d e s m e t r i c Gaus s i an r e spon s e s , gen e ra l i z edl i nea r mode l s a l l ow f o r c a t eg o r i c a l , coun t ed and no n neg a t i ve me t r i c r e sponsevar iables .

S ince suc h r e spo nse s a r e c ommon i n l on g i t ud ina l a s we l l a s c ro s s - s ec t i ona lana lyses , i t i s o f cons iderab le in te res t to ex tend s ta t i c genera l i zed l inear mode lst o a d y n a m ic s e t ti n g l ike t he o ne abov e . S uch ex t ens ions ha ve been sug ges t ed byWes t , Ha r r i so n a nd Migo n (1 98 5 , un i va r i a t e r e spon s e s ) an d F ah r me i r ( 1 988 ,m u l t i v a r i a t e r e sp on s e s) . I n bo th p a pe r s, t he ob s e rv a t i on e qua t i on (1.1 ) is rep l a cedb y s p e c if y in g th e c o n d i t io n a l d i s t ri b u t i o n o fY t , given x tand f it , i n an a log y t o t heco r r e spond in g sp e c i f i c a t i o n i n s t a t i c ge ne r a l i z ed l i ne a r mod e l s . I n o r de r t of ac i li ta t e t h e u se o f t h e d i s c ou n t c oncep t ( A m ee n a n d H a r r i so n 1985), t he tr an s i -t ion e qu a t i on (1 .2 ) is gen era l i zed som ew hat in the f i r s t paper, w hereas i t is r e ta in -ed i n t he s eco nd .

A fu l l Bayes i an f i lt e r wo u ld g i ve a r e cu r sive upd a t e o f t he wh o l e po s t e r i o r d en -si ty P ( f l t l Y l . . . . . Y t ) .In the l inear mode l (1 . I ) , (1 .2 ) wi th Gauss ian e r ro rs , th i sp o s t e r io r d e n s i ty is al so G a u s s i a n , w h e n c e u p d a t i n g t h e m e a n a n d t h e c o v a r i a n c em at r ix su ff i ces. In genera l , however, pos te r io r de ns i t i e s a re no t ava i l ab le in c losedfo rm, f o r c ing nu m e r i ca l i n t eg r a t i o n . Us ing sp l i ne func t i ons , s u ch a f i l t e r ha sr e ce n t l y b e e n p r e s e n t ed b y K i t a gawa (1987). S in ce th i s app r oac h becom es com -p u t a t i o n a l l y u n f e a s i b le f o r h i g h e r d i m e n s i o n a l p a r a m e t e r v e c t o rs a n d l arg e d a t ase ts , recurs ive f i lt e r s wh ich a re s imi la r to the l inear K a lm an f i l te r a re a l so a t t r ac -ti ve . S i m i l a r r e m a rk s ap p ly t o sm oo th ing .

B a s e d o n d i s c o u n t i n g a n d o n u s i n g c o n j u g a t e p r i o r -p o s t e r io r d i s tr i b u t io n s f o rt he l i nea r p r ed i c t o rx ~ f l t ,W est e t a l . (1985) prese nt su ch a f i lter. Ho weve r, the i rap p r o ach r a is e s a n um be r o f p rob l ems , i n pa r t i cu l a r i n ex t en s ion t o m u l t i va r i a t e

responses . These p rob lem s a re d i scussed in Fa hrm ei r (1988), an d a d i ffe re n t fi l te ra p p r o x i m a t i n g t h e p o s t e r i o r m o d e i s p r o p o s e d . I t i s t e r m e d e x t e n d e d K a l m a nf il te r, s in c e it is a na lo g ous t o t h e ex t e n ded K a lm an f il t e r fo r G a us s i a n r e sp onse swh ich ap p l ie s i fx ~ f l ta n d F t f l tin (1 .1 ), (1 .2 ) a re gen era l i zed to non l ine ar s m oo thf u n c t io n s o ff i t (Sage and Melsa 1971 , ch . 9 , Anderson and Moore 1979 , ch . 8 ) .

I n t h i s pap e r, t he r e l eva n t mod e l s a r e d e f i ne d i n Sec t i on 2 .1 . Th ey a r e r e f e r r edto a s m o de l s f o r d y nam i c e xpo nen t i a l f ami ly r eg re s sion . Ac tua ll y, t h ey ex t end dy -namic gene r a l i z ed l i n e a r mod e l s (Sec t i on 2 .2 ) . They a l so c ove r cond i t i ona l l y


3/24

On Kalm an Filtering 39

G a u s s i a n r e sp o n s e s c o m m o n in e x t en d e d K a l m a n f il te r in g ( S e c ti o n 2 .3 ). S o m eq u a n t i ti e s o f r e l e v a n c e i n e s t i m a t i o n a r e n o t e d i n 2.4 .

I n S e c t i o n 3, fi rs t t h e e x t e n d e d K a l m a n f i lt e r a n d s m o o t h e r a r e g i ve n . T h e s ea l g o r i t h m s a p p l y i f s m o o t h i n g i s c a r r i e d o u t a f t e r t c o n s e c u t i v e f il te r s te p s. A t t h ee n d o f 3 .2 , w e o b t a i n a n i n t e g r a te d a l g o r i t h m w h e r e s m o o t h i n g b a c k t ak e s p l ac ea f t e r each f i l t e r s t ep .

I n S e c t i o n 4 , t h e e s t i m a t i o n p r o b l e m is t a k e n u p f r o m a d i f f e r e n t p o i n t o f v ie w.We c o n s i d e r p o s t e r i o r m o d e e s t i m a t i o n o f t h e w h o l e s e q u e n c ef lo . . . . . f i t ,i.e.m a x i m i z i n g t h e p o s t e r i o r d e n s i t y o ff l o . . . . . f i t ,g iven y ~ . . . . . Y r .A l g o r i t h m i c a l l y,t h i s c a n b e p e r f o r m e d b y G a u s s - N e w t o n o r F i s h e r- s c o r i n g i t e r a t i o n s , r e p l a c i n g i nt h e l a tt e r c a se t h e r a n d o m i n f o r m a t i o n m a t r i x ( n eg a ti v e s e c o n d d e r i v at es o f t h e

l o g p o s t e r i o r d e n s i t y ) b y s o m e o t h e r i n f o r m a t i o n m a t r i x .Fo r s t a t i c genera l i zed l inea r mo de l s w here the f l-vec to r s a re a ll equa l , t h i s ap -

p r o a c h is f o l l o w e d b y W e s t ( 1 98 5 , S e c t i o n 4 . 1) . I n t h e d y n a m i c s e t ti n g , i t m a y a tf ir s t s e e m u n f e a s i b l e c o m p u t a t i o n a l l y : i f p is t h e d i m e n s i o n o f fls, s = 0 . . . . . t,t h e n t h e t o ta l n u m b e r o f p a ra m e t e rs i s p ( t + l ) , a n d a c o r r es p o n d i n g l y la rg es y s t e m o f e q u a t i o n s m u s t b e s o l v e d a t e a c h i t e r a t i o n . H o w e v e r, t h e i n f o r m a t i o nm a t r i x , w h i c h f o r m s th e c o e f f i c i e n t m a t r i x o f t h is s y s t em , h a s a b l o c k t r i d i a g o n a ls t ruc tu re (Sec t ion 4 .1 ) . Th i s s t ruc tu re i s inves t iga ted fu r the r in 4 .2 , where a p ro -p o s i t i o n o n f a c t o r i z a t i o n a n d i n v er s io n o f t h e i n f o r m a t i o n m a t r i x i s g iv e n. T h i s

p r o p o s i t i o n is b as i c f o r t h e r e s u lt s w h i c h f o l lo w. T h e f a c t o r i z a t i o n p a r t l e a d s t oa s i m p l e a n d e f f ic i e n t f o r w a r d - b a c k w a r d r ec u rs iv e i m p l e m e n t a t io n o f G a u s s -N e w t o n r es p . F i s h e r s c o r i n g i t e ra t i o n s ( S e c t i o n 4.3 ). T h e i n v e r s io n p a r t y i e ld s a p -p r o x i m a t e e r r o r c o v a r i a n c e m a t r i c e s. I n b o t h p a r t s , o n l y i n v er s es o f " s m a l l "p p -m a t r i c e s o c c u r.

I f G a u s s - N e w t o n ( F i s h e r s c o ri n g ) i t e ra t i o n s a r e a p p l i e d s e q u e n t i a l ly in t im e ,o n e m a y c o n j e c t u r e t h a t i n m a n y c a s e s t h e n e w o b s e r v a t i o n d o e s n o t c h a n g ees t ima tes too d ras t i ca l ly, wi th the impl ica t ion tha t a s ing le i t e ra t ion su ff i ces . Ther e s u l ti n g a l g o r i t h m is c o n s i d e r e d i n S e c t i o n 4 .4 .

R e l a t i o n s h i p s b e t w e e n t h e v a r i o u s a l g o r i t h m s a r e d i s c u s s e d i n S e c t i o n 5 . I t i ss h o w n t h a t t h e y f o r m a h i e r a r c h y o f a p p r o x i m a t i o n s t o p o s t e r i o r m o d e e s t i m a -t io n . W i t h r a t h e r d i f f e re n t arg u m e n t s , e x t e n d e d K a l m a n f il te r in g a n d s m o o t h i n gi s d e r i v e d a s a n a p p r o x i m a t i o n t o p o s t e r i o r m o d e e s t i m a t i o n i n S a g e a n d M e l s a( 1 9 7 9 , c h . 9 ) , f o r G a u s s i a n e r r o r s e q u e n c e s . A l t h o u g h t h e i r a rg u m e n t s e x t e n d t oe x p o n e n t i a l f a m i l ie s ( F a h r m e i r 1 9 88 ), t h e d e r i v a t i o n g i v e n h e r e s h e d s n e w l i g h t o nt h i s r e l a ti o n s h i p . I t s u g g e s ts f o r w a r d - b a c k w a r d r e c u r s iv e e s t i m a t i o n a l g o r i t h m sw h i c h s e e m t o b e n e w a ls o i n th e G a u s s i a n c a se , a n d i t c l a r if i e s t h e n a t u r e o f a p -

p r o x i m a t i o n s . Q u a n t i t a t i v e a s s e r t i o n s o n t h e q u a l i t y o f a p p r o x i m a t i o n s a r edes i rab le . Th i s top ic wi l l be t r ea ted in subsequen t work .


4/24

4 0 L . F a h r m d r a n d H . K a u f m a n n

2 Dynam ic Exponential Fam ily Regression

2 . 1 A s s u m p t i o n s

T h e m o d e l s o f th is p a p e r f o r m a d y n a m i c e x te n s io n o f n o n l i n e a r e x p o n e n t i a lf a m i l y r e g r e s s i o n m o d e l s . L e tY l , Y 2. . . . b e a s e q u e n c e o f o b s e r v a t i o n s( r e s p o n s e s ) , w h e r e e a c h Yt r a n g e s i n a s u b s e t Y o fR q. I n p a r a l l e l , t h e r e e v o l v e sa s e q u e n c e f l o ,f l l . . . . o f u n o b s e r v a b l e p - d i m e n s i o n a l s t at e o r p a r a m e t e r v e c to r s.L e t

y * = ( y ] . . . . . y ; ) ' , f l * = ( , 8 ~. . . . . f l} )' , t = 0 , 1 , 2 . . . .

d e n o t e t h e f i r s t t o b s e r v a t i o n s r e s p . t + 1 p a r a m e t e r v e c t o rs , w h e r e y ~ is t o b er e a d a s a n e m p t y v e c to r. I n v i e w o f t h e d y n a m i c n a t u r e , i t is n a t u r a l t o m o d e l t h ec o n d i t i o n a l d i s t r i b u t i o n o fY t g i v e n Y * - I , t h e p a s t o b s e r v a t i o n s , a n d fl* , t h eh i s t o r y o f t h e p a r a m e t e r p r o c e ss , i n c lu d i n g t h e a c t u a l p a r a m e t e r v e c to r .S p e c if ic a ll y, d e n o t i n g c o n d i t i o n a l d e n s it ie s b y P ( " I ") a s w e s h a l l d o t h r o u g h o u tt h e p a p e r , w e s u p p o s e

P ( Y t l f l * , Y * - , ) = *( Y t [ f l t , Y t - O , t = 1 ,2 . . . . . (2 .1)

L o o s e l y s p e a k i n g , g i v e n Y * -1 , t h e a c t u a l p a r a m e t e r v e c t o rfit c o n t a i n s t h e s a m ei n f o r m a t i o n o nY t a s t h e w h o l e f l * .

T h e c o n d i t i o n a l d e n s i t y i n ( 2 . 1 ) i s a s s u m e d t o b e o f t h e n a t u r a l e x p o n e n t i a lt y p e :

l o g P O ~ t l ~ t ,Y * - l ) = O ~ Y t - b t ( O t ) - c t ( Y t ), t = 1 ,2 . . . . . (2 .2)

w h e r e c t i s a m e a s u r a b l e f u n c t i o n o n Y. T h e d e n s i t y i n ( 2 . 2 ) i s f u l l y s p e c i f i e do n c e t h e n a t u r a l p a r a m e t e r0 t i s g i v e n as a f u n c t i o n o f t h e c o n d i t i o n i n g q u a n -t it ie s . L e t O d e n o t e t h e n a t u r a l p a r a m e t e r s p a c e o f ( 2 .2 ) ( e.g . F a h r m e i r a n d K a u f -m a n n 1 98 5). We s u p p o s e t h a t O c o n t a i n s i n t e r io r p o i n t s . T h e n i n th e i n t e r io r O t h e f u n c t i o n b t is d i f f e r e n t i a b le i n f i n it e l y o f t e n , a n d t h e c o n d i t i o n a l m e a n a n d

c o v a r i a n c e m a t r i x a r e g i v e n b y

O b t ( O t )P t - - - , (2 .3 )

OOt

" ~ 't - - 0 2 b tO t ) ( 2 . 4 )

OOtO0~


5/24

O n K a l m a n F i l te r in g 41

W e a s s u m e t h a t Z " is p o s i t i v e d e f i n i t e ( b r i e f l y 27 > 0) o n O , i n d u c i n g t h a t ( 2 .3 )d e f in e s a o n e - t o - o n e m a p p i n g f r o m 690 o n t o

M = Ob t ( 0 o ) .8 0 t

T h u s , i n s t e a d o f O t , o n e c a n e q u i v a l e n t ly s p e c if y t h e c o n d i t i o n a l m e a n / a t . Wea l l o w f o r t h e g e n e r a l s p e c i f i c a t i o n

/ a t h t *( J 3 t , Y t _ 0 , t = 1 , 2 , . . . , ( 2.5 )

w h e r e h t : R p x y t - I ~ M is a n a r b i t r a r y m e a s u r a b l e f u n c t i o n o f th e c o n d i t io n i n gq u a n t i t ie s , o n l y s u b j e c t t o t h e r e q u i r e m e n t o f b e i n g t w o t i m e s c o n t i n u o u s l y d i f-f e r e n t i a b l e w i t h r e s p e c t t o f i t . A n a l o g o u s m o d e l l i n g i n t h e s t a t i c s e t t i n g h a s b e e np r o p o s e d b y J O rg e n s en ( 1 98 3 ), f o r m o r e g e n e r a l d e n s i ti e s th a n t h o s e o f t h en a t u r a l e x p o n e n t i a l t y p e .

O f c o u r s e , i n r e g r e s s i o n a p p l i c a t i o n s P t d e p e n d s n o t o n l y o n l a g g e d v a l u e sY ~ ' - I o f t h e r e s p o n s e , b u t a l s o o n c o v a r i a t e s , t y p i c a l l y f o r m i n g a p a r a l l e l p r o c e s s{xt ] . I f s u c h c o v a r i a t e s ar e p r e s e n t, w e a s s u m e t h a t t h e y a r e d e t e r m i n is t ic , i n d u c -i ng t h a t f o r m a l l y t h e y c a n b e a b s o r b e d i n t o t h e s u b s c r ip t o fh r In t h i s s ense ,( 2 . 5 ) c o v e r s t h e s e e m i n g l y m o r e g e n e r a l

~at = h t ( f l t , Y * - l , x l . . . . . x t )

C o n d i t i o n i n g i n (2 .1 ), ( 2 .2 ) a d d i t i o n a l l y o n x 1 . . . . x t, a r a n d o m c o v a r i a t e p r o -c es s c a n a l so b e h a n d l e d . U n d e r a f u r t h e r c o n d i t i o n a l i n d e p e n d e n c e a s s u m p t i o n ,i n t e r p r e t i n g t h e l i k e l i h o o d g i v e n l a t e r a s a p a r t i a l l i k e l i h o o d , r e s u lt s c o n t i n u e t oh o l d . F o r s i m p l i c i t y, w e d o n o t d i s c u s s t h i s i s s u e a n y f u r t h e r.

E q u a t i o n ( 2.5 ) c o r r e s p o n d s t o ( 1.1 ) o r, m o r e g e n er a ll y, t o th e o b s e r v a t i o ne q u a t i o n is n o n l i n e a r f i lt e ri n g . I t m u s t b e s u p p l e m e n t e d b y a t r a n s i t io n e q u a t i o n( c o m p a r e ( 1 . 2 ) ) , a s s u m e d t o b e l i n e a r f o r s i m p l i c i t y :

B t = T t B t - t + v t ,

f lO = O lo + U o

t = 1 , 2 . . . . .(2 .6)

T h e p x p - t r a n s i t i o n m a t r i c e s T 1, T 2 . . . . a r e n o n r a n d o m , a n d a 0 is a n o n r a n d o mp - v e c t o r. T h e e r r o r p r o c e s s{vt] i s s u p p o s e d t o b e n o n d e g e n e r a t e G a u s s i a n w h i t en o is e , i.e. a s e q u e n c e o f i n d e p e n d e n t r a n d o m v a r i a b l e s w i t h


6/24

42 L. Fahrmeir and H. Kau fma nn

v t - N ( O , O t) , t = 0 , 1 . . . . . (2 .7 )

w h e r e Q t is p o s i ti v e d e f i n i t e ( Q t > 0 ).I n ( c o n d i t io n a l l y ) G a u s s i a n f il te r in g , t h e n o i s e p r o c e ss e s o f t h e o b s e r v a t i o n

a n d t h e tr a n s i t io n e q u a t i o n s a r e u s u a l ly a s s u m e d t o b e i n d e p e n d e n t . I n o u r m o r eg e n e r a l s e tt in g , th i s a s s u m p t i o n m u s t b e r e p l a c e d b y a f u r t h e r c o n d i t i o n a l in -d e p e n d e n c e a s s u m p t i o n :

p ( f l t l f l ~ - , , y * l ) = p ( f l t l f l * O , t =1 ,2 . . . . . (2.8)

I n v i e w o f ( 2.6 ) a n d ( 2 .7 ), th i s is e q u i v a l e n t t o th e a p p a r e n t l y s t r o n g e r a s s u m p t i o n

p ( / 3 t l f l* _ t , y t * _ 0 = p ( / 3 t [ f l t _ l ) , t =1 ,2 . . . . . (2.9)

T h r o u g h o u t th e p a p e r, w e s u p p o s e t h a t t h e p r o b a b i l i ty m o d e l is c o m p l e t e l ys p e c i f i e d , i . e . c o n t a i n s n o u n k n o w n d e t e r m i n i s t i c " h y p e r s t r u c t u r a l " p a r a m e t e r s .T h u s

a O' 0 0 , 0 1 . . . . . T 1 , T 2 . . . .

a r e a ss u m e d t o b e k n o w n , a s w e ll a s a n y o t h e r n o n r a n d o m q u a n t it ie s a p p e a r i n gi n t h e e x p o n e n t i al f a m i l y d e n s i t y o r in t h e m o d e l e q u a t i o n f o r B t.

2 .2 D y n a m i c G e n e r a l i z e d L i n e a r M o d e l s

T h i s m o d e l f a m i l y is o b t a i n e d b y sp e c i a li z in g t h e o b s e r v a t i o n e q u a t i o n ( 2 .5 ) t o

B t = h ( Z ' t f lt ) , t = 1 ,2 . . . . (2 .10)

w h e r e h : R r ~ M is th e t w o t im e s c o n t i n u o u s l y d i f f e re n t i a b le " r e s p o n s e f u n c -

t i o n " , a n d Z t is a p r - m a t r i x d e p e n d i n g o n l a g g e d v a l u e s y ; -1 a n d o n c o v a r i a t e s.I t is a s s u m e d t o b e k n o w n b e f o r e t h e o b s e r v a t i o nY t is m a d e , s o t h a t i t m a y b er e f e r r e d t o a s t h e p r e d e t e r m i n e d m a t r i x ( in t h e o b s e r v a t i o n e q u a t io n ) . D y n a m i cm o d e l i n g o f{Zt}c a n b e p e r f o r m e d a l o n g t h e li ne s d e s c r i b e d in K a u f m a n n ( 19 8 7) ,F a h r m e i r a n d K a u f m a n n ( 1 9 8 7 ) .

I f q = r = 1 , w e g e t a f a m i l y o f u n i v a r i a t e d y n a m i c g e n e r a l i z e d li n e a r m o d e l sw h i c h a r e r e la t e d to , y e t d i f f e r e n t f r o m t h e m o d e l s o f We st, H a r r i s o n a n d M i g o n( 19 8 5) . F o r c o n t i n u o u s r e s p o n se s , th i s f a m i ly c o v er s c o n d i t io n a l l y G a u s s i a n a n d


7/24


G a m m a m o d e l s . I f t h e r a n g e Y o f t h e o b s e r v a t i o n s i s d i sc r et e, t h e d e n s i t y i n (2 .1 ),( 2 .2 ) m u s t b e t a k e n w i t h r e s p e c t t o c o u n t i n g m e a s u r e , l e a d i n g e.g . t o t h e l o g - l i n e a r

P o i s s o n m o d e l f o r c o u n t e d d a t a a n d t o lo g i t a n d p r o b i t m o d e l s f o r b i n o m i a l d a t a.A p a r t f r o m t h e c o n d i t i o n a l l y G a u s s i a n m o d e l ( se e 2 .3 ) , t h e m o s t in t e r e s ti n gm u l t i v a r i a t e ( q > 1 ) m o d e l s a r e th o s e f o r m u l t i c a t e g o r i c a l o r m u l t i n o m i a l o b s e r v a -t io n s . M o r e d e t a il s a r e g iv e n b y F a h r m e i r ( 1 9 8 8) . I n c o n t r a s t t o t h e s t a n d a r d f o r -m u l a t i o n w h e r e q = r a n d h is a o n e - t o - o n e m a p p i n g , q a n d r m a y d i f f e r i n t h ep r e s e n t s e t ti n g . T h e r e f o r e ( 2 .1 0 ) i n c lu d e s a d y n a m i c e x t e n s i o n o f t h e c o m p o s i t el in k f u n c t io n m o d e l s o f B a k e r a n d T h o m p s o n ( 1 98 1).

L e t u s f i n a l ly n o t e a n a d d i t i o n a l r e s t r ic t i o n n o t n e c e s s a r y in s ta t ic g e n e r a l i z e dl i n e a r m o d e l s . S i n c ef i t c a n v a r y in t h e w h o l e R p , th e r e s p o n s e f u n c t i o n h m u s t

b e d e f i n e d o n t h e w h o l e R ~. T h i s r u l e s o u t e.g . h ( y ) = I / y , y > 0 , w h i c h is t h en a t u r a l r e s p o n s e f u n c t i o n f o r th e g a m m a d i s t ri b u t io n , o r t h e l in e a r P o i s s o nm o d e l ( h ( y ) -- y > 0 ) . T h e p r o b l e m m a y b e o v e r c o m e b y u s in g o t h e r p r i o r d i s tr i b u -t io n s , b u t t h i s w o u l d d e s t r o y t h e s i m p l ic i ty o f t h e n o r m a l p r i o rs a p p r o a c h .

2 .3 C o n d i t i o n a l l y G a u s s i a n O b s e r v a t i o n s

T h e s t a n d a r d f o r m o f t h e n o n l i n e a r o b s e r v a t i o n e q u a t i o n ( e.g . A n d e r s o n a n dMoore 1979 , ch . 8 , fo rmula (1 .2 ) ) i s

Y t = h f ( f l t ) + e t, t = 1 , 2 . . . . . ( 2 .11 )

w h e r e l e t / i s a w h i t e n o i s e s e q u e n c e , i n d e p e n d e n t o f t h e e r r o r s e q u e n c e[v t}o f t h et r a n s i t i o n e q u a t i o n , w i t h e t - N ( O ,Z t ) , Z ' t > 0 . T h u s , g i v e nf i t and (Yt*-0, the

o b s e r v a t i o nY t is c o n d i t io n a l l y G a u s s i a n , a n d t h is f i ts i n t o o u r f r a m e w o r k . E q u a -t i o n ( 2 .5 ) o b v i o u s l y h o l d s , a n d t h e e x p o n e n t i a l f a m i l y a s s u m p t i o n is m e t w i t h

O t = Z ~ l l ~ t , b t = O ~ Z t O t

T h e c o v a r ia n c e m a t r ixZ t is a ( n o n r a n d o m ) p a r a m e t e r o f th e c o n d i t io n a l d e n s i t ys u p p o s e d t o b e k n o w n , a c c o r d i n g to th e a s s u m p t i o n s a t t h e e n d o f 2 .1 . M o r eg e n e r a l ly t h a n (2 .11 ), o u r h y p o t h e s e s a l lo w f o r c o n d i t i o n a l l y G a u s s i a n o b s e r v a -t i o n s w i t h

/ at = h t ( f l t , Y * - I )

d e p e n d i n g o n p r e v i o u s o b s e r v a t i o n s .


8/24

44 L . F ah rme i r and H . Kau f man n

M o r e s p e c i a ll y, i f t h e m o d e l ( 2.1 0 ) is t a k e n w i t h h t h e i d e n t i t y m a p p i n g , t h e nw e g e t b a c k t h e m o s t f a m i l ia r l in e a r s t a t e s p a c e m o d e l .

2 .4 C o n t r i b u t i o n to th e L o g L i k e l i h o o d a n d R e l a te d Q u a n t i t i e s

L e t u s g i v e s o m e p r e l i m i n a r y e x p r e s s i o n s f o l l o w i n g f r o m t h e e x p o n e n t i a l f a m i l ya s s u m p t i o n , w h i c h a r e n e e d e d l a t e r o n . A c c o r d i n g t o S e c t i o n 2 .1 , i n s te a d o f / . t tw e c a n m o d e l t h e n a t u r a l p a r a m e t e r b y

O t t ( f t , Y t - O , t = 1 ,2 . . . . . ( 2 .12 )

w i t h a f u n c t i o n w t : R p y t - 1~ 0 0 . T h e r e l a t io n s h i p b e t w e e n w t a n dh t is

O bh t = O b _ _ _ Z o w t , w t = o h t . (2. t3)

0 0 , \ ~ , , )

S t r e s s i n g d e p e n d e n c e o n t h e p a r a m e t e rf i t , w e w r i t e

, W *f i t ( fl i t ) = h t ( f t , . , v ' ~ - l ) O t ( f t ) = t ( f t , Y t - 1 )

a n d Z t ( f t ) f o r t h e c o n d i t i o n a l c o v a r i a n c e m a t r i x , i n s e r t in g 0 t ( f t ) i n to ( 2 .4 ).D e f i n e f u r t h e r f i r s t d e r i v a t i v e s

0 h t 0 w t

F r o m ( 2 .1 3 ), w e g e t

H t ( f t ) = W t ( f t ) Z t ( f t ) . (2.14)

F i n a l l y, d e n o t i n g t h e c o m p o n e n t s o fw t b y w t j , j = 1 . . . . q , le t

0 2 W t jV t j ( f t ) - O J ~ t O f l ' t ,j = 1 . . . . . q .

T h e c o n t r i b u t i o n o f t h e o b s e r v a t i o nY t t o t h e l o g l i k e l i h o o d i s


9/24

On Ka lm an F i l t e ring 45

l t ~ J t ) = O ' t Y t - b t ( O t ) - c t ( Y t ) , ( 2 . 1 5 )

i n s e r t i n g O t = 0 t ( B t ) . T h e c o n t r i b u t i o n t o i t s f i r s t d e r i v a t i v e , t h e s c o r e f u n c t i o n , i s

r t ( f l t ) = W t ( Y t - P , ) :H t Z ( I Y t - P t ) , (2.16)

w i t h t h e q u a n t i ti e s o n t h e r i g h t e v a l u a t e d a tf i t , a s i n t h e s e q u e l . A s t h e c o n t r i b u -t i o n o f Y t t o th e i n f o r m a t i o n o n fit, w e m a y c o n s id e r t h e r a n d o m i n f o r m a t i o n

G t ( /~ t ) - 02I t ,

in c o n f o r m i t y w i th c o m m o n p r a c ti c e in p o s t e r i o r m o d e e s t i m a t i o n w i th a n o r m a lp r i o r ( e . g . We s t 1 9 8 5 , ( 4 . 4 ) ) . T h e c o n d i t i o n a l i n f o r m a t i o n

G t ( ] 3 t ) = E ( G t ( f l t ) [ f l t , Y ~ - O

s e e m s a r e a s o n a b l e a p p r o x i m a t i o n w h i c h i s i n g e n e r a l e a s i e r t o e v a l u a t e a n d h a st h e a d v a n t a g e o f b e i n g a l w a y s p o s i t i v e s e m i d e f i n i t e . I n o u r s e t t i n g , w e h a v e

G , q~ ,) = W , Z , W ', = H , Z ? ' H ; , (2.17)

q

G t ( ~ t ) = @ ( f i t ) + ~ V t j ( Y t j - P t j ) (2.18)j = l

T h e a l g o r i t h m s w h i c h f o l lo w c a n a l w a y s b e a p p l i e d w i th t h e c o n d i t i o n a l i n f o r -m a t i o n m a t r i x . P r o v i d e d c e r t a i n p o s i t iv e d e f i n it e n e s s c o n d i t io n s h o l d , t h e y c a na l s o b e a p p l i e d w i th t h e r a n d o m i n f o r m a t i o n m a t r i x . T h e r e f o r e th e y a r e w r i t te nd o w n w i t h a n i n f o r m a t i o n m a t r i x R t w h i c h m a y b e G t o r G t- F o r a d y n a m i c g e n -e r a l iz e d l i n e a r m o d e l ( 2 .1 0) w i t h a n a t u r a l r e s p o n s e f u n c t i o n ( o r l in k f u n c t i o n i nt h e m o r e s t a n d a r d t e r m i n o l o g y ) , w e h a v e a l in e a r m o d e lO t = Z ~ f l tf o r t h e n a t u r a l

p a r a m e t e r , l e a d i n g t o

r t = Z t (Yt - P t) , (2.19 )

R t = G t = G t = Z t 2 7 t Z ~ (2.20)

I n p a r t i c u l a r , b o t h i n f o r m a t i o n m a t r i c e s c o i n c i d e .


10/24

46 L. Fahrmeir and H. Kauf mann

3 E x t e n d e d K a l m a n F i l t e r i n g a n d S m o o t h i n g

3 .1 F i l t e r i ng

F i l te r in g a n d s m o o t h i n g a l g o r i t h m s p r o v i d e e s ti m a t es o f p a r a m e t e r v e c to r s u si n gthe obse rva t i ons u p t o a c e r t a i n t i me p o in t , t oge th e r w i th ( ap p rox ima te ) con d i -t i on a l e r r o r c ov a r i a nce m a t r ic e s . L e t ~ l t de n o t e t he e s t im a t e o f fls b a sed on t h eob se rv a t i on o f y * = (y ~ . . . . Y 't), a nd 2Ysl t h e c o r r e s p ond ing ap p rox im a te c ond i -t i ona l e r r o r c ova r i a nce m a t r i x , s , t = 0 , 1 , . . .

Th e ex ten ded K alm an f i lt e r g ives es t imates /~llt-1 , i f t -1 (p red ic t ion s tep)

resp. ~tlt , i f ' t i t(cor rec t ion s tep) , p roceed ing recurs ive ly. The fo l lowing f i l t e r inga l g o r i t h m is a n a l o g o u s t o t h e s t a n d a r d o n e f o r c o n d i t io n a l l y G a u s s i a n o b s e rv a -t ions (e .g . Sage and Melsa 1971 , Sec t ion 9 .3 ; Anderson and Moore 1979 , ch . 8 ) .I t c an be j u s t i f i ed a s a n a pp rox i ma t e pos t e r i o r m ode f i l t e r by ex t e nd i ng t hearg um en ts in Sage an d M elsa (1971) to e xp one n t ia l f ami l i es , see Fa hrm ei r (1988).

E x t e n d e d K a l m a n f i l te r1 . I n i t i a l i z a t i on

/ ~ O l O = a o , i f O l o = Q o

F or t = 1 , 2 , . . . :

2 . P r e d i c t i o n s t e p

/~tlt-1 = T , / ~ , - l l , - 1 ,

~ t l t _ 1 =T t i f t _ l l t _ 1T; + Q t ,

3 . C o r r e c t i o n s t e p

~ t lt = ~ t lt -~ + K t ( Y ~ - a t ) ,

.~t l t ' -( I - K t H t ) Z , t l t_ l

where

K t = i f t l t - l I - I t [H ' t i f tl t - i H t + Z t ] - 1


11/24


i s t h e K a l m a n g a i n , a n d P t ,Z t , H t are evaluated at J~t l t -1.B y an a p p l i c a ti o n o f t h e m a t r i x in v e r s io n le m m a ( e .g . A n d e r s o n a n d M o o r e

1979, p. 138) to the K a lm an ga in , the co r rec t ion s t ep can be g iven in a d i ff e ren tfo rm: I t i s eas i ly ve r i f i ed tha t

~ t l , = ( X t l ) - ,+ H t Z / 1 H ; ) - '= ( ~ d ~ - , + G t ) - '

a n d

K t ( Y t - / l t ) = Z t h t H t Z ? 1 ( Y t - P t ) = Z t l t r t

A s li g h tl y d i f f e r en t f o r m is o b t a i n e d i f t h e c o n d i t i o n a l i n f o r m a t i o nG t i s replacedb y th e r a n d o m i n f o r m a t i o n G t . T h is c a n b e d o n e p r o v i d e d t h e m a t ri c es~ _ 1 + O r, t = 1, 2 . . . . . o c c u r r i n g in th e m o d i f i e d c o r r e c t i o n s te p ar e p o s it iv edef in i t e . Th i s co nd i t ion m akes sense , s ince the inver se o f 27 t~]_1 + G t f o r m s ac o v a r i a n c e m a t r i x e s t im a t e . A c c o r d i n g t o th e r e m a r k s a t t h e e n d o f 2 .4 , t h e c o r r e c -t i o n s t e p i s w r i t t e n d o w n w i t h t h e g e n e r a lR t w h i c h m a y b eG t o r G t .

3 ' C o r r e c t i o n s t e p

Z t l t = ( Z ? ~ J - 1 + R t ) - 1 ,

f l t l t = fl tE t - 1 + ~ t l t r t ,

w i t h r t a n d R t ev alu ated a t /~ t t t 1.A s w i ll b e s e e n in S e c t i o n 5 , t h e f i lt e r is c lo s e l y c o n n e c t e d t o f i n d i n g p o s t e r i o r

m o d e e s t i m a t e s b y G a u s s - N e w t o n r e s p . F i s h e r s c o r i n g it e r a ti o n s . A f i rs t i n d i c a -t i o n o f t h i s r e la t i o n s h i p i s p r o v i d e d b y t h e f o r m 3 ' o f t h e c o r r e c t i o n s te p : t h i n k i n go f c o v a r i a n c e m a t r i c e s a n d i n f o r m a t i o n m a t r i c e s b e i n g i n v e r s e s t o e a c h o t h e r ,Z ~ ] _ I is t h e ( e s ti m a t e d ) in f o r m a t i o n o nf i t g iven y*_ 1- T he m at r ixR t i s the in fo r-m a t i o n o n f i t c o n t r i b u t e d b y t h e n ew o b s e r v a t i o nY t , a n d t h e s u m Z ~ _ 1 + R tist h e i n f o r m a t i o n o n f i t g i v e n Y T. I n v e r t in g t h is m a t r i x , w e g et t h e c o v a r i a n c em a t r i x " ~ tl t- T h e u p d a t e s t ep ]~ tlt = . . . h a s j u s t t h e f o r m o f a s i n g le G a u s s -

N e w t o n r e s p . F i s h e r s c o r i n g i t e r a t i o n .

3 .2 S m o o t h i n g

I f an ex te nd ed Ka lm an f i l te r is run un t i l t ime t , e s tima tes /~o l 0 . . . . /~dt a re ava i l-a b l e . T h e f u l l i n f o r m a t i o n i s o n l y u s e d i n e s t i m a t i n gf i t . I m p r o v e d e st im a t e s f o r


12/24

4 8 L . F a h r m e ir a n d H . K a u f m a n n

fl0 . . . . f l t - i a r e o b t a i n e d w i t h th e f o ll o w i n g b a c k w a r d r e cu r si v e s m o o t h e r ( S a g eand Melsa 1971 , ch . 9 ) .

S m o o t h e r1F o r s = t, . . . , 1 :

~ s - 1 l , - ~ s - 1 l s - i =B s ~ s l t - L L s - , ),

Z s - 1 [ - ~ s - 1 1 s - 1 =B ~ ( ~ s l t - - ~ s b s - , ) B s,

w h e r e

T, - Is = z ~ , s _ l l s _ l s ~ ., s i s _ l .

T h i s s m o o t h e r i s e s s e n t i a l l y t h e s a m e a s t h e f i x e d i n t e r v a l s m o o t h e r i n t h e l i n e a rG a u s s i a n c a se , e x c e p t t h a t ~ s - 11s - 1 ,Z s l s - I a n d t h e r e f o r e B s, s = 1 . . . . t, d e p e n do n ]~1 t 0 . . . . ]~ t_ 11 /_ 2 .

T h e f i x e d i n t e rv a l s m o o t h e r 1 p r o v i d e s i m p r o v e d e s t i m a t e s J ~lt f o rf l s , s< t ,o n l y a f t e r a b l o c k o f t c o n s e c u t i v e f i lt e r st ep s . F o r re c u r s iv e G a u s s - N e w t o n a n dF i s h e r s c o r i n g p o s t e r i o r m o d e f i lt e r in g a n d s m o o t h i n g i n S e c t i o n 4, it is o f in t e r e s tt o s m o o t h b a c k w a r d s a f t e r e a c h f il te r s te p . To d o s o, o n e n e e d s a s m o o t h e r b a s e do n

J ~ O I t - I . . . . . J~ t l t - I , J~ t l , (3.1)

i n s t e a d o f

~010,/~110 .... f l t l t - , , f l t l , (3.2)

W r it in g d o w n s m o o t h e r 1 f ir st f o r o b s e r v a t io n s u n t i l t i m e t a n d t h e n f o r o b s e r v a -t io n s u n t i l t i m e t - 1 a n d f o r m i n g t h e d if f er e n ce , w e f i n d

L - l l , - B s - l l , - , = B s s l ,- L i ,- ,) (3.3)

A n a l o g o u s l y, w e g e t t h e c o r r e s p o n d i n g e q u a t i o n f o r t h e c o v a r i a n c e m a t r i c e s .H o w e v e r, i n e v a l u a t in g ( 3 .3 ), t h e c o v a r i a n c e m a t r i c e s d e t e r m i n i n g B 1 . . . . B tsho u ld a l so d ep en d on /~ l I t -I . . . . . /~ t- I I t - I ins t ead o f/~110 . . . . . /~ ,-1 I t-2 . Th i s r e -q u i r e m e n t c a n b e f u l f i l l e d b y s e p a r a t i n g t h e c o v a r i a n c e r e c u r s i o n f r o m t h e f i l t e r.T h e s a m e t h i n g w i l l b e n e e d e d l a t e r w i t h f u r t h e r s e q u e n c e s , w h e n c e t h e f o l l o w i n g


13/24

O n K a l m a n F i l te r in g 4 9

s c h e m e i s d e v i s ed f o r a n a r b i t r a r y s e q u e n c ei l l , f 2 . . . . N o t e t h a t t h e s c h e m e isf o r w a r d r e cu r s iv e . F o r th e c h o i c eR t ( / 3 t ) = G t ( fl t) , i t m a k e s s e n s e p r o v i d e d

S t / ~ - i + G t ( f l t ) i s p o s i t i v e d e f i n i t e , t = 1 , 2 . . . .

C o v a r i a n c e r e c u r s i o nL e t fl~ ,fl2 . . . b e a s e q u e n c e o f p - v e c t o r s . S t a r t i n g w i t h Z 01 o = Q o , d e f i n e

f o r t = 1 , 2 . . . . :

S t i r _ 1 -~- Z t Z t _ l l t _ !T~ + Q , ,

B t = Z t - l [ t - l r ; S ~ : - i ,

S t l t = [ t ~ - I + R t ( f l t ) ] - 1

( 3 . 4 )

(3 .5 )

(3 .6 )

A c t u a l l y, S t l t _ 1a n d B t ( S t l e ) d e p e n d o n l y o n f ll . . . . . f l t - l( f l l . . . . . f it ). I f t h es e q u e n c e f ll ,f l2 . . . . r e f e r r e d t o in th e c o v a r i a n c e r e c u r s i o n is t o b e s tr e s s e d , w ew r i t e

S t i r - 1 = S t i r - 1~ /~ 1 . . . . . f i t - l ) ,

B t = B t ( fl l . . . . .f i t - l ) , ( 3 . 7 )

S t It = "~tlt(t~ l . . . . . f t )

I f t h e e x t e n d e d K a l m a n f i l te r o f 3 .1 i s r u n u n t i l t i m e t , i t u s e s t h e c o v a r i a n c er e c u r s i o n b a s e d o n / ~ 11 0. . . . ~ t l t - 1 . To b e c o n s i s t e n t w i t h t h e i n t e n d e d s m o o t h e r ,i t s h o u l d i n s t e a d b e b a s e d o n f ll l t - 1 . . . . . ] ~tl t- 1. T h i s l e a d s t o t h e f o l l o w i n g i n -t e g r a t e d a l g o r i t h m f o r fi l t e ri n g w i t h s m o o t h i n g b a c k a f t e r e a c h f i l te r s te p .

F i lt er a n d S m o o t h e r 2L I n i t i a l i z a t i o n

/)olo = ao , Z o l o = Q o ,

F o r t = 1 , 2 , . . . :

2 . P r e d i c t i o n s t e p :

L L , - 1 = T , L - 1 1 , - 1 ,


14/24

5 0 L . F a h r m e i r a n d H . K a u f m a n n

3 . C o v a r i a n c e r e c u r s i o n , f o r s = 1 . . . . t :

f ' s l s - I = Ts . S s - l l s - I T ' ~ + Q s ,

B s = Z ~ s _ l l s _ 1 , - - 1s 2 7 s l s - ~ ,

~ s l s - - 1 R ^ - 1= [ Z s l s _ 1 q - s ( f l s l t _ l ) }

4 . C o r r e c t i o n s t e p :

~ t l t = ~ t l t - ~ + z ~ t l t r t ( ~ t l t - O ,

5 . S m o o t h e r 2 , f o r s = t. . . . . 1 :

L - 1 1 , - L - , I , - 1 =B, (L t , -L I , - , ) ,. ~ s _ 1 1 , - ,_ 1 1 , _ ~ = B s ( . ~ s l , - z . i , _ l ) - B ' . ,

S t e p t o f th i s c o m b i n e d a l g o r i t h m c o n s i s ts o f a f o r w a r d r e c u r s i o n , i n c l u d in gt h e f i lt e r s te p s , a n d t h e b a c k w a r d r e c u r s iv e s m o o t h e r. I n t e r m s o f ( 3 .7 ), i n s te pt w e h a v e

~-~s l s - I = .~ ' s l s - l (~ l l t -1. . . . . / ~ s - l l t - 1 ) ,

B S = B s ( ~ l l t _ 1 . . . . L - l i t - l ) ,

-~ sls = z s l ~ 1 1 , - 1 . . . . . L ~ , - 1 ) ,

s = 1 . . . . t. S in c e e r r o r c o v a r i a n c e m a t r i c e s a r e r e c o m p u t e d b e f o r e th e t - t h c o r -r e c t i o n s t ep , e s t im a t e s a r e in g e n e r a l d i f f e r e n t f r o m t h o s e o f t h e p r e v i o u s e x t e n d e dK a l m a n f i l t e r a n d s m o o t h e r .

4 P o s t e r io r M o d e E s t i m a t i o n

4 .1 S c o r e F u n c t i o n a n d I n f o r m a t i o n M a t r i x

L e t u s n o w l o o k f r o m a d i ff e r e n t a n g l e a t t h e p r o b l e m o f e s t i m a t i n g / /* =(fl~ . . . . . fl~ ). G i v e n y * , t h e w h o l e p a r a m e t e r v e c t o r p * m a y b e e s t i m a t e d b y m a x -i m i z i n g i ts p o s t e r i o r d e n s it y. E q u i v a l e n tl y, o n e c a n m a x i m i z e t h e j o i n t d e n s i t y


15/24


p ( . v , 1 , 6 '? ) p f ? )

o r , t a k i n g l o g a r i t h m s ,

l * ( f * ) + a ~ ( f * ) ( 4 . 1 )

w i th t h e l og li ke l i ho o d l ~ ' ( f * ) = l o g p ( y ~ l f l * ) a n d t h e lo g p r i o r a * ( f * ) =l o g p ( f * ) . I n m a x i m i z i n g (4 .1 ), th e s co r e f u n c t i o n

uP(f,) = ~ (t* (f*)+a ? (f*)) (4.2)

a n d t he i n f o r m a t i o n m a t r i x * *t (flit),s ay, a r e o f i n te r e st . A c c o r d i n g t o c o m m o np r a c ti c e in p o s te r i o r m o d e e s t i m a t io n , o n e w o u ld u s e t h e r a n d o m i n f o r m a t i o n

_ 0 2U ~ ' 0 " ) - O /~ ? O f f*- - - - ~ ( 1 7 ( ,8 * ) + a ~ ' ( ,8 * ) ) . ( 4 . 3 )

A n a l t e rn a t i v e is t o r e p l a c e (4 .3 ) b y s o m e k i n d o f c o n d i t i o n a l i n f o r m a t i o n .D u e t o t h e r e c u rs iv e n a t u r e o f d y n a m i c e x p o n e n t i a l f a m i l y r e g r e s si o n m o d e l s ,

u * a n d U * h a v e a s p e c i a l s t r u c t u r e . F r o m t h e a s s u m p t i o n s i n S e c t i o n 2 , w e g e t

t

Z * ( f ~ ) = ~ I s ( f s ) (4.4)I

f o r t h e lo g li k e l ih o o d b y su c c e ss iv e c o n d i t i o n i n g . U p t o a s u m m a n d i n d e p e n d e n to f f l * , t h e l o g p r i o r is

1

a * ( f ? ) = - y ( f o - a o) ' Q o ~0 o - a o )

- - ~E

( f s - T s f l s - 1 ) ' s 1 ( f s - T s f l s - 1 ) I

(4.5)

T h e s c o r e f u n c t i o n c a n b e p a r t i t i o n e d a s u ~ = ( u~ . . . . , u 't )' , w h e r e t h e s u b v e c t o ru s g i ve s t h e d e r i v a t i v e o f t h e l o g p o s t e r i o r w i t h r e s p e c t t o fls, s = 0 . . . . . t . I no r d e r t o a v o i d s p e c i a l f o r m u l a s f o r t h e b o u n d a r y q u a n t i t ie s u 0, ut, it is c o n v e -n i e n t t o c o m p l e m e n t t h e s c o re f u n c t i o n c o n t r i b u t i o n s r 1 . . . .r t g iven in (2 .16 )an d (2 .19 ) by r 0 = 0 , an d to s e tTt+ 1 = O , T ' t+ lC t+ l = 0 . D e f i n i n g a d d i t i o n a l l y


16/24

52 L . Fah rme i r and H . Kaufm ann

Co = Oo lC a0 -a 0) ,

cs = 0 f ' (#s-T~Zs-,) ,s = l , . . . , t ,

( 4 . 6 )

w e h a v e

u s = r s - c s + T s + l C s + 1 , S = 0 . . . . . t . ( 4 . 7 )

T h e s u b v e c t o rUs o f t h e s c o re f u n c t i o n d e p e n d s o n l y o n P s - l , f ls a n d/ ~ s + l ,s = 1 . . . . t - 1, t h e b o u n d a r y v e c t o r su o, u to n l y o n f l0 , f l, r e s p . f i t - 1, f it . T h e i n -f o r m a t i o n m a t r i x th e r e f o r e h as t r i d i a g o n a l b l o c k s t r u c t u re :

U * =

"U~

U~

U01 0

U~ . .

U t - l , t

O . . . .

U ' t - 1, U .

( 4 . 8 )

w i t h

U ss = R s + Q s I + T ' s + lO s + l lTs + l , s = 0 . . . . . t , ( 4 . 9 )

U s _ l , s = - T ' s Q s 1 , s = 1 , . . . , t , ( 4 . 1 0 )

w h e r e R 0 = 0 ,T t + , = 0 a n d R , . . . . . R ta re t h e ( r a n d o m o r c o n d i t io n a l ) i n f o r m a -t io n m a t r i x c o n t r i b u t i o n s g i v e n i n 2 .4 .

4 .2 F a c t o r i z a t i o n a n d I n v e r s i o n o f t h e I n f o r m a t i o n M a t r i x

A n y p o s i ti v e d e f i n i t e b l o c k - t r id i a g o n a l m a t r i x c a n u n i q u e l y b e f a c to r i z ed i n to

I

- B ~ I 0" . .

- B ~ ,

o "" - B ~ "I

- D o

D 10

D t

- I - B 1 0

I - B 2 ." .

0 " ' . - B tI

( 4 . 11 )


17/24

O n K a l m a n F i lt e ri n g 5 3

w h e r e D O . . . . D t a r e p o s i t iv e d e f in i te m a t r ic e s . C o n v e r s e l y , i f a m a t r i x p o s s e s s e ss u c h a f a c t o r i z a t i o n w i t hD o . . . . D t p o s i t i v e d e f i n i t e , t h e n i t is p o s i t i v e d e f i n i t ea n d b l o c k - t r i d i a g o n a l .

T h e f ir s t p a r t o f t h e f o l lo w i n g p r o p o s i t i o n s p e c i fi e s t h e f a c t o r s f o r t h e i n f o r -m a t i o n m a t r i x U * . I t r e st s o n th e c o v a r i a n c e r e c u r s io n , w h i c h i n c l u d e s e x p r e s -s i o n s f o r t h e m a t r i c e sB 1 . . . . . B t, t o g e t h e r w i t h s i m i l a r e x p r e s s i o n s f o rD o . . . . D , T h e s e c o n d p a r t g i v e s t h e i n v e r s e s D o 1 . . . , D t ~ F i n a l l y, i n p a r t ( i ii )w e o b t a i n t h e i n v er se o f U *, w h i c h y i e l d s a p p r o x i m a t e e r r o r c o v a r i a n c e m a t r i c e s .

P r o p o s i t i o n 1 .(i) Let 27010,X s l s _ t , X s l s , s = 1 . . . . t ,b e d e f i n e d b y t h e c o v a r i a n c e

r e c u r s i o n , a s w e l l a s

T, x - -1B s = X s - l l s - I s ~ s l s - 1 , s = l . . . . . t . ( 4.1 2 )

- 1A s s u m e t h a t 2 ~ s l s _ l + R s, s = 1 . . . , t , is p o s i t i v e d e f in i te . T h e n U * is a l s op o s i t i v e d e f i n i te , a n d t h e m a t r i c e s i n i ts f a c t o r i z a t i o n a r e g iv e n b y ( 4. 12 ) a n d

Ds = Xs-i ls+ Ts + l Q s + l lTs+ 1 , s = 0 . . . . . t . ( 4 . 1 3 )

( i i ) T h e i n v e r s e s a r e

D s 1 = Z s l s - B s + t 2 7 s + I Is B's +l,

D t t = Z ' t l t .

s = 0 , . . . , t - 1 ,( 4 . 14 )

( ii i) T h e ( r , s ) - b l o c k o f t h e i n v e r se o f U ~ is

t

. .- ' ( 4 . 1 5 )r s = ~ B~+~ . . ' B j D f t B ) ' . . B ~ + ~ ,

0


18/24

54 L. Fahrmeir and H. Kaufma rm

P r o o f " (i) Due t o t he a s sum pt ion s i n S ec t i on 2.1 , X ~= Q o 1 i s posi t ive def in i teZ - 1T h e s a m e h o l d s t r u e f o rX s-i~s = s l s _ l + R s, s = 1 . . . . tby hypothes i s . S ince

pos i t i ve s emide f i n i t e ma t r i c e s a r e added , i t f o l l ows t ha tD O . . . . D td e f i n e d b y(4 .13) a re pos i t ive de f in i t e Accord ing to the remarks p receed ing Propos i t ion 1 ,i t r emains to be shown tha t U* possesses a fac to r iza t ion (4 .11) , wi th the sub-m atr ic es of the fa c to rs g iven by (4 .12), (4 .13). F ro m (4.12), (4.13) an d th ecova r i ance r ecu r s i on , we g e t

D s _ l B s , - 1 , - I= T s X s l s - I + T s Q s I Ts 'S s -I I s -1 Ts ' S s ls - 1

T , ~ Z - IT ' ~ Q s l ( O s + Ts X s - t l s - I s ) s is -1 = T ' s Q ~ 1

(4.16)

and s im i l a r l y

D s + B s D s _ I B s R s + Q s 1 , -1+ T s + l Q s + l Ts+ l , (4.17)

s = 1 . . . . t. M u l t i p ly i n g o u t ( 4 .11 ) y ie ld s t h e m a t r i x

D O - D o B 1 0 ]

- B ~ D o B ~ D o B I + D 1 " '.

.. " .. - D t _ l B t

0 - B ~ D t _ 1 B ' t D t _ I B t + D t ]

(4.18)

In se r t i ng (4 . 6 ) a n d ( 4 . 7 ) i n t o ( 4 . 18 ) and c omp ar ing w i th U*

(4 .8 ) - (4 .10) , the des i red resu l t i s ob ta ined .( i i ) Th i s c a n be i n f e r r e d by i n s e r t i ng t he fo rmu la f o rB s + l in to

g iven in

Z~I B 1Z~ l i B '- - S + + S S + 1

an d i n vo k in g t he m a t r i x i nv e r si on l em m a (e .g . A nd e r s o n a nd M oo re 1979, p . 1 38).( i i i ) By m u l t i p ly ing o u t , i t c an be ve r i f i ed t ha t

- I - B l

I

0

-B 20

- B t

I

- 1- I B 1 ... B 1 " . . . ' B t -

I B 2

0 B t

I


19/24

On Katman Filtering 55

F r o m t h i s f o r m u l a , w e g e t ( 4 . 1 5 ) b y m u l t i p l y i n g t h e i n v e r s e s o f t h e m a t r i c e s i n(4 .11) in con verse d i rec t ion . [ ]

4 .3 Gaus s -Newt o n a nd F i s he r S c o r i ng I t e r a t i o ns

A v e c t o r m a x i m i z i n g t h e p o s t e r i o r d e n s i t yp ( p ~ [ y ~ ' ) c a n b e f o u n d b y G a u s s -N e w t o n o r F i s h e r s c o r i n g it e r a ti o n s . I f f l* d e n o t e s t h e c u r r e n t v e c to r, t h e n t h enex t i t e ra te is f l* + 6 " , wh ere 6* = (5~ . . . . , 5 ' t ) ' so lves

(4 .19)

T h i s p r o c e s s i s r e p e a t e d u n t i l c o n v e rg e n c e . I f U ~ i s t h e r a n d o m i n f o r m a t i o n(Rs = Gs) , t h e n ( 4 .1 9 ) is a G a u s s - N e w t o n , o t h e r w i s e(R s = Gs) a F i she r scor ingi t e ra t ion .

D u e t o t h e d e c o m p o s i t i o n ( 4 .11 ), e q u a t i o n s ( 4. 19 ) c a n b e s o l v e d b y f o r w a r d -b a c k w a r d r e c u r s io n , th u s a v o i d in g in v e r si o n o f U * ~ * ) . F i r st o n e s o lv e s t h e

sys tem

i lB ] I 0." , . . ~ -

-B~ t t

f o r t h e a u x i l ia r y v e c t o r e * = (e~ . . . . .e ' ) b y f o r w a r d r e c u r s i o n , a n d t h e n

io 0 i : l L lo ; : ~ ' 1 , =

b y b a c k w a r d r ec u rs io n . I n c o r p o r a t i n g c o m p u t a t i o n o f B l . . . . .B t a n d D O . . . ,D t ,we ge t the fo l lowing .

Ga us s -N e w to n ( F i she r s c o r ing ) s t ep1. Ini t ial izat ion

so = u o , ~ o t o = Q o ,


20/24

56 L. Fahrmeir and H. Kaufm ann

2 . F o r w a r d r e c u r s io n , f o r s = 1 . . . . . t :

c o m p u t e X s l s - ~ , B s , X s l sb y t h e c o v a r i a n c e r e c u r s i o n ,u s by (4.6) , (4 .7) ,

e s = u s + B ' s C s _ 1 ,

3 . F i l t e r c o r r e c t i o n

Ot = .S t l t e t ,

4 . S m o o t h e r c o r r e c t i o n s , f o r s = t . . . . . 1:

D s - ~ = X s - i t s - 1 - B s . S s l s - 1 B's ,

O s - 1 = D s - l i e s - l + B s O s

O f c o u r s e , s e ve r al n u m e r i c a l v a r i a n t s a r e p o ss i b le . I n t h e s m o o t h e r c o r r e c -t io n s , f o r in s t an c e , i n s te a d o f c o m p u t i n gD s - 1 b y m e a n s o f ( 4 .1 4) , o n e c a n f ir s t

c o m p u t e D s _ 1b y (4 .1 3 ) a n d t h e n i n v e rt . A l t h o u g h t h is is m o r e t i m e - c o n s u m i n g ,i t s h o u l d h a v e th e a d v a n t a g e o f g r e a t e r n u m e r i c a l s ta b il it y, si nc e c a n c e l l a t i o n e f -f e c t s a r e a v o i d e d .

A f t e r o n e o r m o r e i te r a ti o n s, o n e c a n a p p l y L e m m a 1 (iii) t o g e t a p p r o x i m a t ee r r o r c o v a r i a n c e m a t r i c e s . To gi ve a n e x a m p l e , le t u s l o o k a t t h e d i a g o n a l

A s s = Z s t t , s = 0 . . . . . t .

F r o m ( 4.1 5 ), w e o b t a i n t h e f o r m u l a

Z s _ l L t = D s ] l + B s X s l t B ' s , s = 1 . . . . t .

T h i s c a n b e a p p l i e d b a c k w a r d r e c u r s i v e l y, s t a r t i n g w i t h Z ' t l t .

(4 .20)

4 .4 A S i n g l e G a u s s - N e w t o n ( F is h e r S c o r i n g ) S t e p

I n th e d i s c u s s i o n o f G a u s s - N e w t o n ( F i s h e r s c o r i n g ) it e ra t io n s , t i m e t h a s b e e nf ix e d , i n c o n t r a s t t o t h e a l g o r i t h m s o f S e c t i o n 3 . I f G a u s s - N e w t o n ( F i s h e r s c o r i n g )s t ep s a r e t o b e a p p l i e d s e q u e n t i a l ly f o r t = 1 ,2 . . . . . t h e n ]~ 0tt-1 . . . . .~ t - l l t - it o g e t h e r w i t h t h e f o r e c a s t / ~ t l t - 1 =T t ~ t - l l t - 1 is a r e a s o n a b l e s t a r ti n g v a l u e a tt i m e t , w h e n t h e n e w o b s e r v a t i o nY t b e c o m e s a v a il ab l e. I n m a n y c a s e s , Y t s h o u l d


21/24

O n K a l m a n F i l te r i n g 5 7

n o t c h a n g e e s t i m a t e s t o o d r a s t i c a l l y, w i t h t h e i m p l i c a t i o n t h a t a s i n g l e G a u s sN e w t o n ( F i s h e r s c o r in g ) i t e r a ti o n s u f f ic e s . T h i s l ea d s t o t h e f o l lo w i n g a l g o r i t h m .

F i l t e r a n d s m o o t h e r 3L I n i t i a l i z a t i o n

/ ~ o l 0= a 0 , Z 0 1 0= Q 0 ,

F o r t = 1 ,2 . . . . :

E 0 = H 0 ,

2 . P r e d i c t i o n s t e p :

f l t l t _ 1= T r O t _ l i t _ 1 ,

3 . F o r w a r d r e c u r s i o n , f o r s = 1 . . . . . t :

c o m p u t e Z~sls_ 1 , B s , Z ' s i sby the cov ar ianc e recurs ion , us by (4 .6) , (4 .7) basedo n

/ ~ l I t - 1 . . . . B t l t - ! ,

c s = U s + B ' s e s _ 1 .

4 . F i l t e r c o r r e c t i o n

B , It = B t I , - 1 + L I , e , ,

5 . S m o o t h e r c o r r e c t i o n s , f o r s = t . . . . .1:

D s ~ l = Z~s_ l l s_ , - B s~ , s l s - , B ' s ,

f l s - l l t - f l s - , I t - ,= B s ( f f s l , - f l s l t - O + D s -t , e s - ,


22/24


5 Relationships Between Algorithms

T h e f o l lo w i n g p r o p o s i t i o n g iv e s t h e c o n n e c t i o n b e t w e e n f il te r a n d s m o o t h e r 2 a n3 , w h i c h l o o k r a t h e r c lo s e. B r i e fl y, t h e y a r e r e f e r r e d a s t o a l g o r i t h m s 2 a n d 3

P r o p o s i t i o n 2 .A s s u m e t h a t s te p t o f a l g o r i t h m s 2 a n d 3 is r u n w i t h t h e s a m e i n p u tv e c t o r s ]~0[t- 1. . . . ]~ t- 1 Lt- 1" T h e n t h e f o l l o w i n g a s s e r t i o n s a r e e q u i v a l e n t :

( i) a l g o r i t h m s 2 a n d 3 p r o v i d e t h e s a m e e s t i m a t e s /~ 0 1 t . . . . . ]~ tlt,

( ii ) i n a l g o r i t h m 3 , i t h o l d s t h a t ~ 0 = . . . = el -1 = 0 ,( ii i) t h e i n p u t v e c t o r ( fl0 1 t-I . . . . . / ~ t - l / t - 0 is a s t a t i o n a r y p o i n t o f

p ( / 3 * l l y * O .

P r o o f . " P r e d i c t i o n s te p s a re o b v i o u s l y th e s a m e , i n d u c i n g t h a t t h e m a t r i c e s c o mp u t e d i n t h e c o v a r i a n c e r e c u r s i o n a r e al so e q u a l f o r b o t h a l g o r i th m s , s in c e t h ea re b a s e d o n t h e s a m e s e q u e n c e . T h u s , w e n e e d o n l y c o n s i d e r t h e f i lt e r a n ds m o o t h e r c o r r e c ti o n s . N o t e f u r t h e r t h a t t h e p r e d i c t i o n s te p i m p l i es

C = O t l ( ~ / i t _ l - Z t ~ t _ l l t _ l ) : 0 ( 5 . 1 )

i n d u c i n g ( c o m p a r e ( 4 . 6 ) , ( 4 . 7 ) )

u t = r t ( t ~ t l t - 1 ) (5 .2 )

I f (i ) h o l d s , t h e n a c o m p a r i s o n o f th e s m o o t h e r f o r m u l a s y i e l d s (i i) . If (i ih o l d s g o o d , t h e n w e h a v e

e t = u t + B ~ g t _ 1 = u t = r t ( ~ t l t _ l )

f r o m (5 .2 ), in d u c i n g t h a t t h e f i lt e r c o r r e c t i o n s a re eq u a l . S i n c e t h e s m o o t h e r c or e c t i o n s a r e o b v i o u s l y e q u a l i f e 0 = . . . =e t _ l = 0 , ( i i ) imp l i e s ( i ) .

To s e e t h a t ( ii ) a n d ( ii i) a r e e q u i v a l e n t , l e t v * _ ~ = (v ~ . . . . v ~ _ l )' d e n o t e t hs c o r e f u n c t i o n o f l o g p ( f l * _ ~ [ y * _ 0 . B y d e f i n i t i o n , ( ii i) is e q u i v a l e n t t o

vLl~?-~) = 0 . ( 5 . 3 )

F r o m ( 4 .6 ) , ( 4 .7 ) , w e g e t


23/24

On Ka lman Filtering 59

U s = V s , s - O . . . . , t - 2 ,

u t - 1 = v t - 1 + T ' t c t

Fo r ev a lu a t io n a t ( f l ol t-1 . . . . . / ~ tl t-1 ), we even hav e

u s = v s , s = 0 . . . . . t - I , (5 .4)

d u e t o ( 5 .1 ) . F r o m e 0 = u0 a n d t h e f o r w a r d r e c u r s i o n , i t c a n e a s i l y b e i n f e r r e d

th a t e0 . . . . . e t ~ = 0 is equ iva l en t t o u0 = . =u t _ ~ = 0 . E qu a t io ns (5 .3 ) and(5 .4 ) y i e ld t he des i r ed equ iva l ence . [ ]

I n S e c t i o n 3 a n d 4 , w e h a v e d i s c u s s e d f o u r a l g o r i t h m s :l . e x t e n d e d K a l m a n f il te r in g a n d s m o o t h i n g ,2 . f i lt e r in g c o m b i n e d w i t h s m o o t h i n g a f t e r e a c h fi lt e r s te p ,3 . f i lt e r in g a n d s m o o t h i n g b y a si n gl e G a u s s - N e w t o n ( F i s h e r sc o r in g ) it e r a t io n

a t s t ep t ,4 . p o s t e r i o r m o d e e s t i m a t i o n b y G a u s s - N e w t o n ( F i s h er s co r i n g ) i te r a t io n s .

R e g a r d i n g t h e r e l a t i o n s h i p b e t w e e n 3 a n d 4 , i t s e e m s p l a u s i b l e t h a t a s i n g l eG a u s s - N e w t o n ( F i s h e r s c o r i n g ) i t e r a t i o n a t s t e p t o f t e n p r o v i d e s a g o o d a p p r o x -i m a t i o n : f o r s fi x e d , t h e s c o r e f u n c t i o n s u b v e c t o r u s a p p e a r s i n a ll s te p st > _ s , a n di t s h o u l d t h e r e f o r e b e c o m e c l o s e r a n d c l o s e r t o z e r o .

A l g o r i t h m 2 m a k e s a f u r t h e r a p p r o x i m a t i o n in t h a t t h e q u a n t i ti e s e 0 . . . .e t _a r e n e g l e c t e d i n s t e p t . I t m a y b e t h o u g h t o f a s a v a r i a n t o f f i l t e r i n g a n ds m o o t h i n g b y a si n g le G a u s s - N e w t o n ( F i s h e r s c o r i n g ) i te r a t io n , w h i c h r el ie s o nt h e i n d u c t i o n h y p o t h e s i s t h a tf ro I t - ~ . . . . f i t - 1 L - ta r e p o s t e r i o r m o d e e s t i m a t e s .I f th i s h o l d s t r u e , th e n i t p r o v i d e s t h e s a m e e s t i m a t e s a s a l g o r i t h m 3 , a c c o r d i n gt o P r o p o s i t i o n 2 . H o w e v e r , t h e i n d u c t i o n h y p o t h e s i s w il l u s u a l l y n o t h o l d e x a ct ly.S i n c e e 0 , . . . , e t _ ~ a r e n e g l e c t e d , t h e s c o r e f u n c t i o n s u b v e c t o ru s e n t e r s o n l y a ts t e p t = s , a n d t h e r e i s n o p o s s i b i l i t y f o r c o r r e c t i o n a t s u b s e q u e n t s t ep s .

T h e o r i g in a l a l g o r i t h m I s im p l i fi e s f u r t h e r b y s m o o t h i n g b a c k o n l y a f t e r tf i l t e r s teps .

To s u m m a r i z e , w e h a v e s h o w n t h a t t h e a l g o r i th m s f o r m a h i er a r ch y o f a p p r o x -i m a t i o n s t o p o s t e r io r m o d e e s t i m a t io n , t h e q u a l it y o f a p p r o x i m a t i o n b e i n g o p -p o s e d t o c o m p u t a t i o n a l e f f o r t . T h e k i n d o f a p p r o x i m a t i o n s h a s b e e n c l a r i f i e d t o

s o m e e x t e n t. O f c o u r se , i t w o u l d b e v e r y v a l u a b l e t o h a v e r e su l ts j u d g i n g t h e q u a l i -t y o f t h e v a r i o u s a p p r o x i m a t i o n s m o r e q u a n t i t a t i v e l y, a n d c h a r a c t e r i z i n g t h es i t u a t i o n s w h e r e t h e y c a n b e s u c c e s s f u l l y a p p l i e d . T h i s i s p r e s e n t l y o n l y k n o w ni n t h e li n e a r G a u s s i a n m o d e l , w h e r e a ll f o u r a l g o r i t h m s p r o v i d e t h e s a m ee s t i m a t e s , b u t s h o u l d b e a t o p i c f o r f u t u r e r e s e a r c h .

A c k n o w l e d g e m e n t .I thank a referee for his valuable com me nts, which helped to im prove he presenta-tion of the paper.


24/24


R e f e r e n c e s

A m e e n J R M , H a r r i s o n P J ( 19 85 ) N o r m a l d i s c o u n t B a y e s i a n m o d e l s . I n : B e r n a r d o JM , D e G r o o t M H ,L i n d l e y D V, S m i t h A F M ( e ds ) B a y e s ia n S t a ti s ti c s 2 : 2 7 1 - 2 9 4

A n d e r s o n B D O , M o o r e J B (1 97 9) O p t i m a l F i l te r in g . P r e n t i c e H a l l , E n g l e w o o d C l if f sB a k e r R J , T h o m p s o n R ( 1 98 1) C o m p o s i t e l in k f u n c t i o n s in g e n e ra l iz e d l in e a r m o d e l s . A p p l S t a t

3 0 : 1 2 5 - 1 3 1F a h r m e i r L ( 1 9 8 8 ) E x t e n d e d K a l m a n f i l t e r i n g f o r d y n a m i c g e n e r a l i z e d l i n e a r m o d e l s a n d s u r v i v a l

d a t a . R e g e n s b u rg e r B e it ra g e z u r S t a ti s ti k u n d O k o n o m e t r i e 1 0F a h r m e i r L , K a u f m a n n H ( 1 9 85 ) C o n s i s t e n c y a n d a s y m p t o t i c n o r m a l i t y o f t h e m a x i m u m l ik e l ih o o d

e s t i m a t o r i n g e n e r a li z ed l i n e a r m o d e l s . A n n S t a ti s t 1 3 : 3 4 2 - 3 6 8Fah rm ei r L , Ka ufm ann H (1987) Regress ion mo de ls fo r non s ta t io nar y ca tegor ica l t ime ser ies. J Time

S e r A n a l 8 : 1 4 7 - 1 6 0JOrgensen B (1983) M axim um l ike l ihood es t imat ion an d la rge sample in fe rence fo r genera li zed linear

a n d n o n l i n e a r r e g r es s io n m o d e ls . B i o m e t r i k a 7 0 : 1 9 - 2 8K auf m ann H (1987) Regress ion mo de ls fo r no ns ta t io na ry ca tegor ica l t ime ser ies: a sy mp to t ic es t ima-

t i o n t h e o r y. A n n S t a t i s t 1 5 : 7 9 - 9 8K i t a g a w a G ( 1 9 87 ) N o n - G a u s s i a n s t a te - s p a c e m o d e l l in g o f n o n s t a t i o n a r y t i m e s e ri es ( w i th c o m m e n t s ) .

J A S A 8 2 : 1 0 3 2 - 1 06 3Nelder JA , Wed derb urn RW M (1972) Gen era l i zed l inear mode ls . J Roy S ta t i s t Soc Ser A 135 :370 - 384S a g e A P, M e l s a J L ( 1 97 1) E s t i m a t i o n T h e o r y w i th A p p l i c a t io n s t o C o m m u n i c a t i o n a n d C o n t r o l .

M c G r a w H i l l , N e w Yo r kWe s t M ( 1 9 8 5 ) G e n e r a l i z e d l i n e a r m o d e l s : s c a l e p a r a m e t e r s , o u t l i e r a c c o m m o d a t i o n a n d p r i o r

d i s t r ib u t i o n s . I n : B e r n a r d o J M , D e G r o o t M H , L i n d l ey D V, S m i t h A F M ( e ds ) B a y e s ia n S t a ti s ti c s2 : 5 3 1 - 5 3 8

We st M , H a r r i s o n R J , M i g o n H S ( 1 98 5 ) D y n a m i c g e n e r a li z e d l i n e a r m o d e l s a n d B a y e s ia n f o r e c as t in g .JA SA 80:73 - 83

Rece ived 9 June 1989Revised vers ion 10 January 1990

on kalman filtering, posterior mode estimation and fisher scoring.pdf

Documents