constrained kalman ﬁlter and recursive garch...

Dissertation summaryKalman filter with state constraints

Recursive estimation of GARCH modelsReferences, contact

Constrained Kalman filter andrecursive GARCH estimation

Tomas Hanzak

seminarStochastic modeling in economics and finance

February 20, 2012

1 / 34 Tomas Hanzak Constrained Kalman filter and recursive GARCH estimation



Content

1 Dissertation summaryMy dissertation topicExpected content of the thesisMy publications

2 Kalman filter with state constraintsBasics of Kalman filterRLS and ARMA estimation by Kalman filterState constraints in Kalman filter

3 Recursive estimation of GARCH modelsBasics of GARCH models2S-LS, 2S-RLS and 2S-PLR estimation of GARCH models1S-RLS estimation method for GARCH models




My dissertation topicExpected content of the thesisMy publications

Basic information

Student’s name: Tomas Hanzak

Doctoral student since: October 2007

Study branch: m-5 Econometrics and operational research

Dissertation title: Some problems of periodic time series

Supervisor: prof. RNDr. Tomas Cipra, DrSc.

Supervising department: Dept. of Probability and Mathematical Statistics

Annotation:

The doctoral student will familiarize with some time series analysis methodswhich are modified to be applicable for time series with irregular observations.The stress will be put on modeling periodicity (seasonality) in these time series.The suggested methods will be transposed into the software form.

Expected completion date: 2013/2014





Looking back...

Topics of my previous presentations here:

1 2008, May 19: Improved Holt method for irregular time series

2 2009, March 16: Holt-Winters method with general seasonality

3 2010, March 22: Exponential smoothing for time series with outliers

4 2011, February 28: Summary, Holt-Winters method reminded

5 2012, February 20: Recursive estimation of parameter constrainedGARCH model





Irregularly observed ARIMA/SARIMA processes

Classical exponential smoothing methods (simple exponential smoothing,Holt method, Holt-Winters method) are MSE-optimal for certainARIMA/SARIMA models.

These models can be taken as a basis for model based approachto construction of extensions of these methods for time series with missingobservations.

Aldrin and Damsleth (1989) derived optimal smoothing coefficients for a caseof a single gap in observations for simple exponential smoothing(ARIMA(0, 1, 1)) and Holt method (ARIMA(0, 2, 2)).

Ratinger (1996) studied optimal smoothing coefficients for a case of a singlegap in observations for Holt-Winters method with additive seasonality(SARIMA model).

I derived optimal smoothing coefficients for a general time series withmissing observations for a simple exponential smoothing.





Improved Holt method for irregular time series (1)

Wright (1986) suggested an extension of Holt method for irregular time series:

Stn+1 = (1− αtn+1) · [Stn + (tn+1 − tn) · Ttn ] + αtn+1 · ytn+1 ,

Ttn+1 = (1− γtn+1) · Ttn + γtn+1 ·Stn+1 − Stn

tn+1 − tn,

αtn+1 =αtn

αtn + (1− α)tn+1−tnand γtn+1 =

γtn

γtn + (1− γ)tn+1−tn.

The division by tn+1 − tn makes Ttn+1 very sensitive in the case thattn+1 − tn ≈ 0 ⇒ danger of slope estimate Ttn+1 destruction.

Suggested modification consists in a modified smoothing coefficient γtn+1 :

γtn+1 =γtn

γtn +tn−tn−1

tn+1−tn(1− γ)tn+1−tn

.

The modified method outperformed the original one in a simulation study.





Improved Holt method for irregular time series (2)

Visual illustration:

Forecasts and smoothed values obtained by the original and modified method(both with fixed α = 0.3 and γ = 0.1).





Holt-Winters method with general seasonality

The paper suggests a generalization of widely used Holt-Winters smoothingand forecasting method for seasonal time series.

The general concept of seasonality modeling is introduced bothfor the additive and multiplicative case.

Several special cases are discussed, including a linear interpolation of seasonalindices and a usage of trigonometric functions.

Both methods are fully applicable for time series with irregularly observeddata (just the special case of missing observations was covered up to now).

Moreover, they sometimes outperform the classical Holt-Winters method evenfor regular time series.

A simulation study and real data examples compare the suggested methodswith the classical one.





Robust exponential smoothing

Occurrence of outliers in times series were discussed and conceptual approachesto this problem were listed.

Several particular methods were mentioned and described: Exponentialsmoothing in L1 norm, discounted M-estimation using IRLS and robustKalman filter for state space models.

Practically the robust Kalman filter can lead to error truncation:One-step-ahead forecasting error is truncated before entering intothe error-correction formulas of the method.

Forecasting errors scale estimator is needed to calibrate the truncation.E.g. GARCH(1, 1) approach can be used.

Error truncation methods were compared with M-estimation in a largesimulation study.





Expected content of the thesis

1 Introduction

2 Topic overview

2.1 Periodicity in time series2.2 Time series irregularities2.3 Overview of approaches

3 Exponential smoothing for irregular time series

3.1 Methods overview3.2 Improved Holt method for irregular time series3.3 Holt-Winters method with general seasonality

4 Irregularly observed ARIMA/SARIMA processes

5 Robust methods

5.1 Methods overview5.2 Error truncation

6 Summary





My publications

P. Mach, T. Hanzak: Razebne ve spojitem case. Politicka ekonomie 4(2004), 531–536.

T. Cipra, T. Hanzak: Exponential smoothing for irregular time series.Kybernetika 44 (2008), 385–399.

T. Hanzak: Improved Holt method for irregular time series. WDS’08Proceedings of Contributed Papers, Part I, 62–67, 2008.

T. Cipra, T. Hanzak : Exponential smoothing for time series with outliers.Kybernetika 47 (2011), 165–178.

T. Hanzak : Holt-Winters method with general seasonality. Kybernetika48 (2012), 1–15.




Basics of Kalman filterRLS and ARMA estimation by Kalman filterState constraints in Kalman filter

Discrete dynamic linear model

Just to remind...

Discrete dynamic linear model (DLM):

St+1 = AtSt + vt , t = 0, 1, 2, ...,

yt = HtSt + wt , t = 0, 1, 2, ...,

where St is the state vector, yt is the vector of observations, At , Ht arematrices known at time t, v is a random state innovation and w is anobservation noise, both being white noise processes, having covariance matricesVt and Wt and being mutually uncorrelated.

Both state and observations can be either scalar or vector of given dimensions.

The first equation is known as state equation, the second one as observationequation.





Kalman filtering

Kalman filter constructs the least squares linear estimate Stt of the unknown

state St based on the information available up to time t. Let us further denoteSt−1

t the estimate of St based on the information up to time t − 1 and Ptt and

Pt−1t be the estimate (or prediction) error covariance matrices of St

t and St−1t

respectively. Then the Kalman filter formulas are:

Stt = St−1

t + Pt−1t H′

t

�HtP

t−1t H′

t + Wt

�−1

(yt −Ht St−1t ) ,

Ptt = Pt−1

t − Pt−1t H′

t

�HtP

t−1t H′

t + Wt

�−1

HtPt−1t ,

St−1t = At S

t−1t−1 ,

Pt−1t = AtP

t−1t−1A

′t + Vt .

One must set S00 and P0

0 somehow to start the recursions.





Recursive Least Squares by Kalman filter

Recursive Least Squares (RLS) method can be derived either from theclassical OLS formulas using the matrix inversion lemma, or by using Kalmanfilter. Consider DLM representing a linear regression model:

bt+1 = bt ,

yt = X′tbt + εt .

yt and εt are scalars, vector of unknown parameters b is the sate vector here,H = X′

t , At = I, Vt = 0 and Wt = σ2t .

It is bt−1t = bt−1

t−1 and Pt−1t = Pt−1

t−1 which simplifies the Kalman filter formulas(we use just one t index for simpler notation):

bt = bt−1 +Pt−1Xt(yt − X′

t bt−1)

X′tPt−1Xt + σ2

t

,

Pt = Pt−1 −Pt−1XtX

′tP

′t−1

X′tPt−1Xt + σ2

t

.





ARMA estimation by Kalman filter

Let us consider ARMA(p, q) model

yt = α0 +

pXi=1

αiyt−i + εt +

qXj=1

βjεt−j .

Let b = (α0, . . . , αp, β0, . . . , βq)′ and Xt = (1, yt−1, . . . , yt−p, εt−1, . . . , εt−q)

′.Then we have

yt = X′tbt + εt

and we can estimate ARMA parameters b using RLS method described before.

We just need to estimate the lagged residuals εt−1, . . . , εt−q at time t to formXt which can be used instead of theoretical Xt . It can be done recursively by

εt = yt − X′t bt .





State constraints - introduction

In practise, there is sometimes an information known a priori about the statevector, which is not contained in the DLM formulation itself and so which isnot reflected by Kalman filter estimates explicitly. The filter is not optimalin this situation and the estimates can violate the a priori knowledge.

Examples:

Portfolio weights sum to 1 (dynamic portfolio analysis by Kalman filter).

Model parameters lie in ceratin regions (a priori known signs, stability andinvertibility regions etc.)

The vehicle located by a radar is on the known track. Etc.

From mathematic point of view, this is formulated as state vector constraints.These can be either

equalities or inequalities and

either linear or non-linear.

There are several ways of incorporating constraints into K-F, see Simon (2010).





Model reduction

When the constraints are linear equalities, they can be reflected by the statespace dimensionality reduction (by the number of equalities).

Is is similar to using a priori known linear relations among regressionparameters to reduce the number of model parameters.

Example ...

Pros and cons:

:-) Reflecting constraints also reduce the computational complexityof the model by reducing its dimension.

:-( Interpretation of the state space components become more difficult.

:-( Cannot be used for neither non-linear equalities not inequalities.





Perfect measurement

When the constraints are again linear equalities, they can be incorporated intoDLM as an additional observation equations.

The dimensionality of observations is increased by adding each equality as oneadditional dimension:

Left side coefficients of the linear equality form the new row of H matrix, rightside coefficient forms an additional component of observation vector y.

The corresponding component of the observation noise (newly introduced) has0 variance (so it really a perfect measurement).

Example: Portfolio weights.

Pros and cons:

:-) Interpretation of the state space components is not affected.

:-( Reflecting constraints increase model dimensionality.

:-( Cannot be used for neither non-linear equalities not inequalities.





State projection

Idea: State vector estimated by the original unconstrained K-F is projectedonto the constraint surface.

The projection can be done simply in Least Squares sense or reflectingelliptical shape of Pt

t .

In the case of linear equalities the projection can be derived analytically.

In the case of linear inequalities, quadratic programming techniquesmust be employed.

When the ”feasible” state set is convex, the projected estimate is alwayscloser to the true state vector than the original estimate.

Example: Stationarity region for AR(2) model parameters.

Pros and cons:

:-) Interpretation of the state space components is not affected.

:-) State projection has a transparent geometrical interpretation.

:-( Projection must be performed which can be non-trivial.




Basics of GARCH models2S-LS, 2S-RLS and 2S-PLR estimation of GARCH models1S-RLS estimation method for GARCH models

ARCH and GARCH models - definition

Many financial time series exhibit so called conditional heteroscedasticity.Engle (1982) suggested modeling this by ARCH models. Bollerslev (1986)extended it to GARCH models.

Let us consider GARCH(p, q) model for time series {yt}:

yt =√

htζt ,

ht = α0 +

pXk=1

αky2t−k +

qXk=1

βkht−k ,

where ζt is iid ∼ N(0, 1) and (α0, . . . , αp, β0, . . . , βq) are non-negative unknownparameters. ht is a conditional variance of yt given the past values of y .

ARCH(p) model is obtained from GARCH(p, q) simply by setting q = 0.

Parameters (α0, α1, . . . , αp, β0, . . . , βq) are usually estimated by maximumlikelihood. However, the likelihood function is known to be very flat, withmultiple local maxima.





GARCH(1, 1)

Among the family of GARCH models, the far most used in practice isGARCH(1, 1) (I have never seen anything else applied):

yt =√

htζt ,

ht = α0 + α1y2t−1 + βht−1 .

It is analogous to simple exponential smoothing: new information aboutthe volatility (y 2

t−1) is composed with the current information (ht−1).

Necessary and sufficient condition for stationarity of GARCH(1, 1) model isα0 > 0, α1, β ≥ 0 and α1 + β < 1. This assures ht > 0 and prevents fromunit-root-like or explosive behavior of {ht} series.

GARCH(1, 1) can be successfully applied to exchange rates or stock pricereturns (or indices based on such a financial data). Typical valuesof GARCH(1, 1) parameters are α1 ∈ (0, 0.2) and β ∈ (0.7, 1).





ARMA representation of GARCH process

GARCH(p, q) model for yt can be viewed as ARMA(p, q) model for thesquared vales y 2

t :

y 2t = α0 +

max(p,q)Xi=1

(αi + βi )y2t−k + vt +

qXj=1

(−βj)vt−j ,

where non-defined α or β coefficients are defined by 0 and vt ≡ y 2t − ht are

zero mean ARMA residuals with variance 2h2t . I.e. the residuals vt are

heteroscedastic (and also skewed).

This ARMA representation can be used for identification of correct orders pand q of GARCH model (e.g. via ACF and PACF plots of {y 2

t }) or it can evenserve as a preliminary estimation tool.

Later we will inspire from ARMA representation of GARCH to use Kalmanfilter for GARCH recursive estimation.





OLS estimation of ARCH model

AR(p) model for y can be estimated by OLS (y regressed on constant and itslagged values up to lag p).

As we know, ARCH(p) for y is AR(p) for y 2. So ARCH(p) can be estimatedby OLS as well (y 2 regressed on constant and its lagged values up to lag p).

Let us suppose we have yt , . . . , yT available. Let Xt = (1, y 2t−1, . . . , y

2t−p)

′ bethe vector of regressors at time t and b = (α0, α1, . . . , αp) the vectorof regression parameters. Then OLS estimate of b is simply

bOLST =

TX

t=1

XtX′t

!−1 TXt=1

Xty2t

!.

Estimate bOLST does not take the heteroscedasticity of vt into account and

thus it is not efficient. But is can serve us as a preliminary estimate for 2S-LSestimation...





2S-LS estimation of ARCH model

Bose and Mukherjee (2003) use OLS estimate bOLST of ARCH(p) parameters as

a preliminary (1st stage) estimate for WLS (Weighted Least Squares) in the 2nd

stage.

Weights are taken reciprocal to the conditional variances h2t (T ) derived from

OLS estimate bOLST :

b2S−LST =

"TX

t=1

XtX′t/h2

t (T )

#−1 " TXt=1

Xty2t /h2

t (T )

#,

where h2t (T ) = Xt b

OLST .

This 2S-LS estimate is consistent and asymptotically fully efficient and itscomputational complexity is much lower when compared to maximumlikelihood estimation.





2S-RLS estimation of ARCH models

OLS and WLS estimation can be done recursively (using RLS method). Butfor a recursive WLS we would need to know the values of weights not laterthan at the current time step.

Aknouche and Guerbyenne (2006) suggested to modify the 2nd stage estimationslightly to

b2S−RLST =

"TX

t=1

XtX′t/h2

t (t)

#−1 " TXt=1

Xty2t /h2

t (t)

#.

I.e. we don’t wait for the final OLS estimate (based on the full sampleof size T ) but we use always OLS estimate from sample (y1, . . . , yt) to formthe weight at time t in the following WLS stage.

The resulting method is truly recursive (2 Stage Recursive Least Squares -2S-RLS)





2S-PLR estimation of GARCH models (1)

Aknouche and Guerbyenne (2006) continued to develop a recursive estimationalgorithm for GARCH models.

They call it 2 Stage Pseudo-Linear Regression (2S-PLR).

The problem is that in GARCH model, as in ARMA model, the ”regression”equation for y 2 contains also the lagged values of its residuals vt .

This can be overcome in the same way as in ARMA estimation by RLS. Boseand Mukherjee (2003) again considered 2 separate stages proceeded recursivelyone along the other:

1 1st one ignoring heteroscedasticity of vt

2 2nd one taking heteroscedasticity of vt into account, using conditionalvariances based on the current estimate from the 1st stage.

In fact, in the 2nd stage at time t they use estimate from the 1st stage fromtime t − 1 (to make it easier to prove the convergence of the final estimate).





2S-PLR estimation of GARCH models (2)

The complete 2S-PLR method, using the notation of the authors, is givenby formulas bellow.

We see that it really consists from two RLS algorithms which we can see alsoas two Kalman filters running one along each other.





Comments to Aknouche’s and Guerbyenne’s 2S-PLR

The computational complexity of the method is quite large: one mustupdate appr. 20 quantities recursively.

It is a question why 2 stages are needed at all?

Why not to use the latest 2nd stage estimate of parameters alsoto calculate the conditional variance to be used in the next 2nd stage timestep to reflect the heteroscedasticity?

Then the 1st stage becomes useless at all and the computationalcomplexity reduce by one half!

What about a non-recursive variant of GARCH estimator, using simpleOLS and/or WLS regressions, possibly in 2, 3 or 4 iterations?

The fact is that the recursion helps us to estimate the lagged residuals(or equivalently the lagged predictions) and conditional variance.





1S-RLS for GARCH(1, 1)

After replacing the 1st stage estimates in the 2nd stage by the lagged 2nd stageestimates, we get a method that can be called 1 Stage Pseudo-LinearRegression (1S-PLR).

This can also be viewed as y 2 - ARMA(1, 1) estimation by Kalman filter,reflecting residual heteroscedasticity.

For GARCH(1, 1) model the 1S-PLR method can be formulated as

bt = bt−1 +Pt−1Xt(y

2t − Xt bt−1)

(Xt bt−1)2 + X′tPt−1Xt

,

Pt = Pt−1 −Pt−1XtX

′tP

′t−1

(Xt bt−1)2 + X′tPt−1Xt

,

Xt = (1, y 2t−1, ht−1)

′ is a regression design vector at time t,

ht = Xt bt is the estimated conditional variance and

bt = (α0t , α1t , βt) is a parameter estimate after yt is used.





Adding parameter constraints - stability region

At the beginning of the series, the recursive methods have very littleinformation available to construct the parameters’ estimates. So it can easilyhappen that the estimates fall outside stability region. Then the estimatedconditional variance can be even negative!

We can add parameters projection onto stability region into each time stepof the algorithms to address this issue.

For example, let us remind that the stability conditions of GARCH(1, 1) areα0 > 0, α1, β ≥ 0 and α1 + β < 1. The stability region is thus the interval(0,∞) for α0 and independently of this the semi-closed triangle definedby vertices (0, 0), (0, 1) and (1, 0) for α1 and β.

Thus a projection of parameters’ estimates onto such a region is quite simplyderived analytically from a geometrical preview.

Stability region is convex and so given that we believe that we work with trulystationary GARCH model, the projected parameters’ estimates are closerto the true values than the original ones.





Implementation attempt, further work

I’ve implemented both 2S-PLR and 1S-PLR methods for GARCH(1, 1) model.Both with and without stability projections.

I run it on simulated GARCH(1, 1) series. Unfortunately it didn’t work,although the implementation seems to be correct (I spent a whole day bychecking it and experimenting with the code). Estimates sometimes convergebut often explode or get stuck in strange points.

I will definitely try to investigate it further since at least 2S-RLS should work(its authors claim it works) and 1S-RLS seems to be just its slight improvement.

I’ve planned to do a simulation study, evaluating the difference of 2S-PLR and1S-PLR methods and primarily to evaluate the impact of parametersprojections (depending on the position of the true parameter vector relativelyto stability region border and on the time series length). Real dataillustrations were also planned (on PX index and CZK/EUR log-returns data).

I will try to construct a non-recursive variant of GARCH estimator. Parametersprojection to stability region can be used after each iteration of OLS/WLS.




ReferencesTeaching and supervised thesesContacts

References

A. Aknouche , H. Guerbyenne : Recursive estimation of GARCH models.Commun. Stat.-Simul. C. 35 (2006), 939–956.

T. Bollerslev : Generalized Autoregressive Conditional Heteroskedasticity.Journal of Econometrics 31 (1986), 307–327.

A. Bose, K. Mukherjee : Estimating the ARCH parameters by solvinglinear equations. J. Time Ser. Anal. 24 (2003), 127–136.

R. F. Engle : Autoregressive conditional heteroskedasticity and estimatesof the variance of UK inflation. Econometrica 50 (1982), 987–1008.

D. Simon : Kalman Filtering with State Constraints: A Survey of Linearand Nonlinear Algorithms. Control Theory & Applications 8 (2010),1303–18.





Teaching and supervised theses

Teaching:

Credit risk in banking (NFAP042). Lecture shared with Milos Kopa.

Currently supervised theses:

Diploma thesis: Aleh Masaila: Regression trees. Assigned in May 2010.

Diploma thesis: Ivana Menhartova: Methods of dynamical analysisof portfolio composition. Assigned in September 2010.

Bachelor thesis: Filip Simsa: Regression goodness-of-fit criteria accordingto dependent variable type. Assigned in November 2011.

Bachelor thesis: Samuel Rıha: Gini coefficient maximization in binarylogistic regression. Assigned in November 2011.





Contacts

Tomas Hanzak

mobile: 604 799 879e-mail: [email protected]: www.thanzak.sweb.cz

Department of Probability and Mathematical StatisticsFaculty of Mathematics and PhysicsCharles University in Prague

Sokolovska 83, 186 75 Praha 8.

e-mail: [email protected]: www.karlin.mff.cuni.cz/ kpms

MEDIARESEARCH, a.s.

Ceskobratrska 1, 130 00 Praha 3.

mobile: 725 535 535e-mail: [email protected]: www.mediaresearch.cz


constrained kalman ﬁlter and recursive garch...

Documents