# hidden markov models for default risk wai-ki ching ... wkc/talks/ ching department

Post on 14-Jun-2018

212 views

Embed Size (px)

TRANSCRIPT

Hidden Markov Models for Default Risk

Wai-Ki CHING

Department of Mathematics

The University of Hong Kong

Abstract: Default risk in commercial lending is one of the majorconcerns of the creditors. In this article, we introduce a new Hid-den Markov Model with multiple observable sequences (MHMM),assuming that all the observable sequences are driven by a commonhidden sequence, and utilize it to analyze default data in a networkof sectors. Efficient estimation method is then adopted to esti-mate the model parameters. To further illustrate the advantagesof MHMM, we compare the hidden risk state process obtained inMHMM with those from the traditional HMMs using credit defaultdata.

A joint work with Terence H. Leung. Research Supported inPart by HK RGC Grant 7017/07P.

1

Introduction

(1) Motivations and Objectives.

(2) The Idea of HMM Through an Example.

(3) Parameters Estimation.

(4) Application to Credit Default Data.

(5) Concluding Remarks.

2

1. Motivations and Objectives.

Hidden Markov Models (HMMs) are widely used in many areas.

-Speech Recognition. L. Rabiner, A Tutorial on Hidden MarkovModels and Selected Applications in Speech Recognition, Proceed-ings of the IEEE, 77 (1989) 257286.

-Computer Vision. H. Bunke and T. Caelli, Hidden Markov Mod-els : Applications in Computer Vision, Editors, Horst Bunke, TerryCaelli, Singapore, World Scientific (2001).

-Bioinformatics. T. Koski, Hidden Markov Models for Bioinfor-matics, Kluwer Academic Publisher, Dordrecht (2001).

-Finance. Rogemar S. Mamon and Robert J. Elliott, Hidden MarkovModels in Finance, New York : Springer (2007).

3

In a HMM, there are two types of states: observable states andhidden states. The hidden states follow a Markov chain process

and the observable states are driven by the hidden states.

To define a HMM, one has to define the number of both typesof states and also their transition probabilities.

The major problem in constructing a HMM is to determine thetransition probabilities of the hidden states because the transitions

among the hidden states are supposed to be unobservable. Ad-

vanced methods based on EM-like algorithm will be proposed to

solve the problem.

Here we propose a HMM for multiple observation sequences.The model is then applied to modeling credit default data. Hid-

den states concerns the risk level (economic condition) and the

observable states are the number of default bonds.

4

2. The Idea of HMM Through an Example.

We consider the process of choosing dice and recording the numberof dots by throwing the dice.

Suppose we have two dice A and B such that Die A is fair andDie B is bias.

The probability distributions of the dots obtained by throwing DiceA and Dice B are given in the table below.

Die 1 2 3 4 5 6A 1/6 1/6 1/6 1/6 1/6 1/6B 1/6 1/6 1/3 1/6 1/12 1/12

Table 1.

5

Each time a die is chosen, with probability , Die B is chosengiven that Die A was chosen last time. And with probability ,Die A is chosen given that Die B was chosen last time. It is a2-state Markov chain process having transition probability matrix:

Die ADie B

(1

1 )

This process is hidden because we dont know actually which dieis being chosen.

The chosen die is then thrown and the number of dots (thisis observable) obtained is recorded. The following is a possiblerealization of the process:

AHidden

AHidden

BHidden

BHidden

AHidden

1Observable

3Observable

6Observable

5Observable

4Observable

6

The followings are the model parameters of our HMM.

N , number of hidden statesK, number of observable sequencesT , length of the observation periodM , number of distinct observable statesS = {S1, . . . , SN}, the set of hidden statesqt, hidden state at time tV = {v1, . . . , vM}, the set of observable statesOk = (ok1, o

k2, . . . , o

kT ), kth observation sequence

wk, the weighting of the kth observation sequence

Q = (q0, q1, q2, . . . , qT ), the sequence of hidden stateaij, transition probabilities from hidden State i to hidden State jbkj (v), the probability of symbol v being observed at state j in thek-th sequence, initial state distribution = (A, B, ), the model training parameters (N , M are fixed)

7

3. Model Parameter Estimation.

In order to define an HMM, one has to solve THREE problems:

(I) To efficiently compute P (O|), the likelihood of of a given ob-servation sequence, when we are given the model = (A, B, ) andthe observation sequence O = O1O2 . . . OT .Forward and Backward Algorithm, (Baum, 1972).

(II) To find the most likely hidden sequence.Viterbi algorithm (Viterbi, 1967), a DP approach.

(III) To adjust the parameters = (A, B, ) of the model so as tomaximize P (O|).Baum-Welch algorithm (Baum-Welch, 1970), an EM-like algo-rithm.

8

For the classical HMM, (one observation sequence) (I), (II) and(III) can be solved by using EM algorithm, Rabiner (1989) and

MacDonald and Zucchini (1999).

A HMM has been proposed by Li et al. (2000) for multiple se-quences.

In our HMM, instead of having K hidden sequences as in Li etal. (2000), here we only consider one common (global) hidden

sequence with weighting given to each observed sequence. We

define the likelihood as

P (O, Q|)) = q0T

t=2

[aqt1,qtK

k=1

(bkqt(okt ))

wk]. (1)

Here wk > 0 is the weighting for the k-th sequence andK

k=11wk

= 1.

9

4. Application to Credit Default Data.

We present the estimation results of our MHMM (multiple ob-servable sequences) and also the classical HMM (one observable

sequence) using the observed default data extracted from the fig-

ures in Giampieri et al. (2005)

Giampieri et al. (2005) apply the HMM to the quarterly bonddefaults data of four sectors ((i) Consumer, (ii) Energy, (iii)

Media and (iv) Transportation) in the United States taken from

Standard & Poors ProCredit6.2 database. The data set covers the

period from January 1981 to December 2002.

The total number of bonds in January 1981 was 281 while thetotal number of bonds in December 2002 was 222. At the beginning

there are 1024,420,650 and 281 non-default bonds in the above

sectors respectively. At the end of observation period, there are

251,71,133 and 59 default bonds in the above sectors respectively.

10

In our study, we assume that there are two hidden states 0 (nor-mal risk) and 1 (high risk) for the global risk state.

We further assume that there are 3 observable states for thecredit default sequences and they are b1 (0 or 1 default), b2 (2 or

3 defaults) and b3 (4 or more defaults).

We remark that this classification of observable states is subjectiveand can be adjusted.

Moreover, we assume that all the weighting wi are equal in ourcalculation and again this can also be adjusted.

11

All computations were done on a Pentium 4HT PC with MATLAB.

We denote the model parameters (Am, B(i)m , m) for our proposedHMM and (Ai, Bi, i), i = 1,2,3,4 for the classical HMM of individual

sequences 1,2,3,4 respectively.

Since the problem is highly non-linear. The EM algorithm mayend up with local optima.

We run the algorithm 100 times with different initial guessesof the model parameters and get the best estimate.

The stopping criteria is that the difference between successiveestimates in ||.||2 is less than 106.

12

The model parameters of our model MHMM (multiple sequences)and the traditional HMM (individual sequence):

Am =

(0.9660 0.03400.0395 0.9605

)m =

(0.96070.0393

)

A1 =

(0.9367 0.06330.0629 0.9371

)1 =

(0.93430.0657

)

A2 =

(0.9502 0.04980.0487 0.9513

)2 =

(0.08700.9130

)

A3 =

(0.9622 0.03780.0390 0.9610

)3 =

(0.95400.0460

)

A4 =

(0.9882 0.01180.0000 1.0000

)4 =

(1.00000.0000

)

13

B(1)m =

(0.6130 0.3040 0.08300.0000 0.2768 0.7232

)B(1) =

(0.7089 0.2911 0.00000.0321 0.3013 0.6666

)

B(2)m =

(0.8171 0.1664 0.01660.6036 0.3964 0.0000

)B(2) =

(0.9561 0.0439 0.00000.4418 0.5298 0.0284

)

B(3)m =

(0.8759 0.1065 0.01760.1134 0.3824 0.5042

)B(3) =

(0.9035 0.0806 0.01590.1437 0.4008 0.4555

)

B(4)m =

(0.9505 0.0495 0.00000.5656 0.3259 0.1085

)B(4) =

(0.8588 0.1412 0.00000.0000 0.0000 1.0000

).

14

4.1 Inferring the Hidden Risk State

We present the hidden sequences obtained from our MHMM formultiple sequences Hm and also the classical HMM for individual

sequences Hi.

We found that Hm is close to H3 (media sector) and H1 (consumersector). A bit different from H2 (energy sector).

But Hm is significantly different from H4 (transportation sec-tor). This is because large number of defaults only occurred in the

last three quarters in the transportation sector. Our MHMM can

give a consistent but a more holistic situation of the global eco-

nomic risk.

15

The estimated hidden sequences (risk states: 0=normal risk and1=high risk):

Hm : MHHM Model0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

H1 : Consumer Sector0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0