Systemic Real and Systemic Real and Financial Risks: Financial Risks: Measurement, Measurement, Forecasting, Forecasting,

and Stress Testing and Stress Testing Gianni De NicolòGianni De Nicolò

International Monetary Fund and CESifoInternational Monetary Fund and CESifo

Marcella LucchettaMarcella LucchettaUniversity of VeniceUniversity of Venice

The views expressed in this paper are those of the authors and do The views expressed in this paper are those of the authors and do not necessarily represent those of the IMF.not necessarily represent those of the IMF.

Motivation Motivation

Available monitoring technologies failed to Available monitoring technologies failed to provide early warnings on the crisis in 2007-provide early warnings on the crisis in 2007-2008. 2008.

Building on De NicolBuilding on De Nicolò and Lucchetta (2010), and Lucchetta (2010), we develop a model that can be useful for we develop a model that can be useful for

positive analysis, positive analysis, andand as a systemic risk monitoring system.as a systemic risk monitoring system.

Limitations of current Limitations of current modelingmodeling

DSGE models DSGE models 1.1. Incorporation of interactions between financial and Incorporation of interactions between financial and

real sectors still in its infancy real sectors still in its infancy 2.2. Forecasting performance not yet firmly established. Forecasting performance not yet firmly established.

Stress testing proceduresStress testing procedures1.1. ““shocked” variables typically shocked” variables typically endogenous (shock to endogenous (shock to

the “cause” or the “symptom”?)the “cause” or the “symptom”?)2.2. difficult to assess the quantitative results.difficult to assess the quantitative results.

Our contributionOur contribution Our model complements DSGE modeling by Our model complements DSGE modeling by

exploiting: exploiting:

the the forecasting power of a Dynamic Factor Model forecasting power of a Dynamic Factor Model (DFM) (DFM) with many predictorswith many predictors

structural identification structural identification based on explicit based on explicit theoretical constructs (such as DSGE models)theoretical constructs (such as DSGE models)

Flexibility (Flexibility (applicable to multiple countries/sector applicable to multiple countries/sector datasets, and different data frequencies). datasets, and different data frequencies).

Output of the ModelOutput of the Model

a)a) density forecasts density forecasts of indicators ofof indicators of systemic real risk and systemic systemic real risk and systemic financial risk;financial risk;

b)b) reduced-form stress tests, reduced-form stress tests, as as historical simulations;historical simulations;

c)c) structural stress-tests, structural stress-tests, as as impulse responses of systemic risk impulse responses of systemic risk indicators to structural shocks indicators to structural shocks identified by standard identified by standard macroeconomic and banking theory. macroeconomic and banking theory.

Systemic Real Risk Systemic Real Risk

Systemic Real Risk is measured by GDP-Expected Shortfall (GDPES), given by the expected loss in GDP growth conditional on a given level of GDP-at-Risk (GDPaR)

GDPaR is the worst predicted realization of quarterly growth in real GDP at a given (low) probability

Systemic Financial RiskSystemic Financial Risk A financial health indicator (FS) : return of a

portfolio of financial firms less the return of the market

Systemic Financial Risk is measured by FS-Expected Shortfall (FSES) , given by the expected loss in FS conditional on a given level of FS-at-Risk (FSaR)

FSaR is the worst predicted realization of the FS indicator at a given (low) probability level

The statistical models The statistical models GDP growth and FS are modeled through GDP growth and FS are modeled through

a version of a a version of a factor-augmented VAR factor-augmented VAR (FAVAR)(FAVAR) model (e.g. Stock and Watson, model (e.g. Stock and Watson, 2002 and 2005)2002 and 2005)

Density forecasts of GDPG and FS obtained estimating a set of quantile auto-regressions

Systemic Risk Indicators constructed using density forecasts

STRESS TESTING = STRESS TESTING = Measurement of Measurement of impact impact and and persistencepersistence

of configurations of of configurations of unexpected unexpected (structural) shocks (structural) shocks

on systemic risk indicatorson systemic risk indicators

Reduced-form stress tests: based on shocks recovered from a statistical model of the

quantiles (distribution) of GDP growth and FS

Structural stress tests: based on shocks derived from theoretical models IdentificationIdentification ofof structural shocks structural shocks accomplished with accomplished with

theory-based sign restrictions (Canova and De Nicolò, JME theory-based sign restrictions (Canova and De Nicolò, JME 2002)2002)

ImplementationImplementation

We use macroeconomic and financial series for the G-7 economies for the period 1980:Q1-2010:Q1

For each country, the vector of quarterly series includes about 95 series classified into

1. equity markets data2. credit conditions3. indicators of real activity

Main Results Main Results

1.1. Significant forecasting power for Significant forecasting power for tail risk tail risk realizations of real activity realizations of real activity and financial healthand financial health

2.2. Both reduced-form and structural Both reduced-form and structural stress tests provide early warnings stress tests provide early warnings of real and financial vulnerabilitiesof real and financial vulnerabilities

3.3. In all countriesIn all countries:: aggregate demand shocks drive the aggregate demand shocks drive the

real cyclereal cycle bank credit demand shocks drive the bank credit demand shocks drive the

bank lending cycle bank lending cycle real drives financialreal drives financial

Plan of the presentationPlan of the presentation

The ModelThe Model Estimation and Forecasting (details)Estimation and Forecasting (details) Forecast EvaluationForecast Evaluation Reduced-Form Stress TestsReduced-Form Stress Tests Structural Stress TestsStructural Stress Tests Modeling DevelopmentsModeling Developments

The Dynamic Factor Model The Dynamic Factor Model (DFM)(DFM)

(static form)(static form)

11 1( )R

t t R t tGDPG F L GDPG u

21 1( )F

t t F t tFS F L FS u

X it

iF

t

iX

it 1 v

it

Ft(L)F

t 1 G

t

(5(5))

(8(8))

(7(7))

(6(6))

Density Forecasts Density Forecasts Density forecasts of GDP growth and FS Density forecasts of GDP growth and FS

obtained estimating 99 quantile auto-obtained estimating 99 quantile auto-regressions: regressions:

These “raw” quantile estimates are These “raw” quantile estimates are “rearranged” at each date to overcome “rearranged” at each date to overcome potential “crossing” (novel application of potential “crossing” (novel application of Chernuzikhov et al. , Chernuzikhov et al. , EconometricaEconometrica 2010) 2010)

1 1ˆˆ ˆ( ) ( ) ( ) ( )( )R

t t R tGDPGQ F L GDPG

1 2ˆˆ ˆ( ) ( ) ( ) ( )( )F

t t F tFSQ F L FS

Density Forecasts (2008q3 and 2010q2) (2008q3 and 2010q2)

0.2

.4.6

.8

- 3 - 2 . 5 -2 - 1 . 5 -1 - . 5 0 .5 1 1 .5 2 2 .5 3x

k d e n s it y s r 2 0 0 8 q3 k d e n s it y s r 2 0 1 0 q 3

D e n s it y F o r e c a s t sG D P G r o w t h

0.0

2.0

4.0

6.0

8

- 3 5 - 3 0 - 2 5 - 2 0 -1 5 -1 0 - 5 0 5 1 0 1 5 2 0 2 5 3 0 3 5x

k d e n s it y s f 2 0 0 8 q 3 k d e n s it y s f 2 0 1 0 q 3

D e n s it y F o r e c a s t sF S I n d ic a t o r

U n i t e d S t a t e s

For any given

Systemic Risk Indicators Systemic Risk Indicators

(0,1)

GDPESt( )

t(GDPG

t| GDPG

tGDPaR

t( ))

FSESt( )

t(FS

t| FS

tFSaR

t( ))

Systemic Risk Fan Charts -2

02

46

1980q1 1990q1 2000q1 2010q1time

ges20 ges5

gesX = GDP Expected Shortfall with probability XSystemic Real Risk

020

40

60

1980q1 1990q1 2000q1 2010q1time

fes20 fes5

fesX = FS Expected Shortfall with probability X

Systemic Financial Risk

United States: Systemic Risk Fan Charts

Estimation and Forecasting

(details)Four steps:1) Number of factors and lags2) Quantile estimation3) Density estimates and

Expected Shortfalls4) Forecasting

(1) (1) Number of Factors and Lags

Extract all factors with eigenvalues Extract all factors with eigenvalues greater than 1greater than 1

Order factors according to the Order factors according to the explanatory power of the variance of explanatory power of the variance of the data and construct the data and construct

Choose the number of lags Choose the number of lags L L and the and the number of static factors that number of static factors that maximize BICmaximize BIC Criterion among 4 by R specifications of the FAVAR

1 1 2 1 2{( ), ( , ),...., ( , ,..., )}r r r RF F F F F F F

r F

(2) Quantile Estimation (2) Quantile Estimation

use the optimal number of lags , the number of static factors , and the estimated factors to estimate quantile auto-regressions for

specified as in (7) and (8)

address the crossing problem by adopting the “rearrangement” procedure introduced by Chernuzukhov, Fernandez-Val and Galichon (2010)

L*

r*

1,2....99

(3) Continuous Density Estimates

obtain continuous densities and compute expected shortfalls as

where is the quantile corresponding to probability and with

ES( )

1

F ( y)dy

0

1

Q( y)dy

0

Q( )

Q( ) F ( )

F ( ) inf(x | F(x) )

(4) Expected Shortfall

Regress the series of 99 quantiles to obtain the continuous function

Then, the expected shortfall estimates are

0

ˆ ˆ( )m

it i

i

Q a

2 1

0 1 20 00

1 1ˆ ˆ ˆ ˆ ˆ ˆ( ) ( ) ( ... )2 3

mmi

t t i mi

ES Q y dy a y dy a a a am

Forecasting in 3 steps

1. construct forecasts of conditional densities and of systemic risk indicators

2. use the VAR of static factors to compute dynamic forecasts k quarters ahead

3. use these forecasts are used to obtain recursive forecasts of quantile estimates

Forecast Evaluation 1

Density forecasts are satisfactory if the Probability Integral Transforms (PIT) based on estimated quantiles satisfies independence and uniformity

We constructed PITs for both our real activity and FS indicators for each of the seven countries

Properties broadly satisfied

Forecast Evaluation 2 Test based on Pearson’s Q est based on Pearson’s Q

statisticsstatistics Is the fraction of observed realizations of Is the fraction of observed realizations of

GDPG and FS close to the fractions GDPG and FS close to the fractions implied by estimated or forecast implied by estimated or forecast quantiles?quantiles?

In sample partitionsIn sample partitions [<Q5,Q5-Q10,Q10-Q20,>Q20] : [<Q5,Q5-Q10,Q10-Q20,>Q20] : left-tailleft-tail [<Q10,Q10-Q25,Q25-Q50,Q50-Q75,Q75-[<Q10,Q10-Q25,Q25-Q50,Q50-Q75,Q75-

Q90,>Q90]. Q90,>Q90]. entire distributionentire distribution Out –of- sample partition: Out –of- sample partition: [<Q20,>Q20] [<Q20,>Q20]

left tailleft tail BROADLY SATISFIEDBROADLY SATISFIED

Forecast EvaluationForecast EvaluationTable 4. Out-of–Sample Goodness of Fit

Each column reports the Q statistics corresponding to the forecast horizon k (in quarters). Significance of the Q- statistics at a 5 percent confidence level is reported in boldface.

GDPG FS k=1 k=2 k=3 k=4 k=1 k=2 k=3 k=4

U.S. 0.03 2.19 1.14 3.57 0.43 2.19 2.19 0.43 Canada 2.19 2.19 2.19 7.36 2.19 0.33 0.33 0.03 Japan 5.30 1.14 1.14 1.14 1.14 1.14 0.43 0.43 France 2.19 3.57 5.30 1.14 0.06 0.06 0.03 0.03

Germany 2.19 1.14 1.14 1.14 7.36 2.19 0.43 0.43 Italy 1.14 1.14 5.30 7.36 0.97 0.97 1.95 1.95 U.K. 2.19 0.03 0.06 1.14 2.19 0.43 0.43 0.43

Reduced-Form Stress Tests

A historical sequence of shocks to the distributions of GDP growth and the FS indicator is obtained by assuming that each quantile follows a AR(1) process

GDPGQ

t( ) a

R( ) b

R( )GDPGQ

t 1( )

tR ( )

FSQt( ) a

F( ) b

F( )GDPGQ

t 1( )

tF ( )

Reduced-Form Stress Tests Statistics

Stressed quantile series

Expected Shortfall ST deviations (ESSTDs)

STATISTICS: Average ESSTDs for each

SGDPGQ

T H ,t( ) GDPGQ

T H( )

tR ( )

, ( ) ( ) ( )FT H t T H tSFSQ FSQ

GDPES

t( ) SGDPGES

T H ,t( ) GDPES

t( )

FSES

t( ) SFSES

T H ,t( ) FSES

t( )

1...99

Average ESSTDs (2008Q1 and 2008Q2)

.4.6

.81

1.2

delta

ges

0 2 0 4 0 6 0 8 0 1 0 0q u a n t i l e

D e l t a R e a l E x p e c t e d S h o r t f a l l

46

810

12

delta

fes

0 2 0 4 0 6 0 8 0 1 0 0q u a n t i l e

D e l t a F i n a n c i a l E x p e c t e d S h o r t f a l l

U n i t e d S t a t e s : S t r e s s T e s t s 2 0 0 8 Q 1

.51

1.5

22.

5

delta

ges

0 2 0 4 0 6 0 8 0 1 0 0q u a n t i l e

D e l t a R e a l E x p e c t e d S h o r t f a l l

510

1520

2530

delta

fes

0 2 0 4 0 6 0 8 0 1 0 0q u a n t i l e

D e l t a F i n a n c i a l E x p e c t e d S h o r t f a l l

U n i t e d S t a t e s : S t r e s s T e s t s 2 0 0 8 Q 2

Structural Stress Testing At a given date, the At a given date, the sizesize of impulse of impulse

responses to identified shocks responses to identified shocks measures the measures the sensitivity of systemic risk indicators to these sensitivity of systemic risk indicators to these shocks. shocks.

Between dates, Between dates, changeschanges in the sizein the size of these of these impulse responses impulse responses provide a measure of provide a measure of changes in the changes in the resilienceresilience of an economy to these of an economy to these shocks. shocks.

The impulse responses of observable The impulse responses of observable variables can be variables can be used to detect which sectors of used to detect which sectors of the economy are most sensitive to a particular the economy are most sensitive to a particular shock (shock (risk mapsrisk maps).).

Theoretical Sign Theoretical Sign RestrictionsRestrictionsTable A. Responses of key variables to

positive shocks

Macroeconomic Model

Aggregate Supply Aggregate Demand

GDP growth Positive Positive

Inflation Negative Positive

Banking Model Bank Credit Demand

Bank Credit Supply

Bank Credit Growth Positive Positive

Change in Lending Rates

Positive Negative

IdentificationIdentification

InIn all all countries all identified shocks countries all identified shocks are are aggregateaggregate demand shocksdemand shocks associated with associated with bank credit bank credit demand shocksdemand shocks

Consistent with results in Canova Consistent with results in Canova and De Nicolò (JIE 2003) and De Nicolò (JIE 2003)

Slowdowns in aggregate bank credit Slowdowns in aggregate bank credit growth are the results of real growth are the results of real activity downturns (consistent with activity downturns (consistent with Berrospide and Edge, 2010)Berrospide and Edge, 2010)

A Simple Example of A Simple Example of Structural Stress TestStructural Stress Test

Gauge weather the stress test signals lower Gauge weather the stress test signals lower resilience to structural shocks in the G-7 resilience to structural shocks in the G-7 economies economies priorprior to 2007Q3 (pre-crisis)to 2007Q3 (pre-crisis)

Compute the Compute the differencedifference of the cumulative of the cumulative impact of the impulse response functions of impact of the impulse response functions of GDPESGDPES and and FSESFSES to each structural shock to each structural shock estimated for (1980Q1-2007Q2) and estimated for (1980Q1-2007Q2) and (1993Q2-2007Q2) (1993Q2-2007Q2)

A positive difference would indicate a lower A positive difference would indicate a lower resilience of the economies to these shocksresilience of the economies to these shocks

Results Results In all countries the first two shocks become

predominant in the last sub-period

Increased risk concentrations in these economies on both the real and financial sides

The U.S. economy had increased its vulnerability to shocks both on the real and financial sides, in absolute terms as well as relatively to the other G-7 economies

Modeling DevelopmentsModeling Developments

Extension of our framework to the simultaneous modeling of countries and regions of the world

Refinement of stress testing statistics and construction of risk maps