financial variables and the simulated out-of-sample forecastability of u.s. output growth since...

FINANCIAL VARIABLES AND THE SIMULATED OUT-OF-SAMPLEFORECASTABILITY OF U.S. OUTPUT GROWTH SINCE 1985:

AN ENCOMPASSING APPROACH

DAVID E. RAPACH and CHRISTIAN E. WEBER*

We reconsider the out-of-sample forecasting ability of a large number of financialvariables with respect to real output growth over the 1985:1–1999:4 period. We showthat models including financial variables display almost no forecasting ability relativeto an autoregressive benchmark model over this period according to a mean squaredforecast error metric. However, tests based on forecast encompassing indicate thatmany financial variables do, in fact, contain information that is useful for forecastingreal output growth over the 1985:1–1999:4 out-of-sample period. Our results suggestthat the extant literature exaggerates the demise of the forecasting power of financialvariables with respect to real activity since the mid-1980s. (JEL C22, C53, E44, E32)

I. INTRODUCTION

It is widely documented that the ability offinancial variables to forecast real outputgrowth has broken down—especially sincethe mid-1980s—according to a mean squaredforecast error (MSFE) criterion; see, for exam-ple, the recent study of Stock and Watson(2003). In this article, we use the concept offorecast encompassing to reexamine the out-of-sample forecasting ability of a large numberof financial variables with respect to U.S. realoutput growth over the 1985:1–1999:4 period.By focusing on forecast encompassing, thepresent work complements Stock and Watson(2003), Thoma and Gray (1998), and otherrecent studies that rely primarily on a relativeMSFE criterion to analyze the out-of-sampleforecasting ability of financial variables.

Forecast encompassing is closely relatedto the construction of optimal compositeforecasts.1 Consider two sets of out-of-sampleforecasts of real output growth, one from anautoregressive distributed lag (ARDL) modelthat includes a financial variable and one froma simple autoregressive (AR) benchmarkmodel, and consider forming an optimal com-posite forecast as a convex combination of theforecasts from the two models. If the optimalweight attached to the forecast fromtheARDLmodel is zero, then the ARDL model does notcontain information that is useful in the for-mation of the optimal composite forecast apartfrom the information already contained in theAR benchmark model. In this case, the AR

ABBREVIATIONS

AR: Autoregressive

ARDL: Autoregressive Distributed Lag

GDP: Gross Domestic Product

HAC: Heteroscedasticity and Autocorrelation

Consistent

MSFE: Mean Squared Forecast Error

OLS: Ordinary Least Squares

RMSFE: Root Mean Squared Forecast Error

1. Forecast emcompassing and the construction ofoptimal composite forecasts date back to Bates andGranger (1969), Nelson (1972), and Granger andRamanathan (1984). Also see Chong and Hendry (1986)and Fair and Shiller (1990). Textbook treatments can befound in Granger and Newbold (1986), Clements andHendry (1998), and Diebold (2001).

*This is a revisedversionofapaperpresentedat the July2002 meetings of the Western Economic Association, andwe are grateful to session participants, especially ToddClark, for helpful comments. We also thank MikeMcCracken, Mark Wohar, and two anonymous refereesfor very useful comments on earlier drafts. The usual dis-claimer applies. The results reported in this article weregenerated using GAUSS 3.6. The GAUSS programs areavailable online at http://pages.slu.edu/faculty/rapachde/Research.htm.

Rapach: Assistant Professor, Department of Economics,3674 Lindell Boulevard, Saint Louis University, SaintLouis, MO 63108-3397. Phone 1-314-977-3601, Fax1-314-977-1478, E-mail [email protected]

Weber: Associate Professor, Department of Economicsand Finance, 900 Broadway, Seattle University,Seattle, WA 98122-4340. Phone 1-206-296-5725, Fax1-206-296-2486, E-mail [email protected]

717

Economic Inquiry(ISSN 0095-2583) doi:10.1093/ei/cbh092Vol. 42, No. 4, October 2004, 717–738 # Western Economic Association International

http://pages.slu.edu/faculty/rapachde/

model forecasts encompass the ARDL modelforecasts. However, if the optimal weightattached to theARDLmodel forecast is greaterthan zero, then the ARDLmodel does containinformation useful for forecasting real outputgrowth apart from that already contained inthe AR benchmark model. Harvey et al.(1998) develop a test statistic for the nullhypothesis that the optimal weight attachedto one out-of-sample forecast is zero againstthe alternative hypothesis that the optimalweight is greater than zero. Clark andMcCracken (2001) develop a variant of theHarvey et al. (1998) statistic that accountsfor the parameter uncertainty inherent in theformation of forecasts and that Monte Carlosimulations show to be considerably morepowerful in detecting forecasting ability thanthe original statistic.

In our applications, we reconsider the fore-casting power of 10 of the financial variablesused in Stock and Watson (2003) with respecttoU.S. real gross domestic product (GDP) andindustrial production growth over the 1985:1–1999:4 out-of-sample period. These are 10of the most popular financial variables in theextant literature: M0, M1, M2, M3, federalfunds rate, 3-month Treasury bill rate, termspread, default spread, real stock prices, anddividend yield. For each financial variable, weconstruct recursive out-of-sample forecasts ofreal output growth over the 1985:1–1999:4period based on anARDLmodel that includesa given financial variable as an explanatoryvariable. Following Stock andWatson (2003),we consider forecasts of both real GDP andindustrial production growth. We use theHarvey and colleagues (1998) and Clark andMcCracken (2001) statistics to test the nullhypothesis that the out-of-sample forecastsof real output growth from an AR benchmarkmodel encompass the forecasts from theARDL model that includes a given financialvariable. As shown in Clark and McCracken(2001, 2004), there are a number of econo-metric issues that arise when comparing fore-casts from two nested models, as is obviouslythe case in our applications. Following therecommendations of Clark and McCracken(2004), we base our inferences on a bootstrapprocedure similar to the one in Kilian (1999).

Interestingly, we reject the null hypothesisthat the AR benchmark model forecastsencompass the ARDL model forecasts overthe 1985:1–1999:4 out-of-sample period for

a number of the financial variables at varioushorizons. The evidence of forecasting ability isespecially strong for industrial productiongrowth, where we are able to reject the nullhypothesis for 9 of the 10 financial variables,often at multiple horizons. This is true despitethe fact, that the ARDL model MSFE almostalways exceeds the AR benchmark modelMSFE (as in Stock andWatson 2003). Overall,our results suggest it is premature to concludethat financial variables no longer provide anyinformation that is useful for forecasting realactivity. When viewed through the lens offorecast encompassing, financial variables do,in fact, appear to contain some informationuseful for forecasting U.S. real output growthover the 1985:1–1999:4 out-of-sample period.Given thatwe consider a largenumberof finan-cial variables, we check the robustness of ourresults using a version of the Inoue and Kilian(2003) bootstrap procedure that explicitly con-trols for datamining. Even after we control fordata mining using this bootstrap procedure, anumber of financial variables continue to dis-play out-of-sample forecasting ability over the1985:1–1999:4 period according to encompass-ing tests.

Our forecast encompassing test results helpexplain a surprising result in Stock andWatson(2003). As indicated above, they find thatout-of-sample forecasts for the 1985:1–1999:4period generated by ARDL models thatinclude financial variables typically have anMSFE that is substantially larger than theMSFE from an AR benchmark model. How-ever, they also find that when these apparentlyinferior individual forecasts are combined—using, for example, simple averaging—theyproduce out-of-sample forecasts that arereliably more accurate in terms of relativeMSFE than the AR benchmark model fore-casts over the 1985:1–1999:4 out-of-sampleperiod. Because we find that several financialvariables individually contain useful informa-tion relative to an AR benchmark modelover the 1985:1–1999:4 out-of-sample periodaccording to our forecast encompassing tests,it is not surprising that combining these indi-vidual forecasts would generate forecastssuperior to the AR benchmark model.

Finally, we explore the role of structuralchange in explaining our findings. Stock andWatson (2003) find extensive evidence ofstructural breaks in ARDL equations forreal output growth over the postwar period,

718 ECONOMIC INQUIRY

and this can help explain the poor forecastingperformance of the financial variables accord-ing to the MSFE criterion over the 1985:1–1999:4 out-of-sample period. Like Stockand Watson (2003), we find evidence ofstructural breaks in a number of our ARDLequations,withbreaks oftenoccurringnear themid-1980s. However, it is not necessarily thecase that structural breaks have led to a com-plete breakdown in the ability of some financialvariables to forecast real activity. For example,there is an apparent structural break in therelationship between real stock price growthand industrial production growth in the mid-1980s, with the coefficients on real stock pricegrowth in an ARDL equation for industrialproduction growth shrinking noticeably—but still remaining significant—after thebreak. Following Clark and McCracken(2003), we conduct aMonte Carlo experiment,and the results indicate that forecast encom-passing tests aremuchmore powerful in detect-ing forecasting ability over the 1985:1–1999:4 out-of-sample period in this environ-ment than tests based on the relative MSFEcriterion.

The rest of the article is organizedas follows.Because some of the econometric methodo-logy employedhere is relativelynew,wediscussthe approaches to testing for forecasting abilityin some detail in section II. In section III, wepresent our forecasting test results. In sectionIV, we check whether the significant results insection III can be attributed to data mining,and in section V we explore structural breaksin the ARDL equations. Section VI containsconcluding remarks.

II. ECONOMETRIC METHODOLOGY

Let Dyt¼ yt� yt�1, where yt is the log-levelof real output at time t. In addition, letztþh ¼

Phi¼1 Dytþi.

2

Consider the following ARDL model:

ztþh ¼aþXq1�1

i¼0

biDyt�iþXq2�1

i¼0

g ixt�iþ etþh,ð1Þ

where h is the forecast horizon, xt is a financialvariable, and etþh is a disturbance term. Note

that the disturbance term is serially correlatedwhen h>1, as the ztþh observations will beoverlapping in this case. We are interested intesting whether xt has forecasting power withrespect to future output growth after control-ling for lagged output growth. Suppose thatafter allowing for lags, we have T observationsfor Dyt�i (i¼0, . . .q1�1) and xt� i (i¼0, . . .q2�1). This leaves us with T�h usable obser-vations for equation (1). It is straightforward toconduct an in-sample test of the forecastingability of xt by using all of the available obser-vations to conduct a Wald test of the nullhypothesis that g0¼ �� ¼gq2�1¼0. If we rejectthis null hypothesis, this is evidence that thefinancial variable xt has in-sample forecastingability with respect to future real outputgrowth. To account for the serial correlationin the disturbance term, a heteroscedasticityand autocorrelation–consistent (HAC) covari-ance matrix, such as that suggested by NeweyandWest (1987), should be used. Although weare primarily interested in out-of-sample testsin the present article, for the sake of compar-ison, we also report in-sampleWald test resultsfor the 10 financial variables we consider insection III.

Although there is considerable evidence thatfinancial variables have forecasting power forfuture U.S. real output growth according toin-sample tests, as discussed in the introduc-tion, out-of-sample forecasts beginningaround 1985 generally fare poorly in the extantliterature. The simulated out-of-sample fore-casting ability of a given financial variablewith respect to output growth can be assessedusing the following recursive scheme, which(apart from data revisions) simulates the situa-tion of a forecaster in real time. First, we dividethe total sample of T observations into in-sample and out-of-sample portions, where thein-sample observations span the first R obser-vations and the out-of-sample observationsthe last P observations. We compute out-of-sample forecasts from the unrestricted versionof equation (1) and also from a restrictedversion that excludes the financial variable(equation [1] with g0¼ . . . ¼ gq2�1¼ 0).

The first out-of-sample forecast for the un-restricted model is generated in the followingmanner. Estimate equation (1) via ordinaryleast squares (OLS) using data availablethrough period R. Using the OLS parameterestimates and the observations for xR�i

(i¼ 0, . . . q1� 1) and DyR� i (i¼ 0, . . . q2� 1),

2. Stock and Watson (2003) define ztþ h as ztþh ¼ð400=hÞ

Phi¼1 Dytþi. Note that this redefinition of ztþ h

has no effect on the statistics that we report in section III.

RAPACH & WEBER: FORECAST ENCOMPASSING 719

we construct a forecast for zRþh based onthe unrestricted model, zz1;Rþh, using the fittedequation zz1;Rþh ¼ aa1;R þ

Pq1�1i¼0 bb1;R;iDyR�i þPq2�1

i¼0 gg1;R;ixR�i, where aa1;R, bb1;R;iði ¼ 0, . . .q1 � 1Þ, and gg1;R;i ði ¼ 0, . . . q2 � 1Þ are theOLS estimates of a, bi (i ¼ 0, . . . q1 � 1),and g i (i¼ 0, . . . , q2 � 1), respectively, in equa-tion (1) using data available through period R.Denote the forecast error by uu1;Rþh ¼zRþh � zz1;Rþh. The initial forecast error corre-sponding to the restrictedmodel is generated ina similar manner, except that we set g0¼ . . . ¼gq2�1¼ 0 in equation (1). That is, we estimateequation (1) with g0 ¼ . . . ¼ gq2�1¼ 0 via OLSusing data available through period R to formthe forecast zz0;Rþh ¼ aa0;R þ

Pq1�1i¼0 bb0;R;iDyR�i,

where aa0;R and bb0;R;iði ¼ 0, . . . q1 � 1Þ are theOLS estimates of a and b (i¼ 0, . . . q1 � 1),respectively. Denote the forecast error corre-sponding to the restricted model as uu0;Rþh ¼zRþh � zz0;Rþh.

To generate a second set of forecasts, weupdate the procedure one period by usingdata available through period Rþ 1. That is,we estimate the unrestricted and restrictedmodels using data available through periodRþ 1, and we use these parameter estimatesand the observations for xRþ1�i (i¼ 0, . . .q1� 1) and DyRþ1�i (i¼ 0, . . . q2� 1) to formunrestricted and restricted model forecastsfor z(R þ 1)þ h and their respective forecasterrors, uu1;ðRþ1Þþh and uu0;ðRþ1Þþh. We repeatthis process through the end of the availablesample, leaving us with two sets of T�R�hþ 1 recursive out-of-sample forecast errors,one each for the unrestricted and restrictedregressionmodels (fuu1;tþhgT�h

t¼R andfuu0;tþhgT�ht¼R ,

respectively).The next step is to compare the simulated

out-of-sample forecasts from the unrestrictedandrestrictedmodels. If theunrestrictedARDLmodel forecasts are superior to the restrictedmodel forecasts, then the financial variable xtimproves the out-of-sample forecasts of ztþh

relative to an AR benchmark model. A simplemetric for comparing forecasts is Theil’s U,the ratio of the root mean squared forecasterror (RMSFE) for the unrestricted modelforecasts to the RMSFE for the restrictedmodel forecasts.3 Clearly, if the MSFE for

the unrestricted model forecasts is less thanthe MSFE for the restricted model forecasts,then U < 1. Stock and Watson (2003) reportthe ratio of the unrestricted model forecastMSFE to the restricted model forecast MSFE,so ourU is simply the square root of this ratio.4

We test whether the MSFE for the unre-stricted model forecasts is less than the MSFEfor the restricted model forecasts in a formalstatistical sense using theDieboldandMariano(1995) and West (1996) statistic, as well asa variant of this statistic due to McCracken(2004). Both statistics are based on the lossdifferential ddtþh ¼ uu20;tþh � uu21;tþh. Letting

�dd ¼ðT � R � h þ 1Þ�1 PT�h

t¼R ddtþh ¼ McSFSFE0�McSFSFE1 and SSdd ¼

PJj¼�J Kð j=JÞGGddð jÞ, where

McSFSFEi ¼ ðT � R � h þ 1Þ�1 PT�ht¼R uu2i;tþh

ði¼ 0,1Þ, GGddð jÞ¼ ðT�R�hþ1Þ�1PT�ht¼Rþj

ðddtþh� �ddÞðddtþh�j � �ddÞ, and GGddð�jÞ ¼ GGddð jÞ,the Diebold and Mariano (1995) and West(1996) statistic can be expressed as

MSE-T ¼ ðT � R� hþ 1Þ0:5 � �dd � SS�0:5dd :ð2Þ

Under the null hypothesis of equal forecastingability,MSFE0¼MSFE1, so that �dd andMSE-T are equal to zero.We test this null hypothesisagainst the one-sided (upper-tail) alternativehypothesis that the MSFE for the unrestrictedmodel forecasts is less than the MSFE for therestricted model forecasts (MSFE0>MSFE1),so that MSE-T> 0 under the alternativehypothesis. In our applications, we followClark and McCracken (2004) and use theBartlett kernel, K( j/J)¼ 1� [ j/(Jþ 1)], and weset J¼ [1.5h] for h> 1, where [�] is the nearest-integer function; forh¼ 1,weuse SSdd ¼ GGddð0Þ.West (1996) shows that the MSE-T statistic isdistributed asymptotically as a standard nor-mal variate under the null hypothesis of equalpredictive ability when comparing forecastsfrom nonnested models. However, for h¼ 1,McCracken (2004) shows that the MSE-Tstatistic has a nonstandard asymptotic distri-bution when comparing forecasts from nestedmodels, as is obviously the case for our appli-cations. In this case, the limiting distribution ofthe MSE-T statistic is a function of stochasticintegrals of quadratics of Brownian motion

3. Strictly speaking, Theil’s U uses a random walkmodel as a benchmark. In our applications, we use anAR model as the benchmark but still refer to the ratio ofthe RMSFEs from the unrestricted and restricted modelsas Theil’s U.

4. See the detailed results appendix to Stock andWatson (2003) available online at www.wws.princeton.edu/~mwatson.


that depends on limP;R!1P=R, as well as thenumber of excluded variables as wemove fromthe unrestricted to the restricted model (q2in our applications). Clark and McCracken(2004) show that the limiting distributionof theMSE-T statistic is also nonstandard forh> 1 when comparing forecasts from nestedmodels. However, the limiting distribution isnot free of nuisance parameters when h> 1, sothat the MSE-T statistic is not asymptoticallypivotal. Given this last result, Clark andMcCracken (2004) recommend basinginference on a bootstrap procedure along thelines of Kilian (1999).

The McCracken (2004) variant of theMSE-T statistic is given by

MSE-F ¼ ðT � R� k þ 1Þ � �dd=McSFSFE1:ð3Þ

When comparing forecasts fromnestedmodelsand for h¼ 1, McCracken (2004) shows thattheMSE-F statistic has a nonstandard limitingdistribution that is a functionof stochastic inte-grals of Brownian motion, whereas Clark andMcCracken (2004) demonstrate that theMSE-F statistic has a nonstandard asymptoticdistribution and is not asymptotically pivotalin the case of nested models and h> 1. Giventhis last result, Clark and McCracken (2004)again recommend basing inference on a boot-strap procedure along the lines of Kilian(1999). Following their recommendation, webase our inferences for the MSE-T andMSE-F statistics on the bootstrap proceduredescribed shortly.

Up to this point, we have described the useof the relative MSFE criterion to assess theforecasting power of a financial variable withrespect to real output growth. An alternativeway to judge forecasting ability is based onthe notion of forecast encompassing. Considerforming an optimal composite out-of-sampleforecast of ztþ h as a convex combination of theout-of-sample forecasts from the unrestrictedand restricted models:

zzc;tþh ¼ lzz1;tþh þ ð1� lÞzz0;tþh,ð4Þ

where 0� l� 1. If l¼ 0, the restrictedmodel forecasts are said to encompass theunrestricted model forecasts, because theunrestricted model does not contribute anyvaluable information—apart fromthatalready

contained in the restricted model—in the for-mation of an optimal composite forecast. Ifl> 0, then the restrictedmodel does contributeuseful information to the formation of anoptimal composite forecast. In this case, therestricted model forecasts do not encompassthe unrestricted model forecasts.

Harvey et al. (1998) develop a statistic thatcanbeused to test the null hypothesis thatl¼ 0in equation (4) against the one-sided (upper-tail) alternative hypothesis that l> 0:

ENC-T ¼ ðT � R� hþ 1Þ0:5 � �cc � SS�0:5cc ,ð5Þ

where cctþh¼ uu0;tþhðuu0;tþh� uu1;tþhÞ, �cc¼ðT�R�hþ1Þ�1PT�h

t¼R cctþh, SScc¼PJ

j¼�J Kð j=JÞGGccð jÞ,GGccð jÞ ¼ ðT � R � h þ 1Þ�1 PT�h

t¼Rþjðcctþh � �ccÞðcctþh�j � �ccÞ, and GGccð�jÞ ¼ GGccð jÞ. We againuse K( j/J)¼ 1� [ j/(Jþ 1)], J¼ [1.5h] for h> 1,and SSdd ¼ GGdd ð0Þ for h¼ 1. As pointed out byClark and McCracken (2004), the ENC-Tstatistic has a standard normal asymptoticdistribution when comparing forecasts fromnonnested models according to the theory inWest (1996). However, for nested models andh¼ 1, Clark and McCracken (2001) show thattheENC-T statistic has a nonstandard limitingdistribution. For h> 1, Clark and McCracken(2004) show that the ENC-T statistic has anonstandard asymptotic distribution and isnot asymptotically pivotal. Clark andMcCracken (2001) propose a variant of theENC-T statistic:

ENC-NEW ¼ðT�R�kþ1Þ ��cc=McSFSFE1:ð6Þ

Clark and McCracken (2001) show that theENC-NEWstatistic has a nonstandard asymp-totic distribution for h¼1, and their later work(2004) shows that it has a nonstandardasymptotic distribution and is not asympto-tically pivotal for h>1, when comparing fore-casts from nested models. Again, Clark andMcCracken (2004) recommend basing infer-ences for the ENC-T and ENC-NEW statisticson a bootstrap procedure, given that the stat-istics are not in general asymptotically pivotal.

The bootstrap procedure we employ is simi-lar to the one in Clark andMcCracken (2004),which is a version of the Kilian (1999) boot-strapprocedure.Weposit that the variablesDyt


and xt are generated by the following processunder the null hypothesis that the financialvariable xt has no forecasting power withrespect to real output growth:

Dyt ¼ a0 þXp1i¼1

aiDyt�i þ e1;t,ð7Þ

xt ¼ b0 þXp2i¼1

biDyt�i þXp3i¼1

cixt�i þ e2;t,ð8Þ

where the disturbance vector et¼ (e1,t, e2,t)0 is

independently and identically distributed withcovariance matrix S. We first estimateequations (7) and (8) via OLS using all of theavailable observations, with the lag orders (p1,p2, p3) selected using the SIC, and compute theOLS residuals, feet ¼ ðee1;t, ee2;tÞ0gTt¼1. To gener-ate a series of disturbances for our pseudo-sample, we randomly draw (with replacement)Tþ 50 times from the OLS residuals, giving usa pseudo-series of disturbance terms, fee�t gTþ50

t¼1.

Note that we draw from the OLS residualsin tandem, thereby preserving the contempora-neous correlation between the disturbancespresent in the original sample. Using fee�t g

Tþ50t¼1 ,

the OLS parameter estimates, equations (7)and (8), and setting the initial lagged observa-tions for Dyt and xt equal to zero, we can buildup a pseudo-sample of Tþ 50 observations forDyt and xt, fDy�t , x�t g

Tþ50t¼1 . We drop the first

50� p transient start-up observations, wherep¼max{p1, p2, p3}, to randomize the initiallagged Dyt and xt observations, leaving uswith a pseudo-sample of Tþ p observations,matching the size of the original sample(including the initial lagged variables). Forthe pseudo-sample, we calculate each of thefour out-of-sample test statistics alreadydescribed. We repeat this process 500 times,giving us an empirical distribution for eachof the test statistics. For each statistic, thep-value is the proportion of the bootstrapstatistics greater than the statistic computedusing the original sample.

Clark and McCracken (2001, 2004) pro-vide evidence on the finite-sample size andpower properties of the four out-of-samplestatistics (MSE-T, MSE-F, ENC-T, andENC-NEW) in extensive Monte Carlo simula-tions with nested models. They find that thefour statistics have good size properties when

inference is based on a bootstrap procedure.In addition, the simulations in Clark andMcCracken (2001, 2004) indicate that ENC-NEW is the most powerful statistic, followedby ENC-T and MSE-F. The least powerfulstatistic is theMSE-T. These rankings suggestthat the forecast encompassing statistics, espe-cially ENC-NEW, can have important poweradvantages over test statistics based on relativeMSFE. We will argue that these power rank-ings are important for understanding the out-of-sample forecasting test results reported insection III.5

The intuition for these rankings, and inparticular for the potential power gains associ-ated with the ENC statistics in comparison tothe MSE statistics, comes from the differentmetrics that lie at the heart of the two differentpairs of statistics. Recall that bothMSE statis-tics are based on �dd ¼ ðT � R� hþ 1Þ�1PT�h

t¼R ddtþh ¼ ðT �R� hþ 1Þ�1PT�ht¼R ðuu20;tþh�

uu21;tþhÞ, so that the MSE statistics comparedifferences in MSFEs. In contrast, the ENCstatistics are based on �cc¼ðT�R�hþ1Þ�1PT�h

t¼R cctþh ¼ ðT � R � h þ 1Þ�1 PT�ht¼R uu0;tþh

ðuu0;tþh� uu1;tþhÞ, so that they are based onthe difference between the variance of therestricted model forecast errors and the covar-iance between the restricted and unrestrictedmodel forecast errors. Suppose that therestricted and unrestricted model forecasterrors have similar variances but are approxi-mately uncorrelated. In this case, the �dd com-ponent of theMSE statistics will be near zero,so that theMSE statistics will likely be insignif-icant, as will any other test statistic based only

5. West (1997) develops a HAC covariance estimatorthat is appropriate for amoving-average process of knownorder. Because optimal forecasts at a horizon of h are amoving-average process of order h� 1, we could use theWest (1997) estimator in the calculation of SSdd ðSSccÞ forthe MSE-T (ENC-T) statistic. However, the West (1997)estimator would significantly increase computationalcosts, because it requires the estimation of an MA(h� 1)model and thus the use of a nonlinear optimization routine,and we would have to do this for every bootstrap replica-tion. Furthermore, at long horizons (large h), it may bedifficult for the optimization routine to achieve con-vergence. For these reasons, we do not employ the West(1997)HACcovariance estimator.Alsonote thatClarkandMcCracken (2004) find that the MSE-T and ENC-Tstatistics have good size properties inMonte Carlo simula-tions when SSdd and SScc are calculated using the Bartlettkernel and a lag truncation of [1.5h], as we do in this article.As shown inequations (3) and (6), themorepowerfulMSE-FandENC-NEW statistics donot requireHACcovarianceestimation, so that issues relating to lag truncation andkernel selection do not arise for these statistics.


on differences in error variances. However, the�cc component of the ENC statistics will be posi-tive in this case, so that the ENC statistics maywell be significant even though the MSE sta-tistics are insignificant. Intuitively, the signifi-cance of the ENC statistics reflects the factthat the restricted model forecast errors havelittle explanatory power for the unrestrictedmodel forecast errors (because by hypothesisPT�h

t¼R uu0;tþhuu1;tþh is approximately zero), sothat the unrestricted model forecasts mustcontain information not found in the restrictedmodel forecasts. That is, if the restrictedmodel forecast errors contain little informa-tion for predicting the unrestricted errors, therestricted model does not forecast encom-pass the unrestricted model, even if the twomodels yield forecast errors with very similarvariances.6

III. EMPIRICAL RESULTS

We use U.S. data from Stock and Watson(2003).7 The quarterly data for the 1959:1–1999:4 period include 2 output variables(real GDP and industrial production) and10 financial variables. The financial variablesare composed of four monetary aggregates,four interest rate variables, and two stockmar-ket variables. The monetary variables are themonetary base (M0), M1, M2, and M3. Theinterest rate variables are the Federal fundsrate, the 3-month Treasury bill rate, the termspread, and thedefault spread.The termspreadis calculated as the difference between theannualized returns on long-term governmentbonds (average maturity of roughly 20 years)and 3-monthTreasury bills. The default spreadis the difference in the yields on Moody’sseasoned Baa corporate bonds and Moody’sseasoned Aaa corporate bonds. The two stockmarket variables are real stock prices and thedividend yield. To work with variables thatappear to be stationary, real GDP, industrialproduction, the fourmonetary aggregates, andreal stock prices are measured in growth rates

(first differences of log-levels multiplied by400), and the Federal funds and 3-monthTreasury bill rates are measured in first differ-ences. Following Stock and Watson (2003),we consider two out-of-sample periods, 1971:1–1984:4 and 1985:1–1999:4, although we areprimarily interested in out-of-sample forecastsof real output growth over the 1985:1–1999:4period. We consider horizons of 1, 2, 4, and 8quarters (h¼ 1, 2, 4, 8); Stock and Watson(2003) consider horizons of 2, 4, and8quarters.Tables 1 and 2 report forecasting test resultswith real GDP growth appearing as thedependent variable in equation (1), whileTables 3 and 4 report the results that obtainwhen industrial production growth serves asthe dependent variable in equation (1). For the1971:1–1984:4 (1985:1–1999:4) out-of-sampleperiod, we use the SIC and data through1970:4 (1984:4) to determine the lag structureof equation (1). We consider values of q1 fromzero to eight. To ensure that the financialvariable appears in the unrestricted model,we consider values of q2 from one to eight.

Real GDP Growth Forecasts

Table 1 reports the 4 out-of-sample teststatistics for real GDP growth correspondingto each of the 10 financial variables at horizonsof 1, 2, 4, and 8 quarters. In addition, Table 1reports the values of q1 and q2 in equation (1)selected by the SIC, the in-sample Waldstatistic, and Theil’s U. For the Wald andfour out-of-sample statistics, p-values gener-ated through the bootstrap proceduredescribed in section II are given in parentheses.FromTable 1, we see that there is considerableevidence of in-sample forecastingpower for thefinancial variables over the full sample, as 9 ofthe 10 variables display significant predictiveability at some horizon according to the in-sample Wald statistic. However, if we concen-trate on the relative MSFE metric, we seeimportant discrepancies in out-of-sample fore-casting performance over the two out-of-sample periods. In general, U increases as wemove fromthe first to the secondout-of-sampleperiod, indicating the deterioration in forecastsof real output growth since 1985 emphasized inthe recent empirical literature. The greatest dis-crepancies in forecasting ability over the twoout-of-sample periods are for M1 growth (athorizons of 2 and8quarters), theFederal fundsrate (at horizons of 2 and 4 quarters), the term

6. The ENC statistics are also likely to be significantwhen the restricted and unrestrictedmodel forecasts errorshave a strong negative correlation. In this case, restrictedmodel forecast errors of a given sign are associated withunrestricted model forecast errors of the opposite sign,so that an optimal composite forecast should incorporateinformation from both the restricted and unrestrictedmodels.

7. The data are available from Mark Watson’s homepage at www.wws.princeton.edu/~mwatson.


TABLE1

ForecastingTestResults,RealGDPGrowth

Horizon(h):

1Quarter

Ahead

2QuartersAhead

4QuartersAhead

8QuartersAhead

Out-of-SamplePeriod

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

M0growth

q1

11

11

01

00

q2

11

11

11

15

Wald

0.63(0.55)

0.23(0.65)

0.24(0.73)

0.01(0.93)

0.29(0.70)

0.01(0.93)

1.77(0.49)

9.21(0.04)

U1.01

1.02

1.02

1.03

1.03

1.03

1.01

1.16

MSE-T

�1.23(0.74)

�1.38(0.76)

�1.84(0.86)

�1.18(0.71)

�1.56(0.78)

�1.18(0.65)

�0.46(0.36)

�1.84(0.89)

MSE-F

�1.01(0.56)

�2.60(0.89)

�2.35(0.66)

�3.12(0.89)

�3.21(0.67)

�2.83(0.74)

�1.22(0.40)

�13.77(0.96)

ENC-T

�0.98(0.78)

�0.93(0.70)

�1.63(0.89)

�0.99(0.73)

�1.32(0.81)

�0.96(0.68)

�0.09(0.44)

0.20(0.36)

ENC-N

EW

�0.40(0.71)

�0.84(0.90)

�1.02(0.83)

�1.27(0.93)

�1.33(0.80)

�1.12(0.82)

�0.12(0.45)

0.68(0.26)

M1growth

q1

01

01

00

00

q2

73

71

68

46

Wald

24.56(0.00)

6.17(0.04)

35.30(0.00)

1.37(0.36)

24.87(0.02)

6.15(0.07)

25.73(0.02)

5.00(0.13)

U1.03

1.19

0.96

1.13

1.00

1.88

0.89

1.62

MSE-T

�0.42(0.38)

�2.67(0.99)

0.30(0.16)

�2.15(0.97)

�0.05(0.22)

�2.06(0.94)

1.29(0.06)

�1.85(0.87)

MSE-F

�2.91(0.86)

�17.58(1.00)

4.12(0.01)

�13.06(1.00)

�0.51(0.38)

�40.85(1.00)

12.27(0.00)

�32.84(1.00)

ENC-T

1.55(0.06)

�0.54(0.57)

1.82(0.05)

�0.40(0.51)

1.03(0.15)

�1.54(0.88)

2.35(0.04)

�1.05(0.70)

ENC-N

EW

5.06(0.01)

�1.47(0.97)

11.68(0.00)

�0.87(0.87)

5.30(0.03)

�5.24(1.00)

12.27(0.02)

�3.73(0.96)

M2growth

q1

01

01

01

00

q2

32

21

11

11

Wald

15.22(0.00)

9.15(0.02)

27.99(0.00)

11.40(0.02)

26.95(0.01)

12.81(0.03)

10.91(0.14)

6.68(0.09)

U1.02

1.15

1.01

1.15

0.99

1.14

1.00

1.03

MSE-T

�0.50(0.38)

�1.84(0.91)

�0.08(0.21)

�1.19(0.69)

0.27(0.15)

�0.97(0.58)

0.03(0.22)

�0.22(0.33)

MSE-F

�2.54(0.84)

�14.68(1.00)

�0.60(0.27)

�14.04(1.00)

1.28(0.12)

�13.32(0.99)

0.10(0.21)

�3.14(0.69)

ENC-T

1.98(0.04)

0.68(0.23)

2.62(0.01)

0.99(0.15)

2.56(0.02)

1.46(0.10)

1.34(0.16)

1.27(0.15)

ENC-N

EW

4.10(0.03)

2.48(0.03)

7.18(0.02)

5.27(0.01)

7.32(0.04)

8.65(0.01)

1.81(0.21)

8.22(0.03)


M3growth

q1

01

01

01

00

q2

31

21

21

11

Wald

4.41(0.11)

1.71(0.20)

4.52(0.19)

3.47(0.16)

2.15(0.37)

2.04(0.30)

0.00(0.99)

0.96(0.50)

U1.08

0.99

1.09

0.99

1.13

0.95

1.07

1.01

MSE-T

�1.71(0.90)

0.45(0.10)

�1.65(0.82)

0.13(0.23)

�2.22(0.94)

0.53(0.15)

�1.46(0.71)

�0.38(0.39)

MSE-F

�7.80(0.98)

1.80(0.04)

�8.76(0.93)

1.40(0.13)

�11.20(0.90)

5.88(0.04)

�6.46(0.72)

�0.72(0.39)

ENC-T

�0.36(0.59)

1.33(0.06)

�0.09(0.40)

1.26(0.12)

�0.36(0.51)

1.60(0.09)

�1.11(0.71)

2.13(0.06)

ENC-N

EW

�0.64(0.84)

2.75(0.02)

�0.18(0.43)

6.86(0.01)

�0.97(0.70)

8.54(0.02)

�2.08(0.79)

1.26(0.25)

Federalfundsrate,firstdifferences

q1

11

01

00

00

q2

22

55

46

15

Wald

14.44(0.01)

16.70(0.00)

41.04(0.00)

43.75(0.00)

40.90(0.00)

36.57(0.00)

3.94(0.24)

41.77(0.00)

U1.01

1.06

0.99

1.21

0.98

1.34

1.10

1.24

MSE-T

�0.13(0.26)

�2.48(1.00)

0.08(0.23)

�2.37(0.99)

0.13(0.20)

�2.17(0.97)

�0.76(0.47)

�1.41(0.80)

MSE-F

�0.89(0.56)

�6.93(0.99)

1.00(0.07)

�19.02(1.00)

1.98(0.02)

�25.39(1.00)

�8.70(0.85)

�18.33(0.98)

ENC-T

1.65(0.05)

�1.58(0.92)

3.32(0.00)

�0.61(0.61)

3.59(0.00)

�0.63(0.65)

3.09(0.00)

0.25(0.37)

ENC-N

EW

7.15(0.01)

�1.71(0.99)

19.74(0.00)

�2.42(1.00)

22.30(0.00)

�2.49(0.99)

6.81(0.04)

1.23(0.06)

3-m

onth

Treasury

billrate,firstdifferences

q1

01

10

10

00

q2

12

48

47

16

Wald

3.83(0.12)

9.29(0.02)

32.15(0.00)

35.29(0.00)

51.42(0.00)

32.53(0.00)

4.26(0.16)

31.40(0.00)

U1.04

1.04

1.14

1.40

1.08

1.47

1.11

1.33

MSE-T

�1.12(0.67)

�1.93(0.95)

�0.95(0.62)

�2.31(0.99)

�0.40(0.42)

�2.10(0.97)

�0.85(0.52)

�1.50(0.85)

MSE-F

�4.23(0.92)

�4.46(0.98)

�12.79(0.99)

�28.76(1.00)

�7.80(0.94)

�30.77(1.00)

�9.29(0.94)

�23.10(0.99)

ENC-T

�0.59(0.62)

�1.05(0.79)

2.20(0.01)

�0.18(0.47)

3.32(0.00)

0.00(0.50)

2.41(0.01)

0.68(0.28)

ENC-N

EW

�0.90(0.90)

�1.04(0.95)

7.17(0.01)

�0.66(0.90)

18.94(0.00)

0.01(0.48)

5.08(0.03)

3.76(0.02)

Term

spread

q1

00

00

00

00

q2

22

12

12

81

Wald

53.78(0.00)

47.46(0.00)

37.62(0.00)

35.37(0.00)

53.63(0.00)

32.72(0.00)

187.58(0.00)

31.55(0.00)

U0.79

1.29

0.74

1.45

0.67

1.50

1.02

1.31

MSE-T

2.60(0.00)

�3.21(1.00)

2.22(0.00)

�2.66(0.99)

2.54(0.00)

�2.09(0.94)

�0.13(0.22)

�1.52(0.79)

MSE-F

33.28(0.00)

�23.86(1.00)

44.41(0.00)

�30.87(1.00)

66.19(0.00)

�31.83(1.00)

�1.57(0.34)

�22.18(0.99)

ENC-T

4.25(0.00)

0.40(0.30)

3.70(0.00)

0.41(0.30)

3.54(0.00)

0.51(0.27)

2.56(0.05)

0.64(0.27)

ENC-N

EW

38.73(0.00)

1.30(0.07)

54.26(0.00)

1.32(0.16)

82.98(0.00)

1.78(0.15)

21.50(0.01)

2.40(0.16)

Defaultspread

q1

00

01

40

00

q2

12

21

12

21

Wald

2.58(0.27)

23.52(0.00)

16.13(0.01)

0.08(0.85)

0.22(0.80)

9.23(0.05)

1.75(0.59)

0.00(0.99)

U1.09

1.02

1.10

1.00

1.24

1.03

1.15

1.00

MSE-T

�1.24(0.70)

�0.29(0.33)

�1.30(0.65)

�0.09(0.28)

�1.73(0.81)

�0.98(0.55)

�2.54(0.93)

�1.01(0.58)

MSE-F

�9.03(0.98)

�2.30(0.88)

�9.29(0.90)

�0.05(0.27)

�18.72(0.90)

�3.27(0.64)

�11.98(0.66)

�0.37(0.33)

ENC-T

�0.34(0.56)

1.18(0.11)

0.71(0.21)

0.00(0.41)

�1.61(0.90)

0.46(0.27)

�3.23(0.97)

�0.97(0.68)

ENC-N

EW

�0.90(0.88)

4.25(0.01)

1.48(0.21)

0.00(0.41)

�2.87(0.84)

0.81(0.26)

�3.33(0.78)

�0.17(0.46)

continued


TABLE1continued

Horizon(h):

1Quarter

Ahead

2QuartersAhead

4QuartersAhead

8QuartersAhead

Out-of-SamplePeriod

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

Realstock

price

growth

q1

00

00

00

00

q2

34

34

22

11

Wald

32.30(0.00)

37.54(0.00)

47.67(0.00)

29.51(0.00)

22.28(0.01)

17.21(0.01)

10.06(0.08)

8.79(0.04)

U0.89

1.24

0.83

1.31

0.88

1.18

0.97

1.04

MSE-T

1.80(0.00)

�2.21(0.97)

1.70(0.00)

�2.15(0.97)

1.59(0.01)

�1.68(0.88)

1.08(0.08)

�0.86(0.56)

MSE-F

14.17(0.00)

�21.06(1.00)

24.36(0.00)

�24.71(1.00)

15.79(0.00)

�15.86(0.99)

2.85(0.03)

�4.30(0.93)

ENC-T

2.90(0.00)

1.04(0.13)

2.29(0.01)

1.26(0.10)

2.20(0.02)

1.13(0.13)

1.67(0.09)

0.62(0.28)

ENC-N

EW

15.57(0.00)

4.17(0.00)

24.34(0.00)

5.42(0.01)

15.31(0.00)

3.68(0.02)

2.52(0.06)

1.19(0.07)

Dividendyield

q1

00

00

00

00

q2

13

13

13

11

Wald

11.82(0.01)

24.58(0.00)

7.03(0.10)

16.69(0.00)

3.32(0.34)

13.93(0.02)

1.03(0.69)

1.01(0.49)

U1.02

1.29

1.04

1.31

1.04

1.19

1.06

1.03

MSE-T

�0.18(0.27)

�2.63(0.99)

�0.32(0.25)

�2.98(0.99)

�0.48(0.26)

�2.55(0.96)

�1.61(0.73)

�0.47(0.32)

MSE-F

�1.69(0.62)

�24.01(1.00)

�4.05(0.68)

�24.69(1.00)

�3.93(0.42)

�16.78(0.96)

�5.22(0.38)

�3.21(0.38)

ENC-T

1.88(0.04)

0.81(0.17)

1.30(0.10)

1.54(0.07)

0.91(0.17)

1.50(0.09)

�1.35(0.81)

0.18(0.35)

ENC-N

EW

11.44(0.00)

2.74(0.05)

11.75(0.01)

6.46(0.03)

3.66(0.15)

5.43(0.09)

�1.82(0.62)

0.76(0.33)

Notes:

q1andq2are

thelagsoftheARDLequation.Wald

isthein-sample

F-statistic

usedto

test

thenullhypothesisthatthevariable

Granger-causesindustrialproduction

growth;Wald

statistic

iscalculatedusingdata

from

1959:2

throughtheendoftheout-of-sample

period.U

istheratiooftheunrestricted

model

out-of-sample

RMSFE

tothe

restricted

model

out-of-sample

RMSFE.TheMSE-T

andMSE-F

statisticsare

usedto

test

thenullhypothesisthattheunrestricted

model

out-of-sample

MSFEisequalto

the

restricted

model

out-of-sample

MSFE

against

theone-sided

(upper-tail)hypothesisthattheunrestricted

model

out-of-sample

MSFE

islower

thantherestricted

model

out-of-

sampleMSFE.TheENC-T

andENC-N

EW

statisticsare

usedto

testthenullhypothesisthattherestricted

modelout-of-sampleforecastsencompass

theunrestricted

modelout-of-

sampleforecastsagainst

theone-sided

(upper-tail)hypothesisthattherestricted

modelout-of-sampleforecastsdonotencompass

theunrestricted

model

out-of-sampleforecasts.

Bootstrapped

p-values

are

given

inparentheses;0.00signifies<0.005.Bold

entriesindicate

significance

atthe10%

level

accordingto

thebootstrapped

p-value.


spread (at horizons of 1, 2, and 4 quarters), andreal stock price growth (at all reported hori-zons). In these instances, U is often wellbelow unity for the 1971:1–1984:4 out-of-sample period (indicating that the ARDLmodel that includes the financial variablehas a lower MSFE than the AR benchmarkmodel), whereas U is often well above unityfor the 1985:1–1999:4 out-of-sample period.For example, at the two-quarters-ahead hori-zon, U is 0.83 for real stock price growth overthe 1971:1–1984:4 out-of-sample period, but1.31 over the 1985:1–1999:4 out-of-sampleperiod. Observe that U� 1 for every financialvariable at every horizon over the 1985:1–1999:4 out-of-sample period, with the excep-tionsofM3growthathorizonsofone, two, andfour quarters.

Given that almost all of the Umeasures aregreater than unity, it is not surprising that thereare very few rejections of the null hypothesis of

equal forecasting ability against the alternativeof a lower MSFE for the unrestricted modelaccording to the MSE-T or MSE-F statisticsover the 1985:1–1999:4 out-of-sample period.If anything, the large p-values for a number oftheMSE-T andMSE-F statistics indicate thatwe would reject the null hypothesis in favor ofthealternative that theunrestrictedmodel fore-cast MSFE is greater than the restricted modelforecast MSFE. For the three cases whereU< 1 (M3 growth at horizons of one, two,and fourquarters), theunrestrictedmodel fore-castMSFEappears to be significantly less thanthe restrictedmodel forecastMSFE at conven-tional significance levels according to at leastone of the two statistics at horizons of one andfour quarters. Neither of the two statistics issignificant at the two-quarters-ahead horizon.To reiterate, there is hardly any evidence ofout-of-sample forecasting ability over the1985:1–1999:4 period for any of the financial

TABLE 2

Least Squares Estimates of the Weight Attached to the Unrestricted Model Forecast in an

Optimal Composite Forecast, Real GDP Growth, 1985:1–1999:4 Out-of-Sample Period

Horizon (h) 1 Quarter Ahead 2 Quarters Ahead 4 Quarters Ahead 8 Quarters Ahead

M2 growth

l 0.1345 0.1642 0.2978 0.1988

U 0.9965 0.9865 0.9715 0.9554

M3 growth

l 0.7984 0.9119

U 0.9597 0.9437

Federal funds rate, first differences

l 0.2175

U 0.9399

3-month Treasury bill rate, first differences

l 0.2652

U 0.9207

Term spread

l 0.1321

U 0.9925

Default spread

l 0.4649

U 0.9779

Real stock price growth

l 0.2892 0.3328 0.3284 0.4557

U 0.9776 0.9550 0.9597 0.9458

Dividend yield

l 0.2790 0.4122 0.4976

U 0.9787 0.9370 0.9254

Notes: l is the estimated weight attached to the unrestricted model out-of-sample forecast in an optimal compositeout-of-sample forecast; the weight is estimated using a regression model with an intercept term. U is the ratio of theoptimal composite out-of-sample RMSFE to the restricted model out-of-sample RMSFE.


TABLE3

ForecastingTestResults,IndustrialProductionGrowth

Horizon(h):

1Quarter

Ahead

2QuartersAhead

4QuartersAhead

8QuartersAhead

Out-of-SamplePeriod

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

M0growth

q1

11

11

01

00

q2

11

11

11

14

Wald

0.92(0.51)

0.47(0.51)

0.38(0.63)

0.23(0.66)

0.54(0.62)

0.20(0.70)

0.62(0.66)

6.84(0.10)

U1.01

1.04

1.02

1.04

1.05

1.04

1.04

1.26

MSE-T

�0.71(0.50)

�1.41(0.79)

�0.85(0.45)

�0.84(0.55)

�0.85(0.44)

�0.94(0.52)

�1.13(0.58)

�3.01(0.99)

MSE-F

�0.88(0.54)

�4.50(0.96)

�1.97(0.59)

�4.02(0.93)

�4.54(0.72)

�4.48(0.82)

�3.27(0.63)

�19.67(0.98)

ENC-T

�0.31(0.56)

�0.62(0.58)

�0.48(0.52)

�0.43(0.54)

�0.45(0.51)

�0.44(0.48)

�0.96(0.69)

�1.04(0.71)

ENC-N

EW

�0.20(0.56)

�0.97(0.96)

�0.55(0.60)

�1.01(0.92)

�1.18(0.76)

�1.03(0.77)

�1.34(0.75)

�2.85(0.94)

M1growth

q1

01

01

00

00

q2

71

78

78

57

Wald

30.82(0.00)

2.69(0.14)

42.80(0.00)

9.88(0.02)

53.89(0.00)

7.74(0.05)

32.52(0.03)

5.75(0.14)

U0.99

1.16

0.92

2.10

0.91

2.29

0.86

2.03

MSE-T

0.24(0.13)

�2.03(0.97)

0.94(0.04)

�2.68(0.99)

0.89(0.08)

�2.48(0.97)

1.41(0.06)

�2.73(0.97)

MSE-F

1.41(0.04)

�15.16(1.00)

10.39(0.01)

�45.63(1.00)

11.22(0.01)

�46.13(1.00)

16.79(0.02)

�40.11(1.00)

ENC-T

2.93(0.00)

0.66(0.19)

2.82(0.0.01)

�1.50(0.89)

2.20(0.04)

�1.54(0.88)

2.54(0.04)

�0.66(0.61)

ENC-N

EW

9.23(0.00)

2.42(0.02)

16.25(0.00)

�3.67(1.00)

15.12(0.01)

�3.41(0.98)

17.35(0.03)

�1.78(0.83)

M2growth

q1

01

01

01

00

q2

31

21

11

11

Wald

33.49(0.00)

8.50(0.02)

36.57(0.00)

13.28(0.01)

39.44(0.01)

9.64(0.04)

24.80(0.05)

3.60(0.23)

U0.98

1.26

0.95

1.46

0.93

1.59

0.96

1.38

MSE-T

0.38(0.15)

�2.54(0.99)

0.66(0.10)

�2.16(0.96)

1.09(0.07)

�1.88(0.88)

0.92(0.08)

�1.16(0.63)

MSE-F

2.53(0.05)

�21.93(1.00)

6.32(0.01)

�31.30(1.00)

8.19(0.03)

�34.42(1.00)

4.28(0.08)

�25.05(0.99)

ENC-T

3.40(0.00)

0.69(0.19)

3.08(0.00)

0.64(0.23)

2.66(0.03)

0.43(0.33)

2.22(0.06)

�0.01(0.44)

ENC-N

EW

12.46(0.00)

2.99(0.02)

16.97(0.00)

3.70(0.02)

17.86(0.01)

2.86(0.10)

6.99(0.07)

�0.10(0.46)

M3growth

q1

01

01

01

00

q2

31

21

21

11

Wald

8.74(0.04)

3.57(0.10)

11.34(0.05)

5.88(0.07)

5.72(0.18)

2.32(0.28)

0.68(0.64)

0.48(0.62)


U1.10

1.09

1.09

1.16

1.12

1.14

1.09

1.09

MSE-T

�2.10(0.96)

�1.02(0.65)

�1.40(0.74)

�1.08(0.60)

�2.07(0.93)

�0.65(0.44)

�1.54(0.74)

�0.57(0.42)

MSE-F

�9.32(0.98)

�9.04(0.99)

�8.36(0.92)

�15.13(1.00)

�11.05(0.89)

�13.27(0.96)

�7.87(0.78)

�8.40(0.84)

ENC-T

0.36(0.32)

0.93(0.14)

0.76(0.21)

0.93(0.13)

0.38(0.30)

0.70(0.23)

�0.55(0.57)

0.01(0.41)

ENC-N

EW

0.59(0.24)

4.31(0.01)

1.94(0.14)

6.50(0.01)

1.72(0.19)

6.47(0.04)

�1.45(0.72)

0.07(0.40)

Federalfundsrate,firstdifferences

q1

11

01

00

00

q2

22

54

38

36

Wald

16.20(0.01)

22.94(0.00)

33.54(0.01)

42.70(0.00)

24.58(0.02)

45.14(0.00)

37.15(0.01)

51.79(0.00)

U1.02

1.01

1.16

1.12

1.04

1.34

0.94

1.22

MSE-T

�0.30(0.35)

�0.20(0.35)

�1.29(0.80)

�1.21(0.71)

�0.28(0.48)

�1.70(0.93)

0.30(0.25)

�1.31(0.79)

MSE-F

�2.43(0.85)

�1.12(0.74)

�14.38(0.98)

�11.93(1.00)

�3.62(0.87)

�25.42(1.00)

6.02(0.01)

�17.60(0.99)

ENC-T

1.89(0.06)

1.45(0.07)

2.83(0.01)

0.68(0.23)

3.33(0.01)

0.73(0.35)

5.18(0.00)

0.88(0.30)

ENC-N

EW

8.11(0.01)

3.47(0.01)

9.42(0.01)

3.02(0.01)

13.33(0.01)

3.79(0.01)

29.18(0.00)

5.18(0.02)

3-m

onth

Treasury

billrate,firstdifferences

q1

11

11

10

00

q2

42

47

47

16

Wald

22.20(0.00)

15.41(0.00)

46.93(0.00)

46.31(0.00)

66.49(0.00)

33.75(0.00)

4.43(0.16)

28.46(0.00)

U1.05

1.02

1.02

1.37

0.99

1.47

1.14

1.30

MSE-T

�0.55(0.43)

�0.55(0.44)

�0.37(0.43)

�2.07(0.98)

0.05(0.37)

�1.83(0.95)

�0.87(0.60)

�1.40(0.86)

MSE-F

�5.22(0.95)

�2.30(0.93)

�2.45(0.88)

�27.62(1.00)

0.84(0.11)

�30.55(1.00)

�11.35(0.95)

�21.80(1.00)

ENC-T

2.30(0.02)

1.02(0.11)

3.13(0.00)

0.30(0.34)

4.83(0.00)

0.78(0.29)

2.81(0.01)

1.15(0.24)

ENC-N

EW

11.62(0.00)

1.94(0.02)

12.54(0.00)

1.29(0.05)

30.88(0.00)

4.09(0.01)

5.94(0.02)

7.46(0.00)

Term

spread

q1

01

00

00

00

q2

22

22

12

81

Wald

50.82(0.00)

31.16(0.00)

55.31(0.00)

31.28(0.00)

39.29(0.01)

33.03(0.00)

318.80(0.00)

46.25(0.01)

U0.83

1.27

0.75

1.40

00.64

1.50

0.89

1.24

MSE-T

1.44(0.04)

�2.46(0.97)

1.87(0.00)

�1.79(0.89)

2.30(0.01)

�1.64(0.83)

0.78(0.10)

�1.19(0.67)

MSE-F

25.00(0.00)

�23.08(1.00)

43.18(0.00)

�29.04(1.00)

75.57(0.00)

�31.61(1.00)

12.41(0.02)

�18.75(0.98)

ENC-T

3.70(0.00)

0.92(0.16)

3.58(0.00)

1.18(0.16)

3.64(0.01)

0.88(0.22)

2.84(0.02)

1.35(0.17)

ENC-N

EW

38.38(0.00)

3.47(0.01)

55.89(0.00)

5.77(0.02)

98.07(0.00)

5.33(0.04)

41.30(0.00)

8.89(0.04)

Defaultspread

q1

01

01

01

00

q2

32

22

21

21

Wald

38.02(0.00)

21.72(0.01)

12.32(0.07)

9.18(0.05)

12.84(0.11)

0.08(0.86)

2.56(0.54)

0.05(0.90)

U1.04

0.99

1.08

1.03

1.15

1.02

1.19

1.00

MSE-T

�0.23(0.34)

0.20(0.22)

�0.73(0.41)

�0.57(0.43)

�1.60(0.73)

�0.97(0.55)

�2.11(0.88)

0.02(0.27)

MSE-F

�3.75(0.84)

1.46(0.07)

�8.00(0.88)

�3.04(0.83)

�12.88(0.79)

�2.35(0.55)

�14.19(0.62)

0.00(0.28)

ENC-T

2.31(0.06)

2.21(0.02)

1.83(0.08)

1.15(0.16)

0.49(0.32)

�0.79(0.65)

�1.62(0.86)

0.06(0.41)

ENC-N

EW

19.23(0.00)

10.44(0.00)

10.62(0.03)

3.47(0.08)

1.52(0.28)

�0.93(0.66)

�4.08(0.77)

0.01(0.42)

Realstock

price

growth

q1

11

10

00

00

q2

22

24

23

11

continued


TABLE3continued

Horizon(h):

1Quarter

Ahead

2QuartersAhead

4QuartersAhead

8QuartersAhead

Out-of-SamplePeriod

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

1971:1–1984:4

1985:1–1999:4

Wald

39.66(0.00)

41.45(0.00)

43.38(0.00)

29.25(0.00)

35.17(0.01)

24.43(0.00)

18.28(0.02)

12.56(0.01)

U0.84

1.28

0.76

1.58

0.79

1.43

0.95

1.08

MSE-T

2.17(0.00)

�1.92(0.96)

1.79(0.00)

�2.37(0.97)

1.96(0.01)

�2.34(0.97)

1.24(0.05)

�0.98(0.63)

MSE-F

23.81(0.00)

�23.41(1.00)

40.68(0.00)

�35.30(1.00)

32.65(0.00)

�29.02(1.00)

5.54(0.01)

�7.42(0.98)

ENC-T

2.96(0.00)

1.34(0.08)

2.25(0.00)

0.61(0.22)

2.34(0.02)

0.60(0.25)

1.81(0.06)

0.92(0.22)

ENC-N

EW

20.87(0.00)

6.62(0.00)

34.89(0.00)

3.58(0.01)

28.33(0.00)

2.84(0.03)

4.88(0.03)

2.55(0.03)

Dividendyield

q1

01

01

01

00

q2

13

13

13

12

Wald

16.82(0.01)

38.63(0.00)

9.56(0.08)

16.86(0.00)

6.36(0.19)

18.41(0.02)

1.84(0.61)

20.43(0.03)

U1.00

1.36

1.01

1.48

1.04

1.42

1.01

1.08

MSE-T

0.03(0.21)

-2.43(0.98)

�0.06(0.20)

�2.40(0.96)

�0.27(0.23)

�2.50(0.96)

�0.56(0.28)

�0.71(0.42)

MSE-F

0.48(0.16)

�27.65(1.00)

�1.18(0.30)

�32.12(1.00)

�3.59(0.44)

�28.88(0.99)

�1.31(0.23)

�7.52(0.64)

ENC-T

1.77(0.08)

1.73(0.04)

1.31(0.11)

1.58(0.08)

1.27(0.14)

1.75(0.08)

0.24(0.38)

1.19(0.20)

ENC-N

EW

17.25(0.00)

7.80(0.00)

19.44(0.00)

8.93(0.01)

9.03(0.09)

6.69(0.08)

0.27(0.41)

7.42(0.15)

Notes:

q1andq2are

thelagsoftheARDLequation.Wald

isthein-sample

F-statistic

usedto

test

thenullhypothesisthatthevariable

Granger-causesindustrialproduction

growth;Wald

statistic

iscalculatedusingdata

from

1959:2

throughtheendoftheout-of-sample

period.U

istheratiooftheunrestricted

model

out-of-sample

RMSFE

tothe

restricted

model

out-of-sample

RMSFE.TheMSE-T

andMSE-F

statisticsare

usedto

test

thenullhypothesisthattheunrestricted

model

out-of-sample

MSFEisequalto

the

restricted

model

out-of-sample

MSFE

against

theone-sided

(upper-tail)hypothesis

thattheunrestricted

model

out-of-sample

MSFE

islower

thantherestricted

model

out-of-

sampleMSFE.TheENC-T

andENC-N

EW

statisticsare

usedto

testthenullhypothesisthattherestricted

modelout-of-sampleforecastsencompass

theunrestricted

modelout-of-

sample

forecastsagainst

theone-sided

(upper-tail)hypothesisthattherestricted

modelout-of-sampleforecastsdonotencompass

theunrestricted

model

out-of-sample

forecasts.

Bootstrapped

p-values

are

given

inparentheses;0.00signifies<0.005.Bold

entriesindicate

significance

atthe10%

level

accordingto

thebootstrapped

p-value.


variables according to a relative MSFE metricin Table 1.

Things are strikingly different when weexamine the encompassing statistics for the1985:1–1999:4 out-of-sample period inTable 1. We reject the null hypothesis thatthe restricted model forecasts encompass theunrestricted model forecasts for 8 of the 10financial variables at some horizon accordingto the ENC-T and/or ENC-NEW statistics.The only financial variables that evince nosignificant out-of-sample forecasting abilityaccording to the encompassing statistics areM0 and M1 growth. M2, M3, real stockprice growth, and the dividend yield producethe most extensive evidence of out-of-sample

forecasting ability over the 1985:1–1999:4period, with the ENC-T and/or ENC-NEWstatistics significant at three or four of theforecast horizons considered.8 Table 1makes an important methodological point:For some of the financial variables, includingM2 growth and all of the interest rate vari-ables, only the ENC-NEW statistic indicatesthat the financial variable in questionhelps forecast real GDP growth over the

TABLE 4

Least Squares Estimates of the Weight Attached to the Unrestricted Model Forecast

in an Optimal Composite Forecast, Industrial Production Growth, 1985:1–1999:4

Out-Of-Sample Period


M1 growth

l 0.1224

U 0.9945

M2 growth

l 0.2573 0.1858 0.1104

U 0.9830 0.9850 0.9936

M3 growth

l 0.5805 0.4646 0.4277

U 0.9651 0.9607 0.9694

Federal funds rate, first differences

l 0.4694 0.1994 0.1821 0.2653

U 0.9858 0.9928 0.9730 0.9666

3-month Treasury bill rate, first differences

l 0.3445 0.0964 0.1678 0.2797

U 0.9940 0.9891 0.9670 0.9502

Term spread

l 0.1750 0.2231 0.2213 0.4604

U 0.9912 0.9753 0.9699 0.9244

Default spread

l 0.5587 0.3651

U 0.9510 0.9878

Real stock price growth

l 0.2365 0.1717 0.1665 0.2573

U 0.9777 0.9810 0.9838 0.9896

Dividend yield

l 0.2997 0.2980 0.2801

U 0.9618 0.9461 0.9554

Notes: l is the estimated weight attached to the unrestricted model out-of-sample forecast in an optimal compositeout-of-sample forecast; the weight is estimated using a regression model with an intercept term. U is the ratio of theoptimal composite out-of-sample RMSFE to the restricted model out-of-sample RMSFE.

8. We found evidence of ARCH in the residuals ofequation (8) of the bootstrap procedure for some of thefinancial variables. For these financial variables, we com-puted p-values for the out-of-sample statistics reported inTable 1 using a wild bootstrap procedure that is robust toARCH. This has little qualitative effect on our inferences.


1985:1–1999:4 out-of-sample period. This isconsistent with the Monte Carlo results ofClark and McCracken (2001, 2004), discussedalready, which indicate that the ENC-NEWstatistic is more powerful than the otherout-of-sample statistics.

Our results to this point indicate that for anumber of the financial variables consideredhere,we can reject the null hypothesis that fore-casts of realGDPgrowth generatedby a simpleAR model encompass forecasts generated bya more general ARDL model that includesa given financial variable. However, suchtests of statistical significance tell us relativelylittle about how much the financial variablesimprove forecasts. The distinction betweenrejecting the null hypothesis of forecast encom-passing and determining how much the finan-cial variables improve forecasts is similar to thecommondistinction between the statistical andeconomic significance of a coefficient estimate.Thus, to get a better sense of the ‘‘forecastingsignificance’’ of the financial variables, Table 2reports the estimated weight (l) attached tothe unrestricted model forecast in forming theoptimal composite forecast in equation (4).Weonly report the estimated weight for thosecases in which the ENC-T and/or ENC-NEWstatistics reported in Table 1 are significant atthe 10% level. From the discussion of optimalcomposite forecasts in section II, the estimatesof l in Table 2 should lie between zero andunity. They give us some idea of the relativeimportance of the unrestricted model forecastin generating the optimal composite forecastand hence of the relative importance of thefinancial variable in question in helping fore-cast realGDPgrowthout-of-sample.Asl rises,the unrestricted model forecast is relativelymore important in generating the optimalcomposite forecast. Following Granger andRamanathan (1984), we report estimates of lthat are obtained with an intercept termincluded in equation (4). If the forecasts arenot unbiased, then the error series will nothave zero means, and excluding an interceptfrom equation (4) could result in a biasedestimate of l.

The estimates of l in Table 2 vary consider-ably, although from the results in Table 1, weknow that all are statistically significant at the10% level. Broadly speaking, we observe thesmallest estimates of l (less than 0.30) in equa-tions involving M2 growth, the Federal fundsrate, the 3-month Treasury bill rate, and the

term spread. Equations involving M3 growth,the default spread, and the two stock marketvariables tend to produce noticeably largerestimates of l, generally between 0.30 and0.90. In these latter cases, forecasts formedusing the financial variable in question play aquantitatively important role in generating theoptimal composite forecast of real GDPgrowth. In a number of cases, it appears fromtheUmeasures reported Table 2 that there is aconsiderable reduction inRMSFE for the opti-mal composite forecasts relative to the ARbenchmark forecasts.

Industrial Production Growth Forecasts

Tables 3 and 4 report results when we useindustrial production growth in place of realGDPgrowth in equation (1) and are analogousto Tables 1 and 2. The patterns of forecastabil-ity are very similar in Tables 1 and 3. As inTable 1, although there is extensive evidenceof in-sample predictability over the full sampleaccording to the Wald statistics, U rises formost financial variables at most horizonsas we move from the 1971:1–1984:1 to the1985:1–1999:4 out-of-sample period. Againreminiscent of Table 1, there is hardly any evi-dence of out-of-sample forecasting ability forthe financial variables over the 1985:1–1999:4period when we use the relative MSFE metric.There is only one instance where U < 1 inTable 3 over the 1985:1–1999:4 out-of-sampleperiod, and this is for the default spread at theone-quarter-ahead horizon. This is also theonly instance where the MSE-T or MSE-Fstatistic is significant over the 1985:1–1999:4out-of-sample period.

According to the encompassing test resultsin Table 3—and in contrast to the relativeMSFE test results in Table 3—there is consid-erable evidence of forecasting ability for thefinancial variables over the 1985:1–1999:4out-of-sample period when we measure realoutput growth using the growth rate of indus-trial production. That is, the contrast betweenthe relative MSFE and encompassing testresults in Table 1 for the 1985:1–1999:4 out-of-sample period is also evident in Table 3.As inTable 1, theENC-NEW statistics indicatethat both M2 and M3 growth help fore-cast industrial production growth over the1985:1–1999:4 out-of-sample period. Theresults for real stock price growth and the divi-dend yield in Table 3 are also similar to those


in Table 1: According to the ENC-T and/orENC-NEW statistics, both real stock pricegrowth and the dividend yield appear to helpforecast growth in industrial production overthe 1985:1–1999:4 out-of-sample period. Thereis actually more extensive evidence that theinterest rate variables possess out-of-sampleforecasting ability over the 1985:1–1999:4period in Table 3 than in Table 1. AlthoughTable 1 contains rather limited evidence thatinterest rate variables help forecast real GDPgrowth over the 1985:1–1999:4 out-of-sampleperiod, the ENC-NEW statistics indicate thatall of the interest rate variables help forecastindustrial production growth at multiplehorizons over this out-of-sample period. Over-all, as inTable 1, the evidence forout-of-sampleforecasting ability over the 1985:1–1999:4period in Table 3 comes primarily from theENC-NEW statistic. This underscores theusefulness of test statistics with potentiallyhigher power when testing for out-of-sampleforecasting ability.

Finally, for those cases where the ENC-Tand/or ENC-NEW statistics reported inTable 3 are significant at the 10% level,Table 4 reports least squares estimates of lin equation (4) (with an intercept term includedin equation [4]). As in Table 2, the estimates ofl in Table 4 show a fair amount of variation.Equations involving M3 growth (at horizonsof one, two, and four quarters), the Federalfunds rate (at a horizon of one quarter), theterm spread (at a horizon of eight quarters),and the default spread (at a horizon of onequarter) tend to produce sizable estimates ofl, between 0.40 and 0.60. We see from the Umeasures inTable 4 that in some cases there is aconsiderable reduction inRMSFEfor the com-posite forecasts relative to the AR benchmarkforecasts.

IV. DATA MINING AND THE OUT-OF-SAMPLEFORECASTING ABILITY OF FINANCIAL

VARIABLES

We have argued that according to the crite-rion of forecast encompassing, some financialvariables do, in fact, evince forecasting abilitywith respect to real output growth over the1985:1–1999:4 out-of-sample period. How-ever, we consider a large number of financialvariables, and it is fair to wonder whether our‘‘significant’’ results for the 1985:1–1999:4 out-of-sample period in Tables 1 and 3 are simply

due to data mining across alternative financialpredictors. Though it is widely believed thatout-of-sample tests help guard against datamining, Inoue and Kilian (2003) argue thatin-sample and out-of-sample tests are equallysusceptible to data mining. In either case, theproblem is that the researcher can consider alarge number of potential predictors and focuson the ‘‘best’’ results. To investigate this issue,we implement a version of the data-miningbootstrap procedure developed by Inoue andKilian (2003). In the present context, the nullhypothesis is that none of the 10 financial vari-ables we consider has out-of-sample forecast-ing power over the 1985:1–1999:4 period,whereas the alternative hypothesis is that atleast one of the financial variables has forecast-ing power over the 1985:1–1999:4 period. Toimplement this test, Inoue and Kilian (2003)recommend using the maximal out-of-sampletest statistic; for example, max{j21,. . .,10} ENC-NEWj, where j indexes the financial variable.That is, we test the null hypothesis that thelargestENC-NEW statistic across the 10 finan-cial variables is equal to zero against the alter-native that it is greater than zero. The data areposited to be generated by the following systemunder the null hypothesis of no forecastingability for any of the financial variables:

Dyt ¼ a0 þXp1i¼1

aiDyt�i þ e1;t,ð9Þ

xt;j ¼ b0;j þXp2;ji¼1

bi;jDyt�i þXp3;ji¼1

ci;jxt�i;jð10Þ

þ e2;t; j, j ¼ 1, . . . , 10,

where the disturbance vector et¼ (e1,t, e2,t,1, . . .e2,t,10)

0 is independently and identically distrib-uted with covariance matrix S. We estimateequations (9) and (10) via OLS, with thelag orders selected using the SIC, and com-pute the OLS residuals feet ¼ ðee1;t, ee2;t;1, . . . ,ee2;t;10Þ0gTt¼1 We resample Tþ 50 times fromthe OLS residuals, giving us a pseudo-seriesof disturbance terms, fee�t g

Tþ50t¼1

. We drawfrom the OLS residuals across all equationstogether, thereby preserving the contempora-neous correlation across all of the disturbanceterms present in the original sample. Using

fee�t gTþ50t¼1

, the OLS parameter estimates, andsetting the initial lagged observations for Dytand xt,j ( j¼ 1, . . . 10) equal to zero, we can use


equations (9) and (10) to generate a pseudo-sample of observations for Dyt and xt,j,fDy�t , x�t;1, . . . , x�t;10g

Tþ50t¼1 . We drop the first

50� p observations, where, p¼max{p1,p2,1, . . . p2,10, p3,1, . . . p3,10}, leaving us with apseudo-sample of Tþ p observations, match-ing the original sample (including the initiallagged variables). For each pseudo-sample,we calculate each of the four out-of-sampletest statistics for each of the xt,j variables(j¼ 1, . . . 10), and we store the maximum ofeach of the out-of-sample statistics across the10 financial predictors. We repeat this process500 times, giving us an empirical distributionfor each of the maximal out-of-sample statis-tics. The empirical distribution is then used tocompute the 10%, 5%, and 1% critical valuesfor each of the maximal statistics.

Table 5 reports data-mining-robust criticalvalues corresponding to Table 1 (real GDPgrowth) and Table 3 (industrial productiongrowth) for the 1985:1–1999:4 out-of-sampleperiod. We are most interested in whetherthe out-of-sample encompassing statisticsthat are significant in Tables 1 and 3 remainsignificantwhenweuse thedata-mining-robustcritical values. Recall that 1 of the 10 financialvariables (corresponding toM3growth) exhib-its significant forecasting abilitywith respect toreal GDP growth over the 1985:1–1999:4 out-of-sample period at the one-quarter-aheadhorizon in Table 1 according to the ENC-Tstatistic, while 6 demonstrate significant

forecasting power according to the ENC-NEW statistic. The ENC-T statistic is not sig-nificant when we use the data-mining-robustcritical values, but two of the ENW-NEWstatistics (corresponding to the default spreadand real stock price growth) remain significantaccording to the data-mining-robust criticalvalues. Two (four) of the ENC-T (ENC-NEW) statistics are significant at a horizonof two quarters in Table 1, and zero (one,corresponding to M3 growth) remain signif-icant according to the data-mining-robustcritical values. None of the encompassingstatistics that are significant at horizons offourandeightquarters inTable1 remain signif-icant according to the data-mining-robustcritical values in Table 5. Note that none ofthe MSE-T or MSE-F statistics in Table 1 issignificant at any horizon when we use thedata-mining-robust critical values in Table 5.

We next compare the out-of-sample teststatistics for industrial production growthover the 1985:1–1999:4 out-of-sample periodin Table 3 to the data-mining-robust criticalvalues in Table 5. Recall that four of theENC-T statistics are significant at a horizonof one quarter in Table 3. Two remain sig-nificant according to the data-mining-robustcritical values in Table 5. At a horizon of onequarter, nine financial variables demonstratesignificant forecasting ability according tothe ENC-NEW statistic in Table 3, and six ofthese variables are still significant when we use

TABLE 5

Data-Mining-Robust Bootstrap Critical Values for the Maximal Out-of-Sample Statistics,

1985:1–1999:4 Out-of-Sample Period

Horizon (h)1 Quarter Ahead 2 Quarters Ahead 4 Quarters Ahead 8 Quarters Ahead

Sig. Level 10% 5% 1% 10% 5% 1% 10% 5% 1% 10% 5% 1%

Table 1 (real GDP growth)

MSE-T 1.76 2.06 2.61 1.88 2.22 2.65 2.07 2.39 3.07 2.41 2.81 3.77

MSE-F 3.60 2.77 8.49 6.07 8.80 15.16 9.40 15.32 24.03 15.63 22.65 44.87

ENC-T 2.28 2.55 3.24 2.44 2.77 3.19 2.66 3.01 3.71 2.96 3.52 4.65

ENC-NEW 3.54 4.96 7.43 6.68 8.13 12.82 10.16 14.83 23.90 17.09 25.74 43.06

Table 3 (industrial production growth)

MSE-T 1.64 1.94 2.47 1.93 2.15 2.76 2.02 2.34 2.80 2.58 2.97 2.47

MSE-F 3.26 4.39 7.49 5.43 7.71 15.30 11.15 16.10 29.93 20.79 33.24 60.26

ENC-T 2.21 2.40 2.99 2.45 2.73 3.30 2.59 3.00 3.61 3.33 4.31 5.82

ENC-NEW 3.21 4.33 6.74 5.62 8.24 14.70 11.99 16.76 32.44 20.72 36.04 81.22

Notes: Critical values correspond to the maximum values of the out-of-sample statistics reported in Table 1 (realGDP growth) and Table 3 (industrial production growth); critical values were computed using the data-mining bootstrapprocedure described in the text.


thedata-mining-robust critical values.Threeofthe eight ENC-NEW statistics that are signif-icant at a horizon of two quarters in Table 3 arealso significant according to the data-mining-robust critical values. (The single ENC-Tstatistic that is significant at a horizon of twoquarters in Table 3, corresponding to the divi-dend yield, is not significant according to thedata-mining-robust critical values.) None ofthe ENC-T or ENC-NEW statistics fromTable 3 is significant at longer horizons whenwe use the data-mining-robust critical values.As is the case for real GDP growth in Table 1,none of the MSE-T or MSE-F statistics inTable 3 is significant at any horizon when weuse the data-mining-robust critical values.

In summary, a number of the financial vari-ables evince forecasting ability at shorter hori-zons according to forecast encompassing testseven afterwe control for datamining across the10 financial variables along the lines suggestedby Inoue and Kilian (2003).

V. STRUCTURAL BREAKS

In linewith the extant literature,we find thatthe predictive ability of many financial vari-ables with respect to real output growth hasdeteriorated since the mid-1980s according toa relativeMSFEmetric. A natural explanationfor this breakdown is structural change, and,indeed, StockandWatson (2003) find extensiveevidence of structural breaks in their realoutput growth ARDL equations, with breaksoften occurring in the mid-1980s (also seeStock and Watson 1996). Following Stockand Watson (2003), we test for structuralbreaks in our real output growth ARDL equa-tions, focusing on the case where h¼ 1.We usetheAndrews (1993)SupF and theAndrews andPloberger (1994) ExpF and AveF statistics totest the null hypothesis of no structural changein the ARDL equation against the alternativehypothesis of a structural break at anunknowndate. Following the recommendation ofAndrews (1993), we set the trimming param-eter to 15%, so that the minimum length ofany regime is required tobe15%of the sample.9

To test for multiple structural breaks, we use

the subsample procedure in Bai (1997). That is,if a structural break is indicated for the fullsample, we then apply the structural changetests to each of the subsamples defined bythe breakpoint for the full sample.We continuein this fashion until all of the subsamplesdefined by any significant breakpoint arestable.10

The results are reported in Table 6. To con-serve space, we only report structural changetest results for the industrial productiongrowth ARDL equations. The results for thereal GDP growth ARDL equations are simi-lar.11 There is no evidence of structural changefor five of the ARDL equations (those includ-ing M0 growth, M1 growth, M3 growth, the3-month Treasury bill rate, and the defaultspread). There is evidence of one structuralbreak for the ARDL equations that includethe Federal funds rate and real stock pricegrowth and evidence of two structural breaksfor the ARDL equations that include M2growth, the term spread, and the dividendyield.Note that a significant breakpoint occur-ring between 1981:4 and 1984:3 is indicated forfour of the ARDL equations (those includingM2 growth, the term spread, real stock pricegrowth, and the dividend yield). Thus, it doesappear that the early to mid-1980s is a time ofstructural change for a number of real outputgrowth equations, confirming the results inStock and Watson (2003).

Although structural change provides apossible explanation for the breakdown inout-of-sample forecasting ability for a numberof financial variables according to a relativeMSFE metric over the 1985:1–1999:4 out-of-sample period, why do these financial variablesstill exhibit forecasting ability over this out-of-sample period according to encompassingtests? We conjecture that for some financialvariables, there has been a structural breaknear the mid-1980s that reduces—but doesnot necessarily eliminate—forecasting ability.A simulation exercise inClark andMcCracken(2003) provides insight into this. Clarkand McCracken (2003) estimate an ARDLmodel for real industrial production growthusing lagged nominal stock price growth as

9. Trimming is necessary for the asymptotic distribu-tion theory to go through. We base inferences on p-valuesgenerated using the response surfaces in Hansen (1997).Similar inferences are delivered by the Hansen (2000)fixed-regressor bootstrap.

10. In some cases, the structural change tests cannot beapplied to a subsample due to theminimum length require-ment defined by the trimming parameter.

11. The results for the real GDP growth ARDL equa-tions are available on request.


an explanatory variable to test the forecast per-formance of nominal stock price growth overthe 1986–2000 out-of-sample period at a hori-zon of one quarter.12 TheMSE-T andMSE-Ftests fail to detect any forecasting ability fornominal stock price growth over the 1986–2000 out-of-sample period, whereas theENC-T and ENC-NEW statistics are signif-icant. This matches our results for real stockprice growth inTable 3.They also find evidenceof a structural break in the ARDL equationin 1984:1, which is very near to our estimatedbreakpoint inTable 6 for the industrial produc-tion growth ARDL model that includes realstock price growth. The coefficient on laggednominal stock price growth is significant atthe 1% level over the first regime. Althoughthe coefficient decreases considerably in

magnitude over the second regime, it neverthe-less remains significant at the 5% level, so thatsome forecasting ability apparently remains.Clark and McCracken (2003) then simulatethe ARDL model using the estimated coeffi-cients with the break in 1984:1 incorporatedinto the data-generating process. They findthat the out-of-sample MSE-T and MSE-Fstatistics are rarely able to reject thenull hypothesis of no forecasting ability overthe 1986–2000 out-of-sample period (rejectionrates of 0.03 and 0.11, respectively), but theENC-T and ENC-NEW statistics reject thenull hypothesis quite frequently (rejectionrates of 0.84 and 0.98, respectively). Thus itappears that encompassing tests are morepowerful in detecting forecasting ability insituations where a financial variable’s pre-dictive ability has been reduced—but not com-pletely eliminated—when the out-of-sampleperiod begins near the time of the break.

TABLE 6

Structural Break Test Results for Industrial Production

Growth ARDL Model, Equation (1), h¼ 1

(1) (2) (3) (4) (5) (6)Financial Variable Sample SupF ExpF AveF Breakpoint

M0 growth 1959:3–1999:4 7.54 [0.45] 1.76 [0.47] 2.17 [0.64] —

M1 growth 1959:3–1999:4 4.75 [0.83] 1.59 [0.54] 2.87 [0.44] —

M2 growth 1959:3–1999:4 11.37 [0.13] 3.56 [0.10] 6.10 [0.05] 1981:4

1959:3–1981:4 7.67 [0.32] 2.44 [0.24] 4.33 [0.19] —

1982:1–1999:4 18.18 [0.00] 5.93 [0.01] 5.24 [0.12] 1991:1

M3 growth 1959:3–1999:4 8.42 [0.35] 2.53 [0.24] 4.26 [0.18] —

Federal funds rate, firstdifferences

1959:4–1999:4 15.02 [0.08] 5.28 [0.04] 5.81 [0.15] 1966:1

1966:2–1999:4 9.39 [0.41] 2.71 [0.37] 3.13 [0.61] —

3-month Treasury billrate, first differences

1959:4–1999:4 13.70 [0.12] 4.30 [0.11] 4.52 [0.32] —

Term spread 1959:4–1999:4 18.52 [0.02] 6.60 [0.01] 8.34 [0.03] 1984:1

1959:4–1984:1 6.80 [0.64] 1.33 [0.81] 2.34 [0.78] —

1984:2–1999:4 15.78 [0.02] 5.64 [0.03] 8.25 [0.06] 1991:1

Default spread 1959:4–1999:4 12.89 [0.16] 2.95 [0.32] 3.73 [0.47] —

Real stock price growth 1959:4–1999:4 13.63 [0.13] 4.45 [0.09] 5.59 [0.17] 1983:3

1959:4–1983:3 3.97 [0.94] 0.96 [0.93] 1.71 [0.92] —

1983:4–1999:4 11.63 [0.11] 3.53 [0.16] 5.35 [0.23] —

Dividend yield 1960:1–1999:4 22.19 [0.01] 7.27 [0.02] 9.90 [0.03] 1975:1

1960:1–1975:1 4.90 [0.81] 2.13 [0.60] 4.23 [0.53] —

1975:1–1999:4 22.34 [0.01] 9.09 [0.00] 9.49 [0.05] 1984:3

1984:4–1999:4 10.52 [0.22] 3.46 [0.27] 6.10 [0.28] —

Notes: SupF, ExpF, and AveF statistics reported in columns (3)–(5) are used to test the null hypothesis of nostructural change over the sample against the one-sided (upper-tail) alternative hypothesis of a structural break; p-valuesbased on the response surfaces in Hansen (1997) are given in brackets; bold entries indicate significance at the 10% level.The minimum length of any regime is required to be 15% of the full sample.

12. Nominal and real stock price growth are highlycorrelated.


We conduct a small Monte Carlo experi-ment along the lines of Clark and McCracken(2003) to provide some preliminary evidence insupport of our conjecture that the forecastingability of some financial variables has beenreduced but not completely eliminated by astructural break near the mid-1980s. We firstestimate anARDLequation for industrial pro-duction growth that includes real stock pricegrowth and that allows for a structural break inthe ARDL equation in 1983:3 (the breakpointdetected in Table 6). The estimated ARDLequation is

Dyt ¼ 2:13þ 0:43Dyt�1

ð0:64Þ ð0:08Þþ 0:08Drspt�1

ð0:02Þð11Þ

þ 0:08Drspt�2

ð0:02Þþet, t � 1983:3,

Dyt ¼ 1:01þ 0:56Dyt�1

ð0:52Þ ð0:09Þþ 0:02Drspt�1

ð0:01Þð12Þ

þ 0:02Drspt�2

ð0:01Þþet, t41983:3,

where Dyt is industrial production growth andDrspt is real stockpricegrowth. Standard errorsfor the coefficient estimates are given in paren-theses. Note that although the coefficients onlagged real stock price growth shrink after thebreak in 1983:3, they remain significant overthe post-1983:3 regime, so that real stock pricegrowth should have some forecasting powerfor industrial production growth after1983:3. We generate 200 simulated observa-tions for industrial production growth and

real stock price growth using equations (11),(12), and (8). For each simulated sample, wecompute the MSE-T, MSE-F, ENC-T, andENC-NEW statistics over the 1985:1–1999:4out-of-sample period, and we generate ap-value for each out-of-sample statistic foreach simulated sample using the bootstrappro-cedure described in Section 2 above. Table 7reports the proportion of the p-values for eachout-of-sample statistic that are less than orequal to 0.10. From Table 7, we see that theENC statistics are much more powerful thanthe MSE statistics at horizons of one and twoquarters in detecting forecasting ability overthe 1985:1–1999:4 out-of-sample period. Atthe longer horizons of four and eight quarters,only theENC-NEW statistic remains consider-ably more powerful than the MSE statistics.The pattern of rejection rates in Table 7 isconsistent with the results in Table 3 for theforecasting ability of real stock price growthover the 1985:1–1999:4 out-of-sample period.The simulation results lend support to thenotion that encompassing tests (especiallythe ENC-NEW statistic) are needed to detectforecasting ability in the presence of a struc-tural break that occurs near the start of the out-of-sample forecastingperiod that reduces—butdoes not completely eliminate—a financialvariable’s forecasting ability.

VI. CONCLUSION

In this article, we find that financial vari-ables are rarely helpful in forecasting futurereal output growth over the 1985:1–1999:4out-of-sample period according to a relative

TABLE 7

Simulated Power for the Out-of-Sample Test Statistics over the 1985:1–1999:4 Period,

Industrial Production Growth ARDL Model with Real Stock Price Growth as the

Financial Variable


MSE-T 0.04 0.08 0.13 0.13

MSE-F 0.13 0.21 0.29 0.30

ENC-T 0.57 0.54 0.41 0.26

ENC-NEW 0.80 0.78 0.71 0.51

Notes:Data-generating process for industrial production growth is the industrial production growth ARDL equationwith real stock price growth as the financial variable and a structural break at 1983:3; based on 200 simulated series forindustrial production growth and real stock price growth and 200 bootstrap replications for each simulation; power isthe proportion of the bootstrap p-values that are less than or equal to 0.10.


MSFE metric, confirming results in the extantliterature. However, if we instead judge theforecasting ability of financial variables usingthe notion of forecast encompassing, we findconsiderable evidence of forecasting abilityfor a number of financial variables. Our resultssuggest that the extant literature understatesthe out-of-sample forecasting ability of finan-cial variables over the 1985:1–1999:4 period.The difference in results between the presentstudyand the extant literature is thatour abilityto detect out-of-sample forecasting abilitysince themid-1980s likely reflects the increasedpower of forecast encompassing tests, espe-cially the ENC-NEW statistic, over otherout-of-sample test statistics. On a purely meth-odological level, the apparent success of theencompassing approach makes a strong casefor including it among tests of out-of-sampleforecasting power.

REFERENCES

Andrews, D. W. K. ‘‘Tests for Parameter Instability andStructural Change with Unknown Change Point.’’Econometrica, 61(4), 1993, 821–56.

Andrews, D. W. K., and W. Ploberger. ‘‘Optimal TestsWhen a Nuisance Parameter Is Present Only underthe Alternative.’’ Econometrica, 62(6), 1994,1383–414.

Bai, J. ‘‘Estimation of a Change Point in Multiple Regres-sions.’’ Review of Economics and Statistics, 79(4),1997, 551–63.

Bates, J. M., and C. W. J. Granger. ‘‘The Combination ofForecasts.’’Operations Research Quarterly, 20, 1969,451–68.

Chong,Y.Y., andD.F.Hendry. ‘‘EconometricEvaluationof Linear Macroeconomic Models.’’ Review ofEconomic Studies, 53(4), 1986, 671–90.

Clark, T. E., and M. W. McCracken. ‘‘Tests of EqualForecast Accuracy and Forecast Encompassing forNested Models.’’ Journal of Econometrics, 105(1),2001, 85–110.

———. ‘‘Forecast-Based Model Selection in the Presenceof StructuralBreaks.’’ Journal of Econometrics, forth-coming, 2003.

———. ‘‘Evaluating Long-Horizon Forecasts.’’ Manu-script, University of Missouri–Columbia, 2004.

Clements, M. P., and D. F. Hendry. Forecasting EconomicTime Series. Cambridge: Cambridge UniversityPress, 1998.

Diebold, F.X.Elements of Forecasting, 2nd ed. Cincinnati:South-Western, 2001.

Diebold, F. X., and R. Mariano. ‘‘Comparing PredictiveAccuracy.’’ Journal of Business and Economic Statis-tics, 13(2), 1995, 253–65.

Fair, R. C., and R. J. Shiller. ‘‘Comparing Informationin Forecasts from Econometric Models.’’ AmericanEconomic Review, 80(3), 1990, 375–89.

Granger, C. W. J., and P. Newbold. Forecasting EconomicTime Series, 2nd ed. San Diego: Academic Press,1986.

Granger, C. W. J., 2nd R. Ramanathan. ‘‘ImprovedMethods of Forecasting.’’ Journal of Forecasting,3(2), 1984, 197–204.

Hansen, B. E. ‘‘Approximate Asymptotic p-values forStructural Change Tests.’’ Journal of Business andEconomic Statistics, 15(1), 1997, 60–67.

———. ‘‘Testing for Structural Change in ConditionalModels.’’ Journal of Econometrics, 97(1), 2000,93–115.

Harvey, D. I., S. J. Leybourne, and P. Newbold. ‘‘Tests forForecast Encompassing.’’ Journal of Business andEconomic Statistics, 16(2), 1998, 254–59.

Inoue, A., and L. Kilian. ‘‘In-Sample or Out-of-SampleTests of Predictability: Which One Should WeUse?’’ Econometric Reviews, forthcoming, 2003.

Kilian, L. ‘‘Exchange Rates andMonetary Fundamentals:What Do We Learn from Long-HorizonRegressions?’’ Journal of Applied Econometrics,14(5), 1999, 491–510.

McCracken,M.W. ‘‘Asymptotics forOut-of-Sample Testsof Granger Causality.’’ Manuscript, University ofMissouri at Columbia, 2004.

Nelson, C. R. ‘‘The Prediction Performance of the F.R.B.-M.I.T.-PennModel of theU.S. Economy.’’AmericanEconomic Review, 62(5), 1972, 902–17.

Newey, W., and K. D. West. ‘‘A Simple, Positive Semi-Definite, Heteroskedasticity and AutocorrelationConsistent Covariance Matrix.’’ Econometrica,55(3), 1987, 703–8.

Stock, J. H., and M. W. Watson. ‘‘Evidence on StructuralInstability in Macroeconomic Time Series Rela-tions.’’ Journal of Business and Economic Statistics,14(1), 1996, 11–30.

———. ‘‘Forecasting Output and Inflation: The Role ofAsset Prices.’’ Journal of Economic Literature, 41(3),2003, 788–829.

Thoma, M. A., and J. A. Gray. ‘‘Financial MarketVariables Do Not Predict Real Activity.’’ EconomicInquiry, 36(4), 1998, 522–39.

West, K. D. ‘‘Asymptotic Inference about PredictiveAbility.’’ Econometrica, 64(5), 1996, 1067–84.

———. ‘‘Another Heteroskedasticity- and Auto-correlation-Consistent Covariance Matrix Esti-mator,’’ Journal of Econometrics, 76(1–2), 1997,171–91.


financial variables and the simulated out-of-sample forecastability of u.s. output growth since...

Documents