forecasting us state-level employment growth: an amalgamation approach

International Journal of Forecasting 28 (2012) 315–327

Contents lists available at SciVerse ScienceDirect

International Journal of Forecasting

journal homepage: www.elsevier.com/locate/ijforecast

Forecasting US state-level employment growth: An amalgamationapproachDavid E. Rapach ∗, Jack K. StraussDepartment of Economics, Saint Louis University, 3674 Lindell Boulevard, St. Louis, MO 63108-3397, United States

a r t i c l e i n f o

Keywords:Regional forecastingLabor market forecastingAutoregressive distributed lag modelGeneral-to-specific modelingBootstrappingFactor modelEncompassingCombining forecastsBusiness cycles

a b s t r a c t

We forecast US state-level employment growth using several distinct econometricapproaches: combinations of individual autoregressive distributed lag models, general-to-specific modeling with bootstrap aggregation (GETS-bagging), and approximate factor (or‘‘beta’’) models. Our results show that these forecasting approaches consistently deliversizable reductions in mean squared forecast error (MSFE) relative to an autoregressive(AR) benchmark model across the 50 US states. On the basis of forecast encompassing testresults, we also consider amalgamating these approaches and find that this strategy yieldsadditional forecasting improvements. These improvements are particularly evident duringnational business-cycle recessions, where the amalgamation approach outperforms the ARbenchmark for nearly all states and leads to a 40% reduction in MSFE on average acrossstates relative to the AR benchmark.© 2011 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

s. P

1. Introduction

Fluctuations in US state-level employment have impor-tant implications for state and local government policy-making, as well as for business location and investmentplanning. Thus, state employment growth forecasts are acrucial input for state and local government and businessdecision-making. National policy-makers are also keenlyinterested in forecasts of state and regional economicconditions, including employment, as exemplified by theFederal Reserve’s Beige Book. Despite the importance offorecasts of state employment growth, to the best of ourknowledge, no papers to date systematically analyze em-ployment growth forecasts across a broad range of USstates.1 The present paper fills this gap.

∗ Corresponding author. Tel.: +1 314 977 3601; fax: +1 314 977 1478.E-mail addresses: [email protected] (D.E. Rapach), [email protected]

(J.K. Strauss).1 Rapach and Strauss (2005) analyze various approaches to forecasting

Missouri employment growth. Indeed, there are relatively few studies ofstate employment fluctuations more generally, and the few studies thatdo exist focus on the relationship between the national business cycle andstate employment (e.g., Hamilton & Owyang, forthcoming; and Owyang,Rapach, & Wall, 2008).

0169-2070/$ – see front matter© 2011 International Institute of Forecasterdoi:10.1016/j.ijforecast.2011.08.004

In addition to its practical relevance, forecasts of stateemployment growth provide a valuable laboratory for theanalysis of forecasting strategies. A host of national, re-gional, and idiosyncratic influences potentially affect stateemployment growth. Many variables conceivably capturethese influences, thus presenting a forecaster with a sub-stantial degree of model uncertainty. Furthermore, stateeconomies often experience structural change relating, forexample, to institutions, technology, and public policy, sothat predictive relationships can vary significantly overtime. Overall, the model uncertainty and structural in-stability surrounding data-generating processes for stateemployment growth provide formidable forecasting chal-lenges. Thus, if we identify robust strategies for improvingemployment growth forecasts across states,we can thenbereasonably confident that they will deliver improvementsin other applications.

We construct simulated out-of-sample forecasts ofstate employment growth using three different econo-metric approaches, as well as an amalgamation of theseapproaches. The first approach combines forecasts fromindividual autoregressive distributed lag (ARDL) models,where each ARDL model is based on a single potentialpredictor. Stock and Watson (1999, 2003, 2004) recently

ublished by Elsevier B.V. All rights reserved.

http://dx.doi.org/10.1016/j.ijforecast.2011.08.004

http://www.elsevier.com/locate/ijforecast

http://www.elsevier.com/locate/ijforecast

mailto:[email protected]

mailto:[email protected]

http://dx.doi.org/10.1016/j.ijforecast.2011.08.004

316 D.E. Rapach, J.K. Strauss / International Journal of Forecasting 28 (2012) 315–327

popularized this approach by showing that combina-tion forecasts frequently outperform autoregressive (AR)model benchmark forecasts with respect to predicting na-tional inflation and output growth. Rapach and Strauss(2008) demonstrate similar gains for combination fore-casts with respect to US employment growth.We use fore-cast combiningweights that are functions of the pastmeansquared forecast error (MSFE) of individual ARDL mod-els, with recent forecasting performance receiving greateremphasis (Stock & Watson, 2004). This discount MSFE(DMSFE) approach performs well for forecasting US em-ployment and real output growth in Rapach and Strauss(2008) and Stock and Watson (2004), respectively; we in-vestigate the efficacy of the DMSFE approach for forecast-ing US state-level employment growth.2 The individualARDL model forecasts are based on eleven potential pre-dictors that are designed to capture various national, state-specific, and regional influences: five of the ConferenceBoard’s national leading economic indicators, oil price, fourstate-level variables (the unemployment rate, real income,real housing price, and housing building permits), and anaverage of adjacent state employment growth rates. Bycombining individual ARDL models based on a variety ofpredictors, the DMSFE approach helps to address modeluncertainty and instability, similarly to using asset diver-sification to reduce portfolio risk (Timmermann, 2006).3

The second approach uses general-to-specific (GETS)modeling in conjunction with bootstrap aggregation(bagging). GETS modeling begins with a general modelthat includes all potential predictors and then reduces thegeneral specification to amore specificmodel by excludinginsignificant variables according to a decision rule. GETSmodeling provides a means of constructing a reasonablyparsimonious forecastingmodel, thereby avoiding a highlyoverparameterized generalmodel; despite good in-samplefit, an overparameterizedmodel frequently generates poorout-of-sample forecasts. A possible drawback of the GETSapproach, however, is an unstable decision rule. Breiman(1996) proposes bagging as a method of improving theperformance of decision rules. Intuitively, instead ofrelying on a single historical realization of the data, webootstrap additional learning sets for the decision rule.In our context, bagging helps to accommodate modeluncertainty and instability by paring downa generalmodelusing a robustified decision rule. In the initial applicationof bagging to macroeconomic forecasting, Inoue and Kilian(2008) show that a GETS-bagging procedure improves USinflation forecasts. Rapach and Strauss (2010) also find thata GETS-bagging approach delivers forecasting gains withrespect to US employment growth. Similarly to Inoue andKilian (2008) and Rapach and Strauss (2010), we generateGETS-bagging forecasts by bagging a decision rule based onindividual t-statistics for a general model that includes alleleven potential predictors of state employment growth.

2 There are numerous methods for combining individual forecasts; seethe extensive survey by Timmermann (2006).3 See Clements and Hendry (2006) and Hendry and Clements (2004)

on the potential for combination forecasts to improve forecast accuracyin the presence of structural breaks.

The third approach is in the spirit of the approximatefactor model used by Owyang et al. (2008) for model-ing state employment growth fluctuations. This approachharnesses the link between state and national employ-ment growth. In essence, we assume an approximate fac-tor model structure for state employment growth, wherethe common component includes national employmentgrowth as the common factor and a state’s exposure to na-tional employment growth (i.e., its ‘‘beta’’). More specif-ically, we first estimate a state’s beta by regressing stateemployment growth on national employment growth us-ing data available at the time of forecast formation. A fore-cast of a state’s employment growth can then be straight-forwardly computed by plugging a forecast of national em-ployment growth into the fitted regressionmodel. (Wealsoaccount for serial correlation in the regressionmodel’s dis-turbance termwhen computing the forecast.) Observe thatthis approach relies on forecasts of national employmentgrowth. As was indicated above, the literature demon-strates that the DMSFE and GETS-bagging approaches pro-vide accurate forecasts of US employment growth rela-tive to an AR benchmark; thus, we use a simple averageof DMSFE and GETS-bagging forecasts as a forecast of USemployment growth in constructing beta forecasts of stateemployment growth.

Previewing our results, in the context of ARDL models,we find that individual predictors display a considerabledegree of variation in forecasting ability across US states.For example, an ARDLmodel based on national unemploy-ment claims reduces MSFE relative to the AR benchmarkby over 30% for Arizona, but it increases MSFE by morethan 100% for West Virginia. While individual ARDL mod-els produce erratic out-of-sample gains, DMSFE combina-tion forecasts generate very consistent gains across states:theDMSFE forecasts have a lowerMSFE than the AR bench-mark for 49 of the 50 states, with an average reductionacross states of 16%. GETS-bagging and beta forecasts pro-vide averageMSFE reductions across states of 14% and 18%,respectively, and more sizable reductions for a number ofstates. However, the performances of the GETS-baggingand beta forecasts are less consistent across states thanthose of the DMSFE forecasts, and they are substantiallyoutperformed by the AR benchmark for a few states. Insummary, the DSMFE, GETS-bagging, and beta approachesdeliver consistent and sizable improvements in forecastaccuracy relative to an AR benchmark, although the bag-ging and beta approaches yield sizable outliers.

We further compare the DMSFE, GETS-bagging, andbeta forecasts in a multiple encompassing framework(Harvey & Newbold, 2000). The results suggest that thereare gains from amalgamating these approaches, and, in-deed, we do find that an amalgam forecast that takes theform of a simple average of the DMSFE, GETS-bagging, andbeta state employment growth forecasts performs verywell. The amalgam forecasts reduceMSFE relative to theARbenchmark by a very sizable 26% on average across states.They also generate highly consistent gains, outperformingthe AR benchmark for 48 states (the only exceptions areAlaska andWest Virginia, where the increases in MSFE are5% and 2%, respectively), with the MSFE reductions beingnarrowly distributed across the states around the averageof 26%.

D.E. Rapach, J.K. Strauss / International Journal of Forecasting 28 (2012) 315–327 317

We also investigate forecast performance separatelyduring NBER-dated US business-cycle expansions and re-cessions. Our results clearly show that the forecasting gainsfrom the various approaches are strongly concentrated innational cyclical downturns. More specifically, the DMSFE,GETS-bagging, and beta forecasts reduce MSFE relative tothe AR benchmark by 22%, 30%, and 34% on average acrossstates during national recessions. Furthermore, the amal-gam forecasts reduceMSFE by 40% on average across statesand have anMSFE that is lower than the AR benchmark for49 states during recessions.

Lastly, we show that our results continue to hold whenwe exclude predictors that are available at a quarterly butnot a monthly frequency, as well as for real-time employ-ment data available for 2007:06–2010:12. By focusing onmonthly data, we can generate state employment growthforecasts in a more timely manner, while the use of real-time data further supports the practical relevance of ourresults. Overall, our results demonstrate the virtue of anamalgamation approach for forecasting state employmentgrowth.

The rest of the paper is organized as follows. Section 2outlines the construction of the various state employmentgrowth forecasts. Section 3 reports the simulated out-of-sample forecasting results. Section 4 concludes.

2. Forecast construction

2.1. DMSFE forecast

The DMSFE approach begins with the following ARDLmodel for state i employment growth:

1yhi,t+h = ai +q1−1j=0

bi,j1yi,t−j +

q2−1j=0

ci,jxk,t−j + ϵhi,t+h, (1)

where yi,t is the log-level of employment for state i at timet, 1yi,t = yi,t − yi,t−1, 1yhi,t+h = (1/h)

hj=1 1yi,t+j, xk,t

is a potential predictor of state employment growth(k = 1, . . . , K ), and ϵh

i,t+h is a zero-mean disturbanceterm.4 After dividing the total sample of T observationsinto an initial in-sample period, comprising the first Robservations, and an out-of-sample period, comprising thelast P = T − R observations, we form a series of P −

(h − 1) recursive simulated out-of-sample forecasts ofstate i employment growth based on Eq. (1). A forecastof 1yhi,t+h based on the predictor xk,t and informationavailable through t is given by

1yhi,t+h,k = ai,t +

q1−1j=0

bi,j,t1yi,t−j +

q2−1j=0

ci,j,txk,t−j, (2)

where ai,t , bi,j,t (j = 0, . . . , q1 − 1), and ci,j,t (j = 0, . . . ,q2 − 1) are ordinary least squares (OLS) estimates of the

4 Due to overlapping observations for the dependent variable whenh > 1, the disturbance term is autocorrelated of order h − 1 in Eq. (1).

parameters in Eq. (1) based on data through t .5 The seriesof P − (h−1) recursive out-of-sample forecasts is given by{1yhi,t+h,k}

T−ht=R for k = 1, . . . , K .

We consider K = 11 in our applications in Section 3.The DMSFE forecast takes the form of a weighted averageof the K individual ARDL forecasts:

1yh,DMSFEi,t+h =

Kk=1

ωk,t 1yhi,t+h,k, (3)

where the combining weights are functions of the MSFEsfor the individual ARDL forecasts:

ωk,t = m−1k,t /

Kk=1

m−1k,t (4)

and

mk,t =

t−hs=R

θ t−h−s(1yhi,s+h − 1yhi,s+h,k)2, (5)

so that individual ARDL models with more accurateforecasts (lower MSFEs) receive greater weight. Theparameter θ is a discount factor which allows greateremphasis to be placed on recent forecasting performancewhen θ < 1.6 We use a θ value of 0.9 in our applications inSection 3; the results for θ values of 1.0 and 0.8 (omittedfor brevity but available upon request) are very similar.Observe that we require a holdout out-of-sample periodfor computing the DMSFE forecasts. We reserve the firstP0 observations of the out-of-sample period as the holdoutout-of-sample period, so that we have P − (h − 1) − P0DMSFE forecasts available for evaluation.

An AR benchmark model, Eq. (1) with ci,j = 0 (j =

0, . . . , q2 − 1), serves as the benchmark model. We com-pute recursive simulated out-of-sample forecasts based onthe AR model in an analogous manner to the ARDL modelforecasts.7 The AR forecast frequently serves as a bench-mark with respect to forecasting macroeconomic vari-ables. Intuitively, if the xk,t variables contain informationuseful for forecasting 1yhi,t+h, then the DMSFE forecast,

5 We select the lag orders, q1 and q2 , based on the AIC, with maximumvalues of four for both q1 and q2 and minimum values of zero and onefor q1 and q2 , respectively; the minimum value of one for q2 ensures thatthe predictor xk,t is included in the ARDL forecasting model. Note that weonly use information through t when selecting the lag orders for the ARDLmodel when forming a forecast at t; that is, there is no ‘‘look-ahead’’ biasin selecting the lag orders.6 When θ = 1, there is no discounting. In this case, the DMSFE

forecast corresponds to the optimal combination forecast under quadraticloss derived by Bates and Granger (1969) for uncorrelated individualforecasts. More generally, optimal combining weights under quadraticloss depend on the complete covariance matrix of forecast errors (Bates& Granger, 1969; Timmermann, 2006), so that the combining weights inEq. (3) are not necessarily optimal under quadratic loss. However, it isdifficult to estimate the covariancematrix reliably in finite samples; thus,theoretically optimal but estimated combining weights often performpoorly in practice. Although the combining weights in Eq. (3) ignorecorrelations among forecast errors, the DMSFE combination forecastfrequently performs well in applications (e.g., Stock & Watson, 2001,2004).7 We select q1 using the AIC and a maximum (minimum) value of four

(zero) for the AR model.


which incorporates information from past state employ-ment growth and the xk,t variables, should outperform theAR forecast, which relies on past state employment growthalone.

2.2. GETS-bagging forecast

The GETS-bagging approach starts with the followinggeneral model that includes all K potential predictors:

1yhi,t+h = ai +q−1j=0

bi,j1yi,t−j +

Kk=1

ci,kxk,t + ϵhi,t+h. (6)

GETSmodeling uses a decision rule to exclude insignificantpredictors from Eq. (6). Following Inoue and Kilian (2008),the decision rule is based on t-statistics: we estimateEq. (6) via OLS using data through t , compute t-statisticsfor each ci,k estimate, and drop the xk,t variables withcorresponding t-statistics less than 1.645 in absolutevalue.8 The GETS forecast is then given by

1yh,GETSi,t+h = ai,t +

q−1j=0

bi,j,t1yi,t−j +

Kk=1

ci,k,t Ii,k,txk,t , (7)

where Ii,k,t is an indicator variable that takes a value of oneif xk,t has a t-statistic greater than1.645 in absolute value inEq. (6) and zero otherwise; and ai,t , bi,j,t (j = 0, . . . , q−1),and ci,k,t (k = 1, . . . , K) are OLS estimates of the specificmodel which excludes the insignificant xk,t variables basedon data through t .

Bagging augments the GETS procedure with a moving-block bootstrap in order to stabilize the decision rule.Instead of relying on a single historical realization of thedata to identify the significant predictors, we generatea large number (G) of pseudo samples of size t for theleft-hand-side and right-hand-side variables in Eq. (6) byrandomly drawing blocks of size m (with replacement)from observations of these variables through t . For agiven pseudo sample (indexed by g), we use the decisionrule to eliminate insignificant predictors and compute thefollowing forecast:

1yh,GETS,gi,t+h = agi,t +

q−1j=0

bgi,j,t1yi,t−j +

Kk=1

cgi,k,t Igi,k,txk,t , (8)

where Igi,k,t is an indicator variable that takes a value of oneif xk,t has a t-statistic greater than 1.645 in absolute valuefor the general model estimated for the pseudo sampleand zero otherwise; and agi,t , b

gi,j,t (j = 0, . . . , q − 1),

and cgi,k,t (k = 1, . . . , K) are OLS estimates of the specificmodel which excludes the insignificant xk,t variables forthe pseudo sample.9 Finally, the GETS-bagging forecast is

8 We again use the AIC to select q in Eq. (6) and amaximum (minimum)value of four (zero). To account for the autocorrelation in ϵh

i,t+h whenh > 1, the t-statistics are computed using Newey and West (1987)heteroskedasticity and autocorrelation consistent standard errors and alag truncation parameter of h − 1.9 Just to be clear with respect to Eq. (8), the significant predictors are

determined and the parameters estimated on the basis of the pseudosample, while we clearly need to plug the values of 1yi,t−j and xk,t fromthe original sample into Eq. (8) when forming the forecast of 1yhi,t+h .

an average of the G forecasts based on each of the pseudosamples:

1yh,GETSBi,t+h = (1/G)

Gg=1

1yh,GETS,gi,t+h . (9)

Following Inoue and Kilian (2008), we use m = h andG = 100 in our applications in Section 3.

2.3. Beta forecast

The beta approach is based on the following model:

1yi,t = αi + βi1yUS,t + ϵi,t , (10)

where 1yUS,t is the growth rate of US employment. Eq.(10) can be viewed as an approximate factor model forstate employment growth, where the single common fac-tor is national employment growth and βi represents statei’s exposure to the national factor.10 Intuitively, forecastsbased on Eq. (10) exploit links between state and nationalemployment fluctuations, with state betas capturing thestrength of these links. In contrast to a ‘‘strict’’ factormodel,which assumes that ϵi,t is white noise, the ‘‘approximate’’factormodel permits serial correlation in ϵi,t . Owyang et al.(2008) estimate an approximate latent factor model forstate employment growth using principal components andfind that the first common factor corresponds closely tonational employment growth, providing motivation forEq. (10). They only consider in-sample estimation, how-ever, so that our beta approach extends the work ofOwyang et al. (2008) to out-of-sample forecasting.

The beta forecast of 1yhi,t+h based on informationthrough t takes the form,

1yh,βi,t+h = αi,t + βi,t 1yhUS,t+h + ϵhi,t+h, (11)

where αi,t and βi,t are OLS estimates of αi and βi, respec-tively, in Eq. (10) based on data through t; 1yhUS,t+h is aforecast of 1yhUS,t+h = (1/h)

hj=1 1yUS,t+j; and ϵh

i,t+h is aforecast of ϵh

i,t+h = (1/h)h

j=1 ϵi,t+j based on informationthrough t .11

For implementing the beta approach, we require aforecast of 1yhUS,t+h. Rapach and Strauss (2010) find thatcombination and bagging forecasts of US employmentgrowth outperform an AR benchmark; furthermore, thecombination and bagging forecasts do not encompass eachother. In light of this, we construct DMSFE and baggingforecasts of US employment growth and use a simpleaverage of the two forecasts for 1yhUS,t+h in Eq. (11).

10 This is similar to the well-known capital asset pricing model (CAPM),where an individual stock’s expected return depends on its exposure tothe market portfolio (i.e., a stock’s market beta).11 More specifically, for generating ϵh

i,t+h , we first computeOLS residualsfor Eq. (10) using data through t .We then estimate anARprocess (withoutan intercept) for the OLS residuals, where the lag order is selected usingthe AIC. Finally, we use the estimated AR process to form forecasts of{ϵi,t+j}

hj=1 conditional on data through t .


3. Empirical results

3.1. Data

We use quarterly data for 1976:1–2010:4. The 1976:1starting date is dictated by the availability of data.Employment levels for the 50 individual US states are fromthe Bureau of Labor Statistics, and we compute annualizedquarterly employment growth as 400 times the differencein employment log-levels.

We consider eleven variables as potential predictors ofstate employment growth. Each variable is transformedto render it stationary. The first four variables are state-specific:

• State unemployment rate, differences (from the Bureau ofLabor Statistics).

• State real income growth (state nominal income is fromthe Bureau of Economic Analysis and is converted toreal income using the personal consumption expendi-ture deflator from the Bureau of Economic Analysis).

• State real housing price growth (state nominal housingprice is from the Office of Federal Housing EnterpriseOversight and is converted to real housing price usingthe personal consumption expenditure deflator).

• State housing building permit growth (from the CensusBureau).

The next six variables are national, and the first five areamong the Conference Board’s leading economic indica-tors:

• US manufacturing hours, differences.• US unemployment claims, log-levels.• US real new consumer goods order growth.• US building permit growth.• US real stock price growth.• US real oil price growth (nominal oil price is from the

Conference Board and is converted to real oil price usingthe personal consumption deflator).

The final variable is regional:

• Average adjacent state employment growth (mean ofemployment growth for adjacent states).12

These eleven predictors are representative of potentialstate-level, national, and regional influences on state em-ployment growth.

3.2. AR and ARDL forecasts

Table 1 presents results for the AR and ARDL forecastsfor the 1990:1–2010:4 forecast evaluation period.13 Weconsider a two-quarter forecast horizon in Table 1; theresults for a four-quarter horizon (omitted for brevitybut available upon request) are similar. Table 1 reportsMSFEs for the AR benchmark forecasts, as well as the

12 For Alaska (Hawaii), we use Washington (Alaska, Washington,Oregon, and California) as the ‘‘adjacent’’ state(s).13 Recall from Section 2.1 that we require a holdout out-of-sampleperiod for the DMSFE forecast. We use the 20 quarters prior to 1990:1(1985:1–1989:4) as the initial holdout out-of-sample period.

ratio of the MSFE for the ARDL model forecast based onthe predictor given in the row heading to the MSFE forthe AR benchmark forecast. Thus, an MSFE ratio less thanone indicates that the ARDL model outperforms the ARmodel according to the MSFE criterion. For each MSFEratio, we also compute the Clark and West (2007) MSFE-adjusted statistic in order to test the null hypothesisthat the ARDL model MSFE is greater than or equal tothe AR model MSFE, against the alternative that theARDL model MSFE is lower than that of the AR modelMSFE; this corresponds to a test of the null that theMSFE ratio is greater than or equal to one against thealternative that it is less than one.14 A bold entry inTable 1 indicates that the MSFE ratio is significantlyless than one at the 10% level according to the Clarkand West (2007) MSFE-adjusted statistic. For reference,Table 1 also reports results for AR and ARDL modelforecasts of US employment growth.15

The MSFEs for the AR benchmark model range from1.47 for Pennsylvania to 10.59 for Louisiana. The relativelyhighMSFE for Louisiana undoubtedly reflects the impact ofHurricane Katrina. The average MSFE across the 50 statesfor the AR benchmark is 3.57 (the median is 3.10). Thisis more than twice the MSFE for the AR forecast of USemployment growth (1.36). The lower MSFE for the US ARforecast relative to the state average is probably due to acertain degree of ‘‘washing out’’ of shocks across individualstates.

Turning to the ARDL forecasts, we observe that thestate-specific predictors perform inconsistently and failto produce sizable out-of-sample gains on average. Stateunemployment rate, real income, real housing price,and housing permits deliver a lower MSFE than the ARbenchmark for 15, 9, 16, and 21 states, and the averageMSFE ratios across states for these variables are 1.07,1.05, 1.03, and 1.02, respectively. As an example of thevariation in predictive power across states, consider stateunemployment rate. While it lowers MSFE by 18% relativeto the AR benchmark for Pennsylvania, it raises it by morethan 100% for Delaware, and state unemployment rateoutperforms the AR benchmark for only 15 of the 50 states.

The six national variables also perform relativelyinconsistently as predictors of state employment growth,although they tend to do better than the state-specificpredictors. The national variables generate lower MSFEsthan the AR benchmark for 35, 46, 49, 38, 45, and 5states for manufacturing hours, unemployment claims,new consumer goods orders, building permits, real stockprice, and real oil price, respectively. New consumer goods

14 The well-known Diebold and Mariano (1995) and West (1996)statistic for comparingMSFEs has a non-standard asymptotic distributionwhen comparing forecasts fromnestedmodels (Clark &McCracken, 2001,2005; McCracken, 2007). This is relevant for the MSFE ratios in Tables 1,2, 4, and 5, since the competing forecast nests the AR benchmark forecast.The Clark and West (2007) MSFE-adjusted statistic modifies the Dieboldand Mariano (1995) and West (1996) statistic so that it is approximatelydistributed as a standard normal variate when comparing forecasts fromnested models. It also performs well in finite samples.15 For the ARDL model forecasts of US employment growth based onthe first four predictors, we replace the state-level variables with theirnational counterparts.


orders appear to perform the best overall, with an averageMSFE ratio well below one (0.84) and a minimum ratio of0.58 (Wisconsin), while real oil price performs relativelypoorly overall, with an average ratio well above one (1.07).

The final individual predictor, average adjacent stateemployment, generates very inconsistent results acrossstates, as can be seen from the standard deviation of theMSFE ratios across states for the ARDL forecasts basedon adjacent state employment. This standard deviation issubstantially larger than the standard deviation for anyother predictor. Adjacent state employment reduces MSFErelative to the AR benchmark by 21%–32% for Alabama,Illinois, Maine, Nebraska, and South Carolina, but it raisesMSFE by over 20% for Alaska, Connecticut, and Oregon,and by 289% for West Virginia. In addition, adjacent stateemployment produces an MSFE ratio of less than one forjust over half of the states.

In summary, the inconsistent performance of the indi-vidual predictors in Table 1 demonstrates the substantialdegree of model uncertainty surrounding state employ-ment growth forecasting. It will be very difficult to iden-tify a priori a particular predictor or small set of predic-tors that will providemore accurate forecasts than a seem-ingly naïve time-series forecast (i.e., the AR model). Wenext investigate the ability of the DMSFE, GETS-bagging,and beta approaches to improve state employment growthforecasts.

3.3. DMSFE, GETS-bagging, and beta forecasts

Table 2 reports MSFE ratios for DMSFE, GETS-bagging,and beta forecasts of state employment growth. Again, abold entry indicates that the MSFE ratio is significantlybelow one at the 10% level. Table 2 shows that the DMSFEapproach delivers very consistent out-of-sample gains:the MSFE ratio for the DMSFE forecast is less than onefor 49 of the 50 states (the exception is Alaska, where itis 1.01), and all 49 of these MSFE ratios are significantat the 10% level according to the MSFE-adjusted statistic.The average MSFE reduction across states is 16%, andthe largest reduction is 32% (Iowa and Wisconsin), withreductions of 20% or more for twelve other states. Thestandard deviation of the MSFE ratios across states is alsorelatively low (0.07). Overall, Table 2 suggests that theDMSFE approach is an effective strategy for consistentlyimproving state employment growth forecasts.

The GETS-bagging forecasts also outperform the ARbenchmark on a reasonably consistent basis. The MSFEratio is less than one for 45 states, with an average MSFEratio across states of 0.86. Furthermore, the reduction inMSFE is 20% or more for 22 states. However, the GETS-bagging approach increases the MSFE by 22% and 98%for Wyoming and Delaware, respectively. Thus, the GETS-bagging approach appears to be somewhat riskier than theDMSFE approach.

Table 2 also shows that the beta forecasts producea substantial average reduction in MSFE across statesof 18% relative to the AR benchmark. The minimumMSFE ratio for the beta forecasts is 0.45 (Wisconsin),well below the minimum MSFE ratio for the DMSFE and

GETS-bagging forecasts. The beta forecasts also yield verysizable reductions in MSFE of 30% or more for 15 otherstates. Similar to the bagging forecasts, the beta forecastsprovide less consistent gains than the DMSFE forecasts,outperforming the AR benchmark for 47 states (comparedto 49 states for the DMSFE forecasts). West Virginia is amarked outlier for the beta approach, with an MSFE ratioof 4.11.While the beta approach performswell for the vastmajority of states, this is tempered by its especially poorperformance for West Virginia.

3.4. Multiple encompassing tests and an amalgam forecast

The results in Table 2 highlight the ability of theDMSFE, GETS-bagging, and beta approaches to produceimproved state employment growth forecasts. Next, wecompare these approaches in the context of multipleforecast encompassing tests. Harvey and Newbold (2000)extend the pairwise encompassing tests developed byHarvey, Leybourne, and Newbold (1998) to comparethree or more forecasts.16 Consider forming a forecast of1yhi,t+h as a (restricted) linear combination of the DMSFE,GETS-bagging, and beta forecasts:

1yh,LCi,t+h = λDMSFEi

1yh,DMSFEi,t+h + λGETSB

i1yh,GETSBi,t+h

+ λβ

i1yh,βi,t+h, (12)

where λDMSFEi + λGETSB

i + λβ

i = 1. We can use Eq. (12) totest whether a given forecast encompasses the other twoforecasts. For example, when λDMSFE

i = 1 and λGETSBi =

λβ

i = 0, the DMSFE forecast encompasses the baggingand beta forecasts. Intuitively, when λDMSFE

i = 1 andλGETSBi = λ

β

i = 0, the GETS-bagging and beta forecasts donot contain any information useful for forecasting 1yhi,t+hbeyond that already contained in the DMSFE forecast. Incontrast, when λGETSB

i = 0 and/or λβ

i = 0, the GETS-bagging and/or beta forecasts contain information usefulfor forecasting 1yhi,t+h — again, beyond that containedin the DMSFE forecast — so that the DMSFE forecastdoes not encompass the GETS-bagging and beta forecasts.Analogously, we can test whether the bagging forecastencompasses the DMSFE and beta forecasts (λGETSB

i =

1, λDMSFEi = λ

β

i = 0) and whether the beta forecastencompasses the DMSFE and bagging forecasts (λβ

i =

1, λDMSFEi = λGETSB

i = 0).Table 3 reports restricted OLS estimates of λDMSFE

i ,

λGETSBi , and λ

β

i in Eq. (12) for each state (allowing for anerror term in Eq. (12)). We also compute the Harvey andNewbold (2000) MS∗ statistic to test the null hypothesisthat the forecast given in the row heading encompasses

16 Forecast encompassing was developed by Chong and Hendry (1986)and Granger and Newbold (1973), among others; see Clements andHendry (1998) for a textbook exposition of forecast encompassing.


Table 1AR and ARDL forecasting results, state employment growth, two-quarter horizon, 1990:1–2010:4.

Predictor AL AK AZ AR CA CO CT DE FL GA HA ID IL IN IA

AR MSFE 3.22 4.25 5.12 2.28 2.65 3.11 2.34 5.51 4.03 3.51 4.45 4.33 2.50 3.91 1.92State unemployment rate, differences 1.00 1.19 1.02 1.01 0.96 1.04 1.20 2.32 1.08 1.09 0.96 1.13 1.14 0.97 1.07State real income growth 1.04 1.02 1.03 1.01 1.01 1.07 1.23 1.00 0.96 1.01 0.99 0.99 1.05 1.06 1.07State real housing price growth 1.01 1.00 1.05 1.01 1.01 1.11 0.98 1.00 1.00 1.03 0.99 0.95 1.05 0.96 1.04State housing building permit growth 0.92 1.36 0.84 1.09 1.02 1.12 1.08 1.00 0.91 0.86 0.99 1.04 1.02 1.05 1.85US manufacturing hours, differences 0.85 1.20 0.94 0.91 1.05 0.86 0.84 0.95 0.89 1.07 0.92 0.98 0.93 0.95 0.75US unemployment claims, log-levels 0.70 0.98 0.67 0.88 0.86 0.75 0.81 0.80 0.74 0.79 1.01 1.00 0.81 0.88 0.82US new consumer good order growth 0.78 1.22 0.76 0.84 0.97 0.67 0.86 0.71 0.91 0.92 0.88 0.87 0.64 0.86 0.69US building permit growth 0.78 1.16 0.87 0.92 1.00 0.94 1.03 0.86 0.82 0.88 0.87 0.79 1.00 0.84 0.99US real stock price growth 0.76 1.06 0.69 0.80 0.81 0.78 0.87 0.90 0.88 0.82 0.89 0.84 0.75 0.81 0.75US real oil price growth 1.07 1.08 1.05 1.05 1.11 1.03 1.07 0.98 1.00 1.04 1.03 1.09 1.17 1.11 1.19Avg. adjacent state employment growth 0.68 1.27 0.87 1.08 1.08 1.05 1.23 0.83 1.03 0.96 0.99 0.98 0.75 0.97 0.95

Predictor KS KY LA ME MD MA MI MN MS MO MT NE NV NH NJ


Predictor NM NY NC ND OH OK OR PA RI SC SD TN TX UT VA


Predictor VT WA WV WI WY US Avg. SD # < 1 Min. Max. Median

AR MSFE 2.65 3.08 2.63 2.45 4.61 1.36 3.57 1.69 – 1.47 10.59 3.10State unemployment rate, differences 1.09 0.99 1.07 0.97 1.04 1.06 1.07 0.21 15 0.82 2.32 1.03State real income growth 1.07 1.01 0.96 1.03 1.00 1.02 1.05 0.07 9 0.96 1.23 1.03State real housing price growth 1.01 1.18 1.14 1.00 0.98 1.06 1.03 0.06 16 0.85 1.19 1.01State housing building permit growth 1.00 0.97 1.13 1.01 0.99 0.78 1.02 0.15 21 0.83 1.85 1.00US manufacturing hours, differences 1.10 0.80 2.32 0.76 0.96 1.00 0.98 0.22 35 0.75 2.32 0.95US unemployment claims, log-levels 0.84 0.91 2.11 0.75 0.96 0.88 0.89 0.20 46 0.67 2.11 0.87US new consumer good order growth 0.92 0.80 0.88 0.58 0.96 0.95 0.84 0.11 49 0.58 1.22 0.85US building permit growth 0.91 0.97 0.92 0.82 1.01 0.91 0.93 0.11 38 0.75 1.23 0.92US real stock price growth 0.84 0.86 0.99 0.74 1.03 0.87 0.86 0.10 45 0.62 1.06 0.86US real oil price growth 1.07 1.07 1.04 1.15 1.11 1.05 1.07 0.07 5 0.96 1.43 1.07Avg. adjacent state employment growth 1.10 1.04 3.89 0.91 1.11 – 1.03 0.43 28 0.68 3.89 0.98

Notes: The AR MSFE rows report MSFEs for the AR benchmark model. The other rows report MSFE ratios for the ARDL model based on the predictor givenin the row heading relative to the AR benchmark model. Avg., SD, Min., Max., and Median are the average, standard deviation, minimum, maximum, andmedian, respectively, across the 50 states; # < 1 is the number of states with anMSFE ratio less than one. Bold indicates that the MSFE for the ARDLmodelis significantly less than the MSFE for the AR benchmark model at the 10% level according to the Clark and West (2007)MSFE-adjusted statistic.

the remaining two forecasts.17 A bold entry in Table 3indicates that the MS∗ statistic is significant at the 10%

17 In simulation experiments, Harvey and Newbold (2000), find thatthe MS∗ statistic generally has better size and power properties than avariety of F-statistics. The MS∗ statistic is constructed using the basicapproach proposed by Diebold and Mariano (1995); see Harvey andNewbold (2000 Section 3). While encompassing test statistics havenon-standard asymptotic distributions when comparing nested forecasts

level. Two or three of the λDMSFEi , λGETSB

i and λβ

i estimatesin Table 3 are often sizable for individual states, suggestingthat relevant information will frequently be excluded ifwe rely solely on one of the three forecasts. In addition,

(Clark & McCracken, 2001, 2005), the MS∗ statistics in Table 3 havestandard asymptotic distributions, since we are comparing three non-nested forecasts.


Table 2DMSFE, GETS-bagging, beta, and amalgam forecasting results, state employment growth, two-quarter horizon, 1990:1–2010:4.

Method AL AK AZ AR CA CO CT DE FL GA HA ID IL IN IA

DMSFE 0.72 1.01 0.72 0.86 0.90 0.84 0.85 0.85 0.86 0.87 0.91 0.86 0.76 0.86 0.68GETS-bagging 0.65 1.13 0.69 0.82 0.87 0.77 0.91 1.98 0.73 0.77 0.95 0.87 0.66 0.71 0.92Beta 0.61 1.15 0.64 0.72 0.94 0.74 0.74 0.70 0.77 0.77 0.83 0.87 0.51 0.76 0.66Amalgam 0.55 1.05 0.61 0.71 0.82 0.75 0.72 0.90 0.73 0.73 0.85 0.75 0.58 0.70 0.60

Method KS KY LA ME MD MA MI MN MS MO MT NE NV NH NJ


Method NM NY NC ND OH OK OR PA RI SC SD TN TX UT VA


Method VT WA WV WI WY US Avg. SD # < 1 Min. Max. Median

DMSFE 0.91 0.88 0.72 0.68 0.89 0.80 0.84 0.07 49 0.68 1.01 0.86GETS-bagging 0.74 0.77 0.68 0.68 1.22 0.73 0.86 0.21 45 0.63 1.98 0.83Beta 0.80 0.70 4.11 0.45 0.88 – 0.82 0.49 47 0.45 4.11 0.75Amalgam 0.75 0.71 1.02 0.51 0.86 – 0.74 0.12 48 0.51 1.05 0.73

Notes: The table reports MSFE ratios for the forecasts given in the row heading relative to the AR benchmark. The amalgam forecast is an average ofthe DMSFE, GETS-bagging, and beta forecasts. Avg., SD, Min., Max., and Median are the average, standard deviation, minimum, maximum, and median,respectively, across the 50 states; # < 1 is the number of states with an MSFE ratio less than one. Bold indicates that the MSFE for the competing forecastis significantly less than the MSFE for the AR benchmark at the 10% level according to the Clark and West (2007)MSFE-adjusted statistic.

Table 3Multiple forecast encompassing test results, state employment growth, two-quarter horizon, 1990:1–2010:4.


DMSFE 0.24 1.86 0.02 0.14 0.40 −0.37 −0.15 −0.54 −0.07 0.00 −0.27 0.44 −0.34 −0.08 0.46GETS-bagging 0.40 −0.09 0.40 0.23 0.51 0.49 0.40 −0.01 0.64 0.51 0.07 0.47 0.36 0.63 0.20Beta 0.36 −0.77 0.59 0.63 0.09 0.88 0.75 1.54 0.43 0.49 1.21 0.09 0.97 0.45 0.35


DMSFE 0.03 0.36 −0.51 0.36 0.32 −0.18 0.04 −0.26 0.28 0.03 0.32 0.40 −0.44 −0.01 0.11GETS-bagging 0.29 0.21 0.45 0.67 0.13 0.38 0.20 0.20 −0.04 0.20 0.61 0.22 0.47 0.19 −0.20Beta 0.69 0.44 1.06 −0.04 0.55 0.80 0.76 1.17 0.76 0.77 0.07 0.38 0.97 0.83 1.09


DMSFE −0.35 0.06 −0.02 0.70 −0.03 −1.24 0.10 0.15 0.82 −0.06 0.34 0.12 −0.19 −0.37 −0.27GETS-bagging 0.36 0.46 0.78 −0.43 0.24 0.59 0.13 0.15 0.42 0.47 0.09 0.37 0.31 0.56 0.26Beta 0.99 0.48 0.25 0.73 0.79 1.65 0.78 0.70 −0.24 0.59 0.58 0.51 0.88 0.81 1.02

Method VT WA WV WI WY Average SD # sig. Min. Max. Median

DMSFE −0.13 0.01 0.44 −0.08 0.12 0.05 0.44 29 −1.24 1.86 0.02GETS-bagging 0.74 0.28 0.60 −0.04 0.21 0.31 0.25 36 −0.43 0.78 0.33Beta 0.39 0.71 −0.04 1.12 0.67 0.63 0.43 9 −0.77 1.65 0.68

Notes: The table reports restricted OLS estimates of λDMSFEi , λGETSB

i , and λβ

i (λDMSFEi +λGETSB

i +λβ

i = 1) in Eq. (12) in the DMSFE, GETS-bagging, and Beta rows,respectively. Bold indicates significance at the 10% level for the Harvey and Newbold (2000) MS∗ statistic corresponding to a test of the null hypothesisthat the forecast given in the row heading encompasses the remaining two forecasts. Avg., SD, Min., Max., and Median are the average, standard deviation,minimum, maximum, and median, respectively, across the 50 states; # sig. is the number of states with significant MS∗ statistics at the 10% level.

there are numerous instances across states where the nullhypothesis of encompassing is rejected for each of theindividual forecasts, again signaling that relying on a singleforecast often ignores useful information. The averages ofthe λDMSFE

i , λGETSBi and λ

β

i estimates across states are 0.05,0.31 and 0.63, respectively.18 These averages, however,

18 The sizable average λβ

i estimate in Table 3 is consistent with theresults in Table 2, where the beta forecast often generates larger out-of-sample gains than the DMSFE and GETS-bagging forecasts.

mask sizable variations in the λDMSFEi , λGETSB

i and λβ

i esti-mates across states, as is evidenced by the standarddeviations in Table 3. In summary, the results in Table 3 donot favor relying exclusively on any of the DMSFE, GETS-bagging, or beta approaches when forecasting US state-level employment growth.

This leads us to consider an amalgamation approach toforecasting state employment growth. More specifically,we amalgamate the DMSFE, GETS-bagging, and betaapproaches by taking a simple average of the three


forecasts.19 MSFE ratios for these amalgam forecasts arereported in the Amalgam rows in Table 2. The amalgamforecasts have an average MSFE ratio across states of0.74, which is well below the average MSFE ratios for theDMSFE, GETS-bagging, and beta forecasts. Furthermore,the amalgam forecasts outperform the AR benchmark for48 states, and the MSFE ratio is only 1.02 (1.05) for WestVirginia (Alaska), thus avoiding the marked outlier forWest Virginia (4.11) produced by the beta approach.The minimum MSFE ratio across states for the amalgamforecasts is 0.51 (Wisconsin), which is well below theminima for the DMSFE and GETS-bagging forecasts andreasonably near the minimum for the beta forecasts. Insummary, the amalgam forecasts generate consistent out-of-sample gains across states and deliver more sizableimprovements in forecast accuracy on average than any oftheDMSFE, GETS-bagging, andbeta forecasts alone.20 Thus,our results indicate that an amalgamation approach is veryuseful for improving state employment growth forecasts,as portended by the results in Table 3.

3.5. National business-cycle phases

Table 4 provides information on the performancesof the various approaches during different phases ofthe national business cycle by reporting AR MSFEs andMSFE ratios separately during NBER-dated US business-cycle expansions and recessions.21 From the AR MSFErows, we see that the MSFEs for the AR benchmark arenearly always substantially higher during recessions thanexpansions, pointing to the difficulty of anticipating thedeclines in economic activity characterizing recessions.The DMSFE, GETS-bagging, beta, and amalgam forecastsall have average MSFE ratios well below one duringrecessions, while the average ratios are much closer toone during expansions. Thus, these approaches generatesizable gains on average during recessions, which areprecisely the periods in which AR forecast accuracydeteriorates. The amalgam forecasts perform especiallywell during recessions: they have an average MSFE ratioof 0.60 (lower than the averages for the DMSFE, GETS-bagging, and beta forecasts during recessions) and aminimum MSFE ratio of 0.25 (Nebraska). The amalgamforecasts also outperform the AR benchmark for 49 of the50 states (the exception is Delaware, where theMSFE ratiois 1.01).

3.6. Additional results

The timing of data releases entails lags in the availabilityof state employment growth forecasts. Consider forminga state employment growth forecast for the first half of

19 We considered different methods of estimating the combiningweights for the amalgam forecast, similar to the DMSFE approach.However, alternative weighting schemes generally failed to outperformequal weighting.20 Observe that the amalgam MSFE ratio for each state is below or verynear the smallest MSFE ratio among the ARDL models for each state inTable 1.21 The cyclical peak and trough dates which define US expansions andrecessions are available at http://www.nber.org/cycles.html.

2010 using observations for the national and state-specificvariables through to the end of 2009. Observations for thenational variables and state employment, unemploymentrates, and housing building permits for 2009:4 areavailable by January 2010. However, observations for statehousing prices are only available in February 2010, andobservations for state incomes are not available untilMarch 2010. Thus, a forecast of the state employmentgrowth for the first two quarters of 2010 based on datauntil 2009:4 can only be produced in March 2010. Thisdelay can limit the value of state employment growthforecasts for their ultimate users, including policy-makers.

We can generate state employment growth forecastsin a more timely manner by focusing on monthly data.Data for all of the national variables, as well as stateemployment, unemployment rates, and housing buildingpermits are available at a monthly frequency, while statehousing price and income data are only available ata quarterly frequency. Consider again forming a stateemployment growth forecast for the first half of 2010(2010:01–2010:06) using data until the end of 2009. Datafor all of the national variables and state employment,unemployment rates, and housing building permits for2009:12 are available by January 2010. By focusing onmonthly data, we can thus move the forecast availabilityfrom March 2010 to January 2010 (although we no longerinclude predictors that are only available with a lag ofmore than one month — state housing prices and income— for which data are not available at amonthly frequency),thereby increasing thepractical usefulness of the forecasts.

Table 5 reports MSFE ratios for the DMSFE, GETS-bagging, beta, and amalgam forecasts of state employmentgrowth based on monthly data for a six-month horizon,so that the results are comparable to those based onquarterly data for a two-quarter horizon in Table 2. Thestarting date for the data is 1976:01, and 1990:01–2010:12serves as the forecast evaluation period (analogous to the1990:1–2010:4 forecast evaluation period in Table 2). Theresults in Tables 2 and 5 are quite similar; for example,the averages of the MSFE ratios across states are 0.84, 0.86,0.82, and 0.74 in Table 2 and 0.83, 0.85, 0.84, and 0.77 inTable 5 for the DMSFE, GETS-bagging, beta, and amalgamforecasts, respectively. The amalgam forecasts continue toperform very well in Table 5, generating consistent andsizable out-of-sample gains.22 In short, amalgam forecastsof state employment growth computed frommonthly data,which are available on a more timely basis for ultimateusers, produce gains similar to those of amalgam forecastsbased on quarterly data.

Up to this point, our simulated out-of-sample forecastsdo not account for data revisions. Since employmentdata are sometimes substantially revised, to add furtherrealism, we analyze state employment growth forecastsconstructed on the basis of real-time data. The ALFREDdatabase at the Federal Reserve Bank of St. Louis provides

22 The AR MSFEs and MSFE ratios for the individual ARDL models basedonmonthly data (omitted for brevity but available upon request) are alsovery similar to those in Table 1. The same holds for the encompassingtest results in Table 3 and the performance over national business-cyclephases in Table 4.

http://www.nber.org/cycles.html


Table 4AR, DMSFE, GETS-bagging, beta, and amalgam forecasting results computed separately for NBER-dated business-cycle expansions and recessions, stateemployment growth, two-quarter horizon, 1990:1–2010:4.


AR MSFE, expansion 1.69 3.86 3.42 1.51 1.46 1.83 1.66 3.87 3.06 1.85 3.34 2.32 1.32 2.35 1.25DMSFE, expansion 0.76 1.06 0.77 0.88 0.97 0.88 0.89 0.89 0.87 0.87 0.95 0.90 0.89 0.90 0.73GETS-bagging, expansion 0.93 1.23 0.78 1.05 1.09 0.86 1.11 0.90 0.73 0.93 1.11 1.23 0.72 0.91 1.04Beta, expansion 0.92 1.06 0.67 1.02 1.23 0.76 0.85 0.80 0.72 0.88 0.92 1.11 0.72 1.06 0.87Amalgam, expansion 0.74 1.07 0.67 0.89 0.97 0.80 0.80 0.83 0.72 0.77 0.96 0.95 0.69 0.87 0.75AR MSFE, recession 12.27 6.53 15.21 6.84 9.69 10.70 6.33 15.22 9.76 13.32 11.03 16.28 9.46 13.11 5.84DMSFE, recession 0.68 0.84 0.66 0.83 0.84 0.80 0.79 0.78 0.85 0.87 0.83 0.83 0.65 0.83 0.61GETS-bagging, recession 0.43 0.81 0.56 0.52 0.68 0.67 0.61 3.60 0.73 0.64 0.65 0.57 0.61 0.50 0.71Beta, recession 0.36 1.49 0.59 0.32 0.69 0.71 0.58 0.56 0.87 0.69 0.68 0.66 0.34 0.44 0.39Amalgam, recession 0.40 0.99 0.53 0.47 0.70 0.70 0.60 1.01 0.75 0.69 0.67 0.59 0.50 0.52 0.41





Method VT WA WV WI WY Avg. SD # < 1 Min. Max. Median

AR MSFE, expansion 1.66 1.69 2.18 1.30 3.32 2.38 1.61 – 0.85 11.52 1.86DMSFE, expansion 0.98 0.91 0.85 0.70 0.95 0.89 0.07 48 0.70 1.06 0.89GETS-bagging, expansion 0.86 1.01 0.82 0.95 1.54 1.02 0.19 26 0.72 1.56 0.96Beta, expansion 0.84 0.89 3.64 0.66 1.00 0.97 0.41 35 0.55 3.64 0.92Amalgam, expansion 0.81 0.85 1.16 0.68 1.06 0.86 0.12 46 0.64 1.16 0.83AR MSFE, recession 8.49 11.34 5.33 9.25 12.22 10.57 5.15 – 3.69 35.13 9.74DMSFE, recession 0.83 0.86 0.40 0.66 0.80 0.78 0.09 50 0.40 0.94 0.80GETS-bagging, recession 0.60 0.56 0.34 0.42 0.70 0.70 0.47 47 0.30 3.60 0.61Beta, recession 0.75 0.52 5.22 0.27 0.68 0.66 0.70 48 0.18 5.22 0.58Amalgam, recession 0.67 0.58 0.67 0.37 0.55 0.60 0.15 49 0.25 1.01 0.60

Notes: TheARMSFE rows report theMSFEs for the AR benchmarkmodel. The other rows report ratios of theMSFEs for the forecasts given in the rowheadingrelative to the AR benchmark. The amalgam forecast is an average of the DMSFE, GETS-bagging, and beta forecasts. Avg., SD, Min., Max., and Median arethe average, standard deviation, minimum, maximum, and median, respectively, across the 50 states; # < 1 is the number of states with an MSFE ratioless than one. Bold indicates that the MSFE for the competing forecast is significantly less than the MSFE for the AR benchmark at the 10% level accordingto the Clark and West (2007)MSFE-adjusted statistic.

vintages of real-time monthly state employment databeginning in June of 2007, thus allowing us to analyzesimulated real-time state employment growth forecastsfor 2007:06–2010:12. While data availability limits us tothis relatively short forecast evaluation period, this perioddoes cover the recent Great Recession in the United States,making it of considerable interest.23

We form monthly state employment growth forecastsfor a six-month horizon in the same manner as the

23 Real-time national and state employment data are available fromALFRED at http://alfred.stlouisfed.org/. The other monthly variables areeither not revised or subject to onlyminor revisions, especially relative tothe revisions in state employment.

forecasts based on monthly data in Table 5, with theexception that we now use real-time national andstate employment data available at the time of forecastformation. Consider the first employment growth forecastfor a given state based on data through 2007:05 for asix-month horizon (2007:06–2007:11). We generate thisforecast using the June 2007 vintage of employmentdata for the given and adjacent states (as well as theUnited States as a whole), which provides data for1976:01–2007:05, where the 2007:05 figures representinitial estimates of state and national employment for thismonth. In thisway,we forma forecast of state employmentgrowth for 2007:06–2007:11 based on employment data

http://alfred.stlouisfed.org/


Table 5DMSFE, GETS-bagging, beta, and amalgam forecasting results for monthly data, state employment growth, six-month horizon, 1990:01–2010:12.









Notes: The table reports MSFE ratios for the forecasts given in the row heading relative to the AR benchmark. The amalgam forecast is an average ofthe DMSFE, GETS-bagging, and beta forecasts. Avg., SD, Min., Max., and Median are the average, standard deviation, minimum, maximum, and median,respectively, across the 50 states; # < 1 is the number of states with an MSFE ratio less than one. Bold indicates that the MSFE for the competing forecastis significantly less than the MSFE for the AR benchmark at the 10% level according to the Clark and West (2007)MSFE-adjusted statistic.

Table 6DMSFE, GETS-bagging, beta, and amalgam forecasting results for real-timemonthly data, state employment growth, six-month horizon, 2007:06–2010:12.









Notes: The table reports MSFE ratios for the forecasts given in the row heading relative to the AR benchmark. The amalgam forecast is an average ofthe DMSFE, GETS-bagging, and beta forecasts. Avg., SD, Min., Max., and Median are the average, standard deviation, minimum, maximum, and median,respectively, across the 50 states; # < 1 is the number of states with an MSFE ratio less than one. Bold indicates that the MSFE for the competing forecastis significantly less than the MSFE for the AR benchmark at the 10% level according to the Diebold and Mariano (1995) and West (1996) statistic.

until 2007:05 that was actually available in June 2007.The second state employment growth forecast is computedusing the July 2007 vintage, which provides nationaland state employment data for 1976:01–2007:06; thisvintage includes the initial estimates for 2007:06 andpossibly revised estimates for the earlier months. Wecontinue in this manner until the end of the out-of-

sample period. When analyzing the forecast accuracy, wemeasure the forecast error as the difference between actualstate employment growth computed using revised dataaccording to the March 2011 vintage and the simulatedreal-time forecast.

Table 6 reports MSFE ratios for the DMSFE, GETS-bagging, beta, and amalgam forecasts of state employment


growth based on real-time data for the 2007:06–2010:12forecast evaluation period and a six-month horizon.24 Theresults in Table 6 continue to indicate the usefulness of theDMSFE, GETS-bagging, beta, and amalgam forecasts. TheDMSFE, GETS-bagging, and beta forecasts outperform theAR benchmark in terms of MSFE for 50, 47, and 45 states,respectively, and lower MSFEs vis-á-vis the AR benchmarkby 19%, 36%, and 32%, respectively, on average acrossstates. The amalgam forecasts deliver lower MSFEs thanthe AR benchmark for all 50 states and produce averageMSFE reductions of 35% across states relative to theAR benchmark. The GETS-bagging, beta, and/or amalgamforecasts also generate MSFE reductions of greater than50% for a number of states. Table 6 indicates that consistentand sizable out-of-sample gains are evident for amalgamforecasts of state employment growth based on real-time data, at least for the period surrounding the GreatRecession.

4. Conclusion

We investigate several distinct econometric approachesto forecasting state employment growth. The DMSFE,GETS-bagging, and beta approaches produce consistentand sizable improvements in forecast accuracy relativeto an AR benchmark. DMSFE forecasts outperform theAR benchmark in terms of MSFE for 49 of the 50 USstates for the 1990:1–2010:4 forecast evaluation period.While GETS-bagging and beta forecasts generate evenlarger reductions in MSFE for a number of states, there arealso a few states where these forecasts are substantiallyoutperformed by the AR benchmark.

On the basis of forecast encompassing tests, we con-sider amalgam forecasts of state employment growth thatare simple averages of the DMSFE, GETS-bagging, and betaforecasts. The amalgam forecasts perform very well over-all, delivering more consistent out-of-sample gains thanthe GETS-bagging and beta forecasts and larger averagereductions in MSFE than the DMSFE forecasts. Amalgamforecasts generate the largest improvements over the ARbenchmark during cyclical downturns. Our results are ro-bust to the use ofmonthly data,which permitsmore timelyforecast availability, as well as the use of real-time data (atleast around the Great Recession). The consistent and sub-stantial out-of-sample gains delivered by the amalgama-tion approach are especially encouraging in light of the dif-ficult challenges inherent in forecasting state employmentgrowth.

24 Clark and McCracken (2009) show that the asymptotic distributionsof test statistics for non-nested and nested model comparisons changewhen applied to real-time instead of revised data; furthermore,appropriate adjustments of test statistics can depend on the nature ofdata revisions and nuisance parameters. In Table 6, we test whetherthe competing forecast has a lower MSFE than the AR benchmark usingthe Diebold and Mariano (1995) and West (1996) statistic and standardnormal critical values, since simulations in Clark and McCracken (2009)suggest that this approach provides a reasonable guide to statisticalsignificance in our context.

Acknowledgments

We thank two referees and an Associate Editor, aswell as session participants at the 2010 InternationalSymposium on Forecasting in San Diego, for very helpfulcomments. The usual disclaimer applies. The authorsacknowledge financial support from the Simon Center forRegional Forecasting at Saint Louis University.

References

Bates, J. M., & Granger, C. W. J. (1969). The combination of forecasts.Operational Research Quarterly, 20, 451–468.

Breiman, L. (1996). Bagging predictors.Machine Learning , 24, 123–140.Chong, Y. Y., & Hendry, D. F. (1986). Econometric evaluation of linear

macro-economic models. Review of Economic Studies, 53, 671–690.Clark, T. E., & McCracken, M. W. (2001). Tests of equal forecast accuracy

and encompassing for nested models. Journal of Econometrics, 105,85–110.

Clark, T. E., & McCracken, M. W. (2005). Evaluating direct multi-stepforecasts. Econometric Reviews, 24, 369–404.

Clark, T. E., & McCracken, M. W. (2009). Tests of equal predictive abilitywith real-time data. Journal of Business and Economic Statistics, 27,441–454.

Clark, T. E., & West, K. D. (2007). Approximately normal tests for equalpredictive accuracy in nested models. Journal of Econometrics, 138,291–311.

Clements, M. P., & Hendry, D. F. (1998). Forecasting economic time series.Cambridge: Cambridge University Press.

Clements, M. P., & Hendry, D. F. (2006). Forecasting with breaks. In G.Elliott, C. W. J. Granger, & A. Timmermann (Eds.), Handbook ofeconomic forecasting (pp. 605–657). Amsterdam: Elsevier.

Diebold, F. X., & Mariano, R. S. (1995). Comparing predictive accuracy.Journal of Business and Economic Statistics, 13, 253–263.

Granger, C.W. J., & Newbold, P. (1973). Some comments on the evaluationof economic forecasts. Applied Economics, 5, 35–47.

Hamilton, J. D., & Owyang, M. T. (forthcoming). The propagation ofregional recessions. Review of Economics and Statistics, forthcoming.

Harvey, D. I., Leybourne, S. J., & Newbold, P. (1998). Tests forforecast encompassing. Journal of Business and Economic Statistics, 16,254–259.

Harvey, D. I., & Newbold, P. (2000). Tests for multiple forecastencompassing. Journal of Applied Econometrics, 15, 471–482.

Hendry, D. F., & Clements, M. P. (2004). Pooling of forecasts. EconometricsJournal, 7, 1–31.

Inoue, A., & Kilian, L. (2008). How useful is bagging in forecastingeconomic time series? A case study of US CPI inflation. Journal of theAmerican Statistical Association, 103, 511–522.

McCracken, M. W. (2007). Asymptotics for out of sample tests of Grangercausality. Journal of Econometrics, 140, 719–752.

Newey, W. K., & West, K. D. (1987). A simple positive semi-definiteheteroskedasticity and autocorrelation consistent covariance matrix.Econometrica, 55, 703–708.

Owyang, M. T., Rapach, D. E., & Wall, H. J. (2008). States and the businesscycle. Journal of Urban Economics, 65, 181–194.

Rapach, D.E., & Strauss, J.K. (2005). Forecasting employment growth inMissouri with many potentially relevant predictors: an analysis offorecast combining methods. Federal Reserve Bank of Saint Louis,Regional Economic Development, 1, 97–112.

Rapach, D. E., & Strauss, J. K. (2008). Forecasting US employment growthusing forecast combining methods. Journal of Forecasting , 27, 75–93.

Rapach, D. E., & Strauss, J. K. (2010). Bagging or combining (or both)? Ananalysis based on forecasting US employment growth. EconometricReviews, 29, 511–533.

Stock, J. H., & Watson, M. W. (1999). Forecasting inflation. Journal ofMonetary Economics, 44, 293–335.

Stock, J. H., &Watson, M.W. (2001). A comparison of linear and nonlinearunivariate models for forecasting macroeconomic time series. In R.F. Engle, & H. White (Eds.), Cointegration, causality, and forecasting: afestschrift in honour of Clive W. J. Granger (pp. 1–44). Oxford: OxfordUniversity Press.

Stock, J. H., & Watson, M. W. (2003). Forecasting output growth andinflation: the role of asset prices. Journal of Economic Literature, 41,788–829.

Stock, J. H., & Watson, M. W. (2004). Combination forecasts of outputgrowth in a seven-country data set. Journal of Forecasting , 23,405–430.


Timmermann, A. (2006). Forecast combinations. In G. Elliott, C. W. J.Granger, & A. Timmermann (Eds.), Handbook of economic forecasting(pp. 135–196). Amsterdam: Elsevier.

West, K. D. (1996). Asymptotic inference about predictive ability.Econometrica, 64, 1067–1084.

David E. Rapach is an Associate Professor of Economics and ResearchFellow of the Simon Center for Regional Forecasting at Saint LouisUniversity. His research interests include time-series econometrics,macroeconomics, international finance, and financial economics. He haspublished numerous articles in a range of scholarly journals, includingEconometric Reviews, International Journal of Forecasting, Journal of Applied

Econometrics, Journal of Forecasting, Journal of International Economics,Journal of International Money and Finance, Journal of Macroeconomics,Journal of Money, Credit, and Banking, Journal of Urban Economics, andReview of Financial Studies.

Jack K. Strauss is a Professor of Economics and the Director of the SimonCenter for Regional Forecasting at Saint Louis University. His research in-terests include time-series econometrics, macroeconomics, internationalfinance, and financial economics. He has published numerous articles ina range of scholarly journals, including Econometric Reviews, InternationalJournal of Forecasting, Journal of Applied Econometrics, Journal of FinancialResearch, Journal of Forecasting, Journal of International Money and Finance,Journal of Macroeconomics, and Review of Financial Studies.

forecasting us state-level employment growth: an amalgamation approach

Documents