real-time squared: a real-time data set for real-time gdp forecasting

Available online at www.sciencedirect.com

sting 24 (2008) 368–385www.elsevier.com/locate/ijforecast

International Journal of Foreca

Real-time squared: A real-time data set forreal-time GDP forecasting

Roberto Golinelli a,⁎, Giuseppe Parigi b

a Department of Economics, University of Bologna, Strada Maggiore 45, 40125 Bologna, Italyb Research Department, Bank of Italy, via Nazionale 91, 00184 Rome, Italy

Abstract

This paper uses real-time data to mimic real-time GDP forecasting activity. Through automatic searches for the bestindicators for predicting GDP one and four steps ahead, we compare the out-of-sample forecasting performance of adaptivemodels using different data vintages, and produce three main findings. First, despite data revisions, the forecasting performanceof models with indicators is better, but this advantage tends to vanish over longer forecasting horizons. Second, the practice ofusing fully updated datasets at the time the forecast is made (i.e., taking the best available measures of today's economicsituation) does not appear to bring any effective improvement in forecasting ability: the first GDP release is predicted equallywell by models using real-time data as by models using the latest available data. Third, although the first release is a rationalforecast of GDP data after all statistical revisions have taken place, the forecast based on the latest available GDP data (i.e. the“temporarily best” measures) may be improved by combining preliminary official releases with one-step-ahead forecasts.© 2008 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

Keywords: Short-term GDP forecasting; Bridge model; Real-time data-set; Automatic forecasting; First release; Final GDP prediction

1. Introduction

Decision makers in different parts of the economy(business, government, the central bank, financialmarkets, etc) operate in real-time and base their choices

⁎ Corresponding author. Tel.: +39 0 51 2092638; fax: +39 0 512092664.

E-mail addresses: [email protected] (R. Golinelli),[email protected] (G. Parigi).

URL: http://www2.dse.unibo.it/golinelli (R. Golinelli).

0169-2070/$ - see front matter © 2008 International Institute of Fdoi:10.1016/j.ijforecast.2008.05.001

orecaste

on an early understanding of the state of economicactivity, usually measured by GDP. Quarterly GDP dataare available, however, only with a significant delay(from 30 days after the end of the reference quarter in theUSA to 70 days in the euro-area countries), whichweakens their role in short-term policy decisionmaking.The situation is further complicated by the fact thatstatistical institutes revise their GDP estimates severaltimes as more complete information comes in.

To draw an early picture of the evolution of currenteconomic activity – i.e., a real-time GDP estimate – alarge variety of monthly indicators (easily accessible

rs. Published by Elsevier B.V. All rights reserved.

mailto:[email protected]

mailto:[email protected]

http://www2.dse.unibo.it/golinelli

http://dx.doi.org/10.1016/j.ijforecast.2008.05.001

369R. Golinelli, G. Parigi / International Journal of Forecasting 24 (2008) 368–385

and promptly available) may be fruitfully exploited. Inthis context, “real-time” refers to the use of availableinformation on the current quarter for computing earlyestimates of GDP.

Among alternative techniques, the bridge model(BM) approach “translates” the information content ofshort-term indicators into the consistent and complete“language” of national accounts through lineardynamic equations where GDP and/or its componentsare regressed on a set of timely indicators generallyselected on the basis of researchers' experience. FromTrehan (1989) to Golinelli and Parigi (2007), a largebody of literature on BM has shown that, unsurpris-ingly, the current-quarter information content ofindicators results in a real-time GDP estimate that isbetter than the forecasts from traditional univariate andmultivariate competitors. However, this literaturecompares the performance of competing modelsthrough pseudo out-of-sample forecasting exercisesusing the latest available data rather than the real-timedata actually at the forecaster's command (seeCroushore & Stark, 2001). A real-time data-set – i.e.,a collection of data vintages that gives the modeller asnapshot of the macroeconomic data available at anygiven date in the past – makes it possible to take intoaccount the revision process applied by statisticalinstitutes after the first published data. The preliminaryGDP estimate of the same quarter is updated until, aftera number of both statistical and definitional changes, allrelevant information is incorporated and a stablemeasure is reached: the “actual” or final GDP estimatefor that quarter.1 Thus, data revisions imply twopossible alternative targets in prediction: (i) the firstrelease or (ii) the final GDP data.

The ability to forecast the first GDP release iscrucial, given that financial markets react quickly tofirst-issued data, and policymakers base their decisionson early data releases. On the other hand, optimaleconomic decisions depend on an accurate assessmentof the true underlying evolution of the GDP, which

1 Statistical changes stem from additional information that comesin as time passes, and generally concern the most recent quarters.Definitional changes (in the base year and/or due to methodologicalchanges such as different classifications) are more pervasive, occurless frequently (every four to eight years, depending on the countryand the period) and imply a retrospective change of the wholehistorical sample.

should be represented by the final data. The trade-offbetween preliminary and final data may be mitigated iffirst releases are rational in the sense indicated byMuth (1961). However, the analysis of rationality iscomplicated, as the latest available GDP data are notreally final: they themselves will be revised because ofdefinitional changes (see Oller & Hansson, 2004; andCorradi, Fernandez, & Swanson, 2007). In this paper,we propose two measures of the unobservable finalGDP: the latest available GDP time series and theintermediate GDP outturn, i.e., the latest availablevintage immediately before a benchmark revision (fora similar definition, although in a different context, seeKeane & Runkle, 1990).2

In this paper we bring together the strand ofliterature about “real-time” (i.e. early) estimation andthat about “real-time” forecasting, assessing theforecasting practice for predicting alternative targets:preliminary and final data. In doing so, we tackle anumber of topics.

First, adaptive versus non-adaptive automaticmodelling. The selection and specification of allmodels is allowed to change over time using automaticgeneral-to-specific procedures across vintages. In thiscase, only the sample information available at eachdate is used, with no possible leakages from the future.Moreover, adopting such adaptive models should helpavoid the misspecification risks of time-varying GDP-indicator relationships. The models used in this paperrange from the simple, non-adaptive, random walk tomore complex models with indicators; the completelist is in Table 1.

Second, the use of real-time versus the latestavailable data for evaluating the GDP predictiveability. We mimic the real-time forecasting practiceby computing one- and four-quarter ahead GDPpredictions across data vintages. Alternatively, werun the same exercise with the latest available data, asis usually done in the literature. To select the mostaccurate one- and four-step ahead forecast we applythe test for equal accuracy of alternative forecastingmethods proposed by Giacomini and White (2006,

2 In this paper, we classify as “benchmark” the more importantdefinitional changes introduced by the statistical institutes for longspans of retrospective data. Appendix A discusses the Italiannational accounts revision process and gives the motivations for ourspecific definition of benchmark revisions.

Table 1Summary of our forecasting methods

Model-based

RW Random walk with driftARIMA Univariate ARIMA model

Dynamic models with only lagged explanatory (leading) indicatorsLIA general-to-specific reduction from a generalised unrestricted model (GUM) with regressors in differences only if trendingLIB general-to-specific reduction from GUM with regressors in both differences and levels (error-correction specification)LIC general-to-specific reduction from GUM with all regressors in differencesLID general-to-specific reduction from the union of the final LIA, LIB and LIC regressorsLIM simple average of the LIA, LIB and LIC predictions

Bridge Models with both lagged and simultaneous explanatory indicators. Missing simultaneous indicators are predicted using AR(5models (pure forecast)

BMPFA general-to-specific reduction from a generalised unrestricted model (GUM) with regressors in differences only if trendingBMPFB general-to-specific reduction from GUM with regressors in both differences and levels (error-correction specification)BMPFC general-to-specific reduction from GUM with all regressors in differencesBMPFD general-to-specific reduction from the union of the BMPFA, BMPFB and BMPFC regressorsBMPFM simple average of the BMPFA, BMPFB and BMPFC predictions

Bridge Models with both lagged and simultaneous explanatory indicators. One-step-ahead predictions are obtained assuming that alof the simultaneous regressors are known (nowcast)

BMNOA general-to-specific reduction from a generalised unrestricted model (GUM) with regressors in differences only if trendingBMNOB general-to-specific reduction from GUM with regressors in both differences and levels (error-correction specification)BMNOC general-to-specific reduction from GUM with all regressors in differencesBMNOD general-to-specific reduction from the union of the BMNOA, BMNOB and BMNOC regressorsBMNOM simple average of the BMNOA, BMNOB and BMNOC predictions

Target variables (official GDP releases)

FRL First release of GDP data that have never been revisedINT Intermediate outturn: the data that have been subject to many revisions before the occurrence of a benchmark revision. It merges 3

vintages: the 30th for the first block of vintages (published between 1991Q2 and 1995Q4), the 50th for the second block (1996Q1–2000Q4) and the 70th for the third block (2001Q1–2005Q4). See also Appendix A

LA Latest available (fully revised) GDP data. The 75th vintage; see also Appendix A

370 R. Golinelli, G. Parigi / International Journal of Forecasting 24 (2008) 368–385

hereafter GW). Given the availability of model-basedforecasts using both real-time and the latest availabledata, we can investigate whether revised data lead tomodels with better forecasting ability for the prelimin-ary figures (on this topic, see Corradi et al., 2007).

Third, the rationality of the first GDP release. Therationality test of the first release with respect to thefinal one (measured by both the latest available and theintermediate GDP outturn) is conducted by applyingthe test of Fair and Shiller (1990). In this context,model-based forecasts may be seen as an efficientsynthesis of the information set. Moreover, in the caseof rejection of the null of rationality, the Fair-Shiller testgives some guidance on how to complement the firstrelease to improve its final GDP forecasting ability.

This paper is organized as follows. Section 2describes the main methodological issues of real-timemodelling and forecasting GDP with indicators,presents details about the “suite” of models summar-

)

l

ized in Table 1, and defines the settings of theautomatic specification searches. Section 3 analyzesthe one- and four-quarter-ahead out-of-sample perfor-mance of alternative forecasting methods, based onalternative (real-time and latest-available) data. Sec-tion 4 reports the results of a number of rationality testsof the first release against model-based forecasts,interpreted as a parsimonious summary of indicatorinformation. Section 5 concludes. Appendix A givesdetails about the structure of the Italian real-time dataset.

2. The empirical framework

2.1. Methodological issues

The problem of extracting reliable signals fromhigh frequency indicators is not new. Following Klein


and Sojo (1989), the literature can be divided into twobroad methodological approaches: empirical indicatorsand bridge models. The empirical indicator approachleads to factor-based models; see Stock and Watson(2002a,b) and Forni, Hallin, Lippi, and Reichlin(2005). The approach used in this paper, bridgemodels,delivers early GDP estimates by translating theinformation content of short-term indicators into aconsistent and complete picture of the nationalaccounts through linear dynamic equations, where theGDP or its components are explained by suitable short-term indicators, selected on the basis of the researcher'sexperience and statistical testing procedures. For this,bridge models can hardly be automated. For a recentapplication comparing factor-based and BM forecast-ing ability for Italy's industrial production index, seeBulligan, Golinelli, and Parigi (2008).

Early examples of the BM approach can be found inTrehan (1989, 1992) for the USA and in Parigi andSchlitzer (1995) for Italy. Since then, the use of bridgemodels has steadily spread in current short-term GDPforecasting practice; see, among others, Kitchen andMonaco (2003); Runstler and Sedillot (2003); Sedillotand Pain (2003); Baffigi, Golinelli, and Parigi (2004);Klein and Ozmucur (2005), and Golinelli and Parigi(2007). A common finding is that bridge modelsgenerally deliver better current-quarter (i.e., nowcast)GDP estimates than alternative benchmarks, thanks totheir efficient use of statistical information belonging tothe quarter to be forecast (see Golinelli & Parigi, 2007,and, with different techniques, Stark, 2000). However,this finding has been arrived at through pseudo ex-anterecursive forecasts based on data available after theperiod analyzed, and not on the statistical informationtruly available at the time the forecasts were computed.3

This contradicts the opinion of Pesaran and Timmer-mann (2005) that any real-time econometric modelshouldmake use of real-time data in all stages so as not tooverstate the degree of predictability, as shown byDiebold and Rudebusch (1991).

3 Partial exceptions are Diron (2006), who reports results onaggregate euro-area GDP forecasting with real-time data over a veryshort sample (2001–2004) using fixed-specification models, andSchumacher and Breitung (2006), who use factor-based models toforecast German GDP over the period 1999–2005 with real-timedata for GDP and for only a part of the monthly indicators they use.

Actually, the use of real-time data concerns not onlythe model estimation phase, but also the wholemodelling activity, as, in principle, there could be adifferent model for each vintage of data. The modellingapproach in which a new specification is chosen beforeeach forecast is computed is called adaptive, while thenon-adaptive approach involves re-estimating the para-meters of an unchanged specification at each point intime. Swanson and White (1997) find that adaptivemodels estimated over rolling windows perform betterthan fixed-specificationmodels, since theymay limit theeffects of heterogeneity over time and structural change(see Clements & Hendry, 1998, 1999). Stock andWatson (1996) and Giacomini and White (2006) reportevidence of the advantages of the rolling-windowapproach over the recursive approach.

Across vintages of real-time data, parameter shiftsare also induced by GDP definitional changes, becausepart of the historical data are recovered with simplelevel-shift “backcasts” on the basis of earlier vintages,while the entire dynamic structure of the series mayhave changed (see Corradi et al., 2007). In this context,rolling regressions are preferable, as they are lessaffected by past, unreliable, observations than recur-sive approaches.

Our empirical framework tackles all of the above-mentioned methodological issues on the basis of threemain building blocks. The first is a real-time datasetrepresenting the data availability at any given date inthe past. The second consists of a wide range of simpleand complex models to exploit indicator information.The third is a procedure emulating the real-timebehaviour of the modeller.

2.2. The Italian real-time dataset

The GDP real-time dataset is obtained by mergingquarterly GDP data vintages at constant pricesregularly issued by the Italian National Institute forStatistics (Istat), starting from the time series publishedin autumn 1988 for the period from 1970Q1 to1988Q2. The latest available (75th) vintage, publishedin spring 2007, covers the period from 1981Q1 to2006Q4. Seventy-five GDP vintages span sevendefinitional changes and are available over the period1970–2006; see Table A.1 in Appendix A.

The modeller selects the quantitative and qualitativeindicators belonging to the information set on the basis

4 Two alternatives to automatic modelling are either to use“vintage” models built in the past (for the US case, Tetlow &Ironside, 2007, use 30 vintages of the macro model of the FederalReserve Staff), or to keep model specifications fixed over time. Thefirst option is difficult because past models are seldom available (seeBusetti, 2006, for an exception in a different context), while thesecond is the non-adaptive case.5 An 8th order autoregression is fit to the GDP log-differences to

obtain the residuals, then all combinations of AR and/or MAcomponents up to the 4th order are estimated (MA regressors areapproximated by the AR(8) residuals), and the ARIMA model foreach vintage is chosen from the combination minimizing the Akaikecriterion. These ARIMA models are estimated and slightly refinedon the basis of correlogram inspection, residual autocorrelation andparameter significance tests.


of their reliability and timeliness. The indicators listedin Table A.2 of Appendix A are divided into threesubsets (I1, I2 and I3) according to the period in whichthey were first published. The indicators dataset is notorganized in vintages, since these data are never revised,with the sole exception of the industrial productionindex. Therefore, our real-time dataset is limited to GDPat constant prices and the index of industrial production.

However, given that our target variable is quarterly,seasonally adjusted Italian GDP, monthly indicatorshave to be both converted into quarterly averages and,if seasonal, adjusted (using only the time spanavailable at each time), before being used as potentialpredictors. In particular, seasonal indicators belongingto the I1 subset were adjusted using the X11-ARIMAprocedure, those belonging to I2 using X12-ARIMA,and those belonging to I3 using the TRAMO-SEATSprocedure. As a result, though their raw releases arenot subject to revisions, seasonally adjusted indicatorsare also characterized by vintages.

2.3. The alternative classes of models

The dependent variable of each model is the firstdifference of GDP log-levels. It is well known thatmisspecification and estimation errors entail a trade-offin the use of complex versus simple models: the formerreduce the approximation error because they exploitmore information, but they have many parameters to beestimated; the latter may be worse approximations, buthave the advantage of smaller estimation errors.

We use four alternative classes of models thatexploit an increasing amount of information. Theclasses of simple univariate models are our bench-marks: the random walk (RW) and ARIMA models.The leading indicator class (LI) is a GDP dynamicequation with explanatory lagged GDP and laggedindicators. Therefore, with respect to the univariatemodels, LI models exploit additional information frompast indicator data, but exclude the contribution ofsimultaneous indicators. In this sense, LI may beinterpreted as the GDP equation of a parsimoniousVAR (i.e. with parameter restrictions). Finally, bridgemodels are GDP dynamic equations, with past GDPrealizations plus lagged and simultaneous indicators.

There is a growing burden of specification workgoing from the RW to BM classes. The RWwith drift isa fixed-specification model based on a GDP constant-

growth assumption; the class of ARIMA modelsneeds only some more specification searches aboutGDP dynamics; and LI and BM models are obtainedfrom the search for a parsimonious specification basedon the indicators listed in Table A.2 of Appendix A. Inthese two cases the relevance of the researcher'sexperience is self-evident, and any modelling activitymust cope with the problem of how to “model themodeller”without exploiting the advantage of knowinghow the data look ex-post (see Stark & Croushore,2002).

Automatedmodelling and inference is an interestingoption (Phillips, 2005) because it is based onpredetermined rules and uses only the ex-ante informa-tion actually available to the researcher. In other words,automatic model selection guards against futureinformation creeping into the model specification,and thus into the pseudo ex-ante forecasts with real-time data.4 The performance of such automatic model-search procedures has to be taken as a sort of “lowerbound”, since they cannot fully replicate the research-er's modelling ability, i.e. the “art” of forecasting.

2.4. The automatic modeller setup

Among the four types in Table 1, the RW is the onlymodel with a fixed specification; the ARIMA modelvaries by vintage and is chosen according to theHannan and Rissanen (1982) procedure.5 The challen-ging task of finding the best specification of LI andbridge models for each GDP vintage is automatedthrough the general-to-specific search and the relatedencompassing theory, as proposed by London Schoolof Economics (LSE) practitioners. More specifically,


we follow the theory of reduction with the goal ofobtaining a parsimonious, undominated, representa-tion of an initial model – the general unrestrictedmodel (GUM) – that includes all the relevant GDPpredictors (lags of both GDP and its indicators pluscontemporaneous indicators in the BM case). Thespecification search consists of the following steps. Westart from the GUM and check for its congruency withthe data; we then perform a systematic search usingmultiple reduction paths and check for the validity ofeach reduction until the terminal selection. When allpaths have been explored, models are repeatedly testedagainst their union until a final unique congruentmodel is obtained. If several congruent models arefound, the final model can be chosen using selectioncriteria. In this way, we obtain specific LI and BMmodels for each vintage that can be used in h-stepahead GDP forecasts as if the virtual LSE practitionerselected them. Among various mechanical algorithmsreferring to the LSE approach – see Hoover and Perez(1999), Clark (2004) andGiacomini andWhite (2006)–we used the variable-selection and model-reductionprocedure of PcGets (see Hendry & Krolzig, 1999,2003, 2005a,b). Another automatic modelling proce-dure, not based on the general-to-specific approach, isthe RETINA algorithm of Perez-Amaral, Gallo, andWhite (2005). However, the general-to-specificapproach is better designed for macroeconomic timeseries, and is more inclined toward modest sample sizes,see Perez-Amaral et al. (2005) and Castle (2005).

The general-to-specific automatic modellingapproach needs inputs concerning: (a) the list ofregressors, (b) the strategy of the reduction processfrom the GUM, (c) the most appropriate data transfor-mations, (d) the lag length, and (e) the deterministiccomponents.

Regarding point (a), the GDP vintages have beensplit into three blocks corresponding to the majordefinitional changes and indicator data availability,thus reducing the number of potential explanatoryindicators. Given its prominence, the industrialproduction index always forms part of the GUMregressor list. Starting from the complete set ofvintages, we discard the first eleven vintages of thereal-time GDP because indicators with enough obser-vations were lacking (below, we assume a window of80 quarters as the minimum sample for estimation); the75th vintage, being the latest available, is one of our

two definitions of the final GDP. In the first block, theGDP vintages from 12 to 30, classified in ESA 79(base-year 1985), GDP is explained by the fifteenseries in I1. The second block comprises the vintagesfrom 31 to 50: twelve ESA 79 vintages in the base-year1990 and eight incomplete ESA 95 vintages in thebase-year 1995. In principle, the second block could beexplained by both the I1 and I2 information sets, but adegrees-of-freedom constraint would rise. Therefore,the GUM specification in this second block uses theten indicators in I2 plus those in I1 that were retained inthe final models of the last two vintages (29 and 30) ofthe first block. The third block includes the ESA 95vintages from 51 to 74. As above, the regressors in theGUM of this block are the 14 new indicators in the I3set plus those retained in the final models for the lasttwo vintages of the second block (49 and 50).

Regarding point (b), the GUM is reduced to the finalparsimonious specification according to a conservativestrategy, with strict significance levels that minimizethe risk of retaining irrelevant variables and discardingrelevant variables (see Hoover & Perez, 1999; and, fora conflicting view, Campos, Hendry, & Krolzig, 2003).The reduction process involves a wide variety ofmisspecification and parameter restriction tests.

Regarding point (c), although alternative parame-trisations of the same GUM are equivalent specifica-tions, the algorithmic simplification from each re-parametrisation has been shown to yield different finalmodels (Campos & Ericsson, 1999). Adopting the ex-ante transformation of variables fosters the best andmost parsimonious specification. In our “virtual-modeller” experiments, therefore, we adopt fouralternative parametrisations (labelled A, B, C and D)for each vintage, corresponding to alternative datatransformations. Parametrisation A follows the tradi-tional approach of first-differencing only trendingvariables (such as GDP and the industrial productionindex). Parametrisation B corresponds to the error-correction model, with all the regressors specified bothin levels and in first differences. Parametrisation Ctransforms all the variables into first differences.Parametrisation D starts from a GUM obtained bypooling all the regressors of the previous three finalmodels A, B and C (see Hendry & Krolzig, 2005a).

Points (d) and (e) entail less problematic choices. Asis usual with quarterly models, we set the lag lengthequal to 4 for all the variables in the GUM for LI models


and to 3 in the GUM for BM, where the simultaneousrelationships between GDP and its indicators have alsobeen considered. Finally, each GUM always includes aconstant; the GUM with non-stationary levels (para-metrisation B) includes a time trend.

The suite of specific models emerging fromprevious classes and settings may be summarised byassuming, for the sake of simplicity, that only oneindicator x matters (e.g., the log-index of industrialproduction, IPI) and that second-order dynamics areenough to have well behaved residuals. The general-isation to more complex dynamics and to a vector ofindicators is straightforward. For a single vintage ofdata v, the ARDL(2,2) model is the following generalspecification:

Tvþ1Dyt ¼ aþ q1 Tvþ1yt�1 þ q2 Tvþ1yt�2

þ b1 Tvþ1xt�1 þ b2 Tvþ1xt�2

þ g Tvþ1xt þTvþ1et; ð1Þ

where Tv+1Δyt is the quarterly GDP growth rate(proxied by the first difference of log-levels, labelledy), Tv+1xt denotes the IPI log-levels, and Tv+1ɛt is awhite noise process. The subscript on the right of eachvariable denotes the time period of the datum (witht=1, 2, …, Tv), and that on the left the time when itbecomes available (e.g., the one-quarter-delayedavailability for the vintage v is defined as Tv+1).

The RW model implies the restrictions ρ1=ρ2=β1=β2=γ=0; the ARIMA(1,1,0) model implies thatρ1+ρ2=0, and β1=β2=γ=0; the LIA and LIC modelsimply that ρ1+ρ2=0, β1+β2=0, and γ=0; the LIBmodel implies that ρ1+ρ2b0, and γ=0; the BMA andBMCmodels imply that ρ1+ρ2=0, and β1+β2+γ=0;and the BMB model implies that ρ1+ρ2b0. Note that,in this simplified case, the restrictions induced byparametrisations A and C for both LI and BM coincide,because it is assumed that the IPI trending indicator isthe only explanatory variable in Eq. (1).

Adaptive modelling involves the search for andestimation of the ten alternative models (RW, ARIMA,LIA, LIB, LIC, LID, BMA, BMB, BMC and BMD) foreach of the 63 vintages of our real-time data set. Foreach vintage, the parameters are estimated with OLSover rolling windows of 80 quarters. Overall, the“automatic-modeller” does a good job of exploitingindicator information: in the LI- and BM-class models,in-sample GDP predictability is improved by 30–50%

over the RW standard error of the regression. Modelautomatic selection leads to many specificationchanges across vintages. This suggests that in theshort-run both LI and BM entail unstable GDP-indicator relationships. Among the alternative para-metrisations, B has the highest rate of specificationvariability because, in the absence of cointegration(see Golinelli & Parigi, 2007), the inclusion of non-stationary levels leads to models that are more prone toparameter shifts (further information on in-sampleresults is available upon request).

3. The assessment of GDP forecasting practice inreal time

The out-of-sample performance of the models isassessed by a number of forecasting exercises, one andfour quarters ahead, which are designed to mimic thetypical behaviour of practitioners in real-time. On theassumption that revisions improve statistical data, it isthe common practice to search and estimate the modelusing the latest vintage, so that the data set is fullyupdated prior to each new forecast. We go further intothis point in the second part of this section.

The generic h-quarter-ahead forecast using vintagev time series may be defined on the basis of Eq. (1) asfollows:

Tvþhþ1DyTvþh ¼ aþ q1 Tvþ1yTvþh�1

þ q2 Tvþ1yTvþh�2

þ b1 Tvþ1xTvþh�1

þ b2 Tvþ1xTvþh�2

þg Tvþ1xTvþh

þTvþhþ1eTvþh:

ð2Þ

Eq. (2) suggests three basic points. First, GDPprediction errors one and four quarters ahead (h=1 and4) from a vintage v (its historical data end in Tv and arefirst released in Tv+1) require data published, respec-tively, one and four quarters later than when theforecast is made. For example, the first regression usesthe 11th GDP vintage to estimate model parametersover the period 1971Q2-1991Q1. Then, the resultingmodels are used to forecast the 12th GDP vintage (onestep ahead) and the 15th GDP vintage (four stepsahead). Second, when hN1, the h-step-ahead modelsimulation of Eq. (2) is that of the iterated one-step-


ahead forecasts. Third, when h=1 the vintage v timeseries (t=1, 2, ..., Tv) is enough to forecast GDP withthe RW, ARIMA and LI models as they assume γ=0,while BM requires indicator data for the quarter to beforecast; when h=4, we need indicator predictionsfour quarters ahead (for all LI- and BMPF-classmodels), or three quarters ahead (for BMNO-classmodels).

Table 2Alternative GDP forecasts

Horizon: One-step-ahead (from 1991 Q2 to 2006 Q4)

Data used:1 RT LA

Models 2 RMSE 3 RMSE 3 Rati

RW 0.61 0.56 1.09ARIMA 0.63 0.47 1.33LIA 0.63 0.52 1.20LIB 0.59 0.54 1.08LIC 0.61 0.59 1.02LID 0.58 0.53 1.09LIM 0.53 0.50 1.05BMPFA 0.69 0.57 1.21BMPFB 0.66 0.48 1.37BMPFC 0.67 0.56 1.18BMPFD 0.67 0.53 1.25BMPFM 0.65 0.51 1.27BMNOA 0.44 0.43 1.02BMNOB 0.44 0.34 1.30BMNOC 0.38 0.43 0.89BMNOD 0.45 0.41 1.09BMNOM 0.38 0.36 1.05

Ratio between the RMSE of each model in the row to that of RW 5

ARIMA 1.037 0.850⁎⁎

LIA 1.031 0.936LIB 0.967 0.975LIC 1.001 1.067LID 0.945 0.948LIM 0.862 0.898BMPFA 1.132⁎ 1.024BMPFB 1.085 0.863⁎

BMPFC 1.096 1.011BMPFD 1.090 0.951BMPFM 1.061 0.915⁎⁎

BMNOA 0.722⁎⁎⁎ 0.774⁎⁎⁎

BMNOB 0.729⁎⁎⁎ 0.612⁎⁎⁎

BMNOC 0.622⁎⁎⁎ 0.766⁎⁎

BMNOD 0.732⁎⁎⁎ 0.730⁎⁎⁎

BMNOM 0.619⁎⁎⁎ 0.643⁎⁎⁎

1 RT = real-time; LA = latest-available. 2 For a description see Table 1. 3 RMbetween RTand LA RMSEs. 5 ⁎, ⁎⁎, and ⁎⁎⁎ indicate significance at the 10%test for equal conditional predictive ability.

At the time the forecast is made, monthly indicatordata may cover either all of the months of the quarter tobe predicted (nowcast, BMNO) or none (pure forecast,BMPF). When indicator observations are missing forone or two months, they must be extrapolated to fill thequarter using auxiliary models. Simple AR(5) modelsare used as auxiliary models (alternative AR orders donot significantly change the results). As was done by

Four-step-ahead (from 1992 Q1 to 2006 Q4)

RT LA

o 4 RMSE 3 RMSE 3 Ratio 4

7 1.54 1.81 0.8528 1.61 1.84 0.8758 1.73 1.68 1.0258 1.23 1.79 0.6898 1.61 1.92 0.8404 1.45 1.78 0.8143 1.16 1.64 0.7052 1.64 2.01 0.8139 1.58 1.54 1.0209 1.52 2.01 0.7577 1.55 1.78 0.8761 1.49 1.77 0.8412 1.55 1.85 0.8355 1.38 1.32 1.0490 1.45 1.78 0.8159 1.51 1.58 0.9535 1.37 1.55 0.878

1.041 1.0131.119 0.9300.798⁎ 0.9861.046 1.0610.937 0.9810.750⁎⁎ 0.9061.061 1.112⁎⁎

1.021 0.852⁎⁎

0.988 1.112⁎⁎

1.008 0.9800.965 0.9781.003 1.0220.897 0.729⁎⁎⁎

0.938 0.9800.976 0.872⁎⁎

0.885⁎ 0.858⁎⁎⁎

SE expressed in % over the forecast period indicated above. 4 Ratio, 5%, and 1% levels respectively of the Giacomini andWhite (2006
)


Baffigi et al. (2004), we take only extreme cases ofBMNO and BMPF; this entails no loss of generality, asis shown by the results of Clements and Galvao (inpress) for the USA, and Golinelli and Parigi (2007) forthe G7 countries.

Overall, on the basis of ten forecasting models, wehave 28 GDP predictions (one and four steps ahead),of which eight are nowcasts. We also computed theaverages of the A, B, and C predictions of the LI andBM models to check whether the ‘average’ puzzle (seeHendry & Clements, 2002; and Stock & Watson,2004) applies in our context; this raises the totalnumber of GDP forecasts to 34.

The same real-time exercises are computed usingthe latest available data (in our context the 75thvintage), which is what is usually found in theliterature on the assessment of the forecasting abilityof alternative models through pseudo ex-ante exer-cises. In other words, in Eqs. (1) and (2) we substitutedf for the left subscripts Tv+1 and Tv+h+1, respec-tively, and used the latest available time series for the“final” data.

In each column, the upper part of Table 2 reports theroot mean squared errors (RMSE) of alternative GDPpredictions, while the lower part reports the RMSEratios with respect to the RW,with the results of the GWtest for equal predictive ability.6 Different columns

6 The GW test is the most appropriate in the present contextbecause: (i) it is valid for limited-memory forecasting methods (e.g.rolling window forecasts) that are fully consistent with our adaptiveframework; (ii) it is valid under very general data assumptions(including non-constant data generating processes, as for forecastingwith indicators); and (iii) it can handle comparisons of both nestedand non-nested models. For nested models, tests for equal RMSE,as proposed by Diebold and Mariano (1995), are not normallydistributed and should be corrected. However, these adjustmentsrefer to non-adaptive forecasting models estimated with non-reviseddata (for preliminary advances, see Clark & McCracken, 2007). Thenull hypothesis of the GW test implies that one cannot predict whichforecasting method will be more accurate at the forecast target datet+h using the information set available at time t. Under the nullhypothesis, the test is distributed as c2 with q degrees of freedom; qis the dimension of the test function that ranges from the simplestcase where only the constant is contained (q=1), to cases whereother variables (including lags) are included in order to helpdistinguish between the forecasting performance of the two methods(qN1). Here, we assumed q=1, but test results with q=2 aresimilar, and are available upon request.

report the real-time and latest available outcomes overone- and four-quarter forecast horizons.

Table 2 shows that the simultaneous indicatorinformation significantly improves BMNO perfor-mance one step ahead with respect to the otherforecasting methods: the RMSE of BMNO is 20–40% lower than that of any of the other methods (andthe difference is statistically significant), as is typicallyfound in the literature on BM using the latest availabledata (see Trehan, 1989, 1992, for the USA; Parigi &Schlitzer, 1995, for Italy; Baffigi et al., 2004, for theeuro area; and Golinelli & Parigi, 2007, for the G7countries). In the following column, the results of thesame exercise using the latest available data stronglyconfirm the finding.

When indicator data for the quarter to be forecastare not known, bridge models lose their outstandingpredictive ability. In fact, BMPF performance isusually worse than that of models that do not exploitsimultaneous information, because – without monthlyindicator data for the quarter to be forecast – they arejust a more complex way (i.e., using more parameters)to exploit the same past information as is in the LI andARIMA models. For the same reason, the BMNOforecasting ability tends to vanish over longerhorizons: four steps ahead, only the simple averageof LI A, B and C forecasts (LIM) significantlyimproves the RW performance at the 5% level (the“average puzzle” works also with real-time data).

The use of the latest available vintage weakens thefindings on the good performance of the LI modelsfour quarters ahead with real-time data and, moregenerally, worsen the ability of all of our models topredict GDP (the RMSE ratios in the last column inTable 2 are less than one; the GW test cannot becomputed here as the target is different). As was foundby Diebold and Rudebusch (1991), the latest availabledata would overstate the BMNO's ability to improvethe performance of RW.

So far we have assumed that practitioners forecastthe first GDP release with mixed-release vintages(supposed to be of better quality) and specify a newmodel every quarter, i.e., for each vintage. Corradiet al. (2007) show the sub-optimality of this practice,as they obtain better predictions by using only first-release GDP data. Though we cannot replicate theseresults for Italy as the number of vintages is too small,we can exploit previous GDP forecasts obtained using


real-time and latest available data to provide someevidence on this point. More specifically, if it is truethat higher quality (revised) data allow better forecastsof the first GDP release, then predictions based on thelatest-available data should be better.

At the same time, we tackle the issue of adaptive/non-adaptive modelling approaches, given the highvariability of automatic-model specifications. In thenon-adaptive approach, the model specification is keptfixed within two consecutive benchmark revisions. Inparticular, we label the use of the specification forv=11 across data vintages until v=29, the specifica-tion for v=30 until v=49, and the specification forv=50 until v=63 as non-adaptive.

Table 3 compares the RMSEs of one- and four-quarter ahead predictions of the first GDP releaseobtained from real-time adaptive modelling (thebaseline forecast) with those from real-time non-adaptive modelling and those from the latest availabledata under both the adaptive and non-adaptiveapproaches (the columns report the RMSE of thebaseline, and of the three alternatives expressed asratios of the baseline RMSE).

The use of the latest available data with adaptivemodelling does not substantially improve the baselineone-step-ahead forecast (in the case of BMNO modelsthere is a statistically significant deterioration of about20%), while in the four-steps-ahead case the BMNOmodels' forecasting ability improves with respect tothe baseline. Although pseudo ex-ante forecasts usingthe latest available data do not provide compellingevidence in favour of using revised data for theprediction of the first GDP release, they basicallyconfirm the results of Corradi et al. (2007).

According to Table 3, adaptive and non-adaptivemodels give equivalent results, probably becauseautomated searches may oversimplify the researcher'sreal-time specification activity.

7 In principle, any other data release might be used; see Swansonand van Djik (2006).

4. The forecasting ability and rationality of thefirst release relative to final GDP

Economic decision-making depends on an accurateassessment of the actual state of the economy, which inthe short-run can only be based on preliminary data. Itis therefore relevant to test whether these first releasesof data are rational in the sense of Muth (1961). A

crucial point is the definition of the “final” GDP,supposedly no longer subject to revision. We have twoalternatives. The first is to approximate the “final”GDPwith the latest available data; in this case, the differencewith respect to the first release (a sort of the “forecasterror” of the latter) embodies both statistical anddefinitional revisions. The second is to use theintermediate GDP outturn to proxy “final” GDP; thisdiffers from the first release only because of statistical(non-benchmark) revisions.

A huge body of literature has tested the rationalityof early GDP estimates in the light of the “final” state(see Swanson & van Djik, 2006, and Corradi et al.,2007, for recent evidence). In this paper we try tomerge this literature with the classical forecast test ofFair and Shiller (1990). This will allow us to assess,within the same framework, the rationality of the firstGDP release, and, if it is rejected, the appropriatenessof a combination with the outcome of forecastingmethods to improve the estimate of the “final” GDP.

Using the same notation as in Eq. (2), the Fair-Shiller test equation can be written as:

f YTvþh �f YTvf YTv

¼ aþ b Tvþhþ1YTvþh �Tvþhþ1 YTvTvþhþ1YTv

þ gTvþ1Y

pTvþh �Tvþ1 YTv

Tvþ1YTvþ f eTvþh; ð3Þ

wheref YTvþh� f YTv

f YTv is the h-step GDP growth, i.e., theactual state of the economy (Y are GDP levels, the fsubscript on the left indicates any sort of final GDP

measure); Tvþhþ1YTvþh� Tvþhþ1YTvTvþhþ1YTv

is the corresponding

first release of the h-step growth;7Tvþ1Y

pTvþh� Tvþ1YTvTvþ1YTv

is the h-step growth of the GDP forecast using vintagev information available at Tv+1 and obtained from ageneric forecasting method p; f eTvþh is a stochasticdisturbance, autocorrelated for hN1 and possiblyheteroskedastic. The constant α allows for comparisonbetween biased forecasts. Under the null hypothesisγ=0, the forecasting method p contains no informationrelevant to the forecast h quarters ahead of the actual

Table 3Forecasting the first GDP release

Horizon One-step-ahead (from 1991 Q2 to 2006 Q4) Four-steps-ahead (from 1992 Q1 to 2006 Q4)

Specification Adaptive Non-adaptive Adaptive Non-adaptive

Data used 1 RT LA RT LA RT LA RT LA

Models 2 RMSE 3 Ratios 4 RMSE 3 Ratios 4

RW 0.61 1.007 1.000 1.007 1.54 1.030 1.000 1.030ARIMA 0.63 0.927 1.053 0.998 1.61 1.000 1.057 1.131⁎

LIA 0.63 0.993 1.148 1.074 1.73 0.838 1.291 1.078LIB 0.59 1.072 0.940 1.133 1.23 1.266⁎⁎ 0.880 1.401⁎⁎

LIC 0.61 1.052 1.028 0.957 1.61 1.063 0.928 0.752⁎⁎

LID 0.58 1.099 1.145 1.150 1.45 1.069 1.331⁎⁎ 1.234⁎

LIM 0.53 1.116 1.069 1.138 1.16 1.204⁎ 1.216⁎⁎ 1.224⁎

BMPFA 0.69 0.948 0.930⁎ 1.026 1.64 1.099⁎⁎ 0.983 1.194⁎⁎

BMPFB 0.66 0.865⁎ 0.976 0.964 1.58 0.820⁎ 0.973 0.979BMPFC 0.67 0.895 0.922⁎⁎ 0.933 1.52 1.129⁎⁎⁎ 0.981 1.093BMPFD 0.67 0.941 1.003 0.993 1.55 0.984 1.028 1.010BMPFM 0.65 0.901 0.962 0.995 1.49 1.010 0.999 1.139⁎⁎

BMNOA 0.44 1.285⁎⁎ 0.903⁎ 1.445⁎⁎⁎ 1.55 1.044 0.990 1.173⁎

BMNOB 0.44 1.145 0.925 1.274 1.38 0.790⁎⁎ 1.030 0.977BMNOC 0.38 1.353⁎⁎⁎ 1.065 1.448⁎⁎ 1.45 1.025 0.993 0.979BMNOD 0.45 1.237⁎ 0.960 1.376⁎⁎ 1.51 0.881 0.996 0.953BMNOM 0.38 1.319⁎⁎ 1.005 1.501⁎⁎⁎ 1.37 0.939 1.031 1.0971RT = real-time; LA = latest-available. 2For a description see Table 1. 3Baseline RMSEs expressed in % over the forecast period indicated above.4RMSE ratio with respect to the baseline (i.e. adaptive model using real time data); ⁎, ⁎⁎, and ⁎⁎⁎ indicate significance at the 10%, 5%, and 1%levels, respectively, of the Giacomini and White (2006) test for equal conditional predictive ability.


GDPgrowth that is not already in the first release; for thealternative null hypothesis β=0 the converse holds.

Using the notation of Eq. (3), the traditional test forthe rationality of the first release for Tv+h is thefollowing expression:

f YTvþh � f YTvf YTv

¼ aþb Tvþhþ1YTvþh � Tvþhþ1YTvTvþhþ1YTv

þW VTvþhþ1gþ f eTvþh; ð4Þ

where W is a vector of m variables representing theconditioning information set available at time Tv+h+1,and γ is the corresponding vector of parameters. InEq. (4), the first GDP release is rational under the nullhypothesis α=0, β=1, γ=0. Therefore, the interpreta-tion of the Fair and Shiller's test Eq. (3) as a rationalitytest for the first GDP release implies that the predictionof the forecasting method p represents the conditioninginformation set available at time Tv+h+1 (i.e., thetime of the first release). Under the null of rationality,f eTvþh in Eq. (3) represents the unpredictable news

unrelated to the release, see Faust, Rogers, and Wright(2005). Comparing Eqs. (3) and (4), the Fair-Shiller testis a realistic assessment of the rationality of the firstrelease only when h=1, because for hN1 the informa-tion set of method p is lagged by at least h quarters withrespect to the release.

The rationality test using Eq. (3), i.e., the Fair-Shiller test, enjoys a number of advantages over thetraditional rationality test as in Eq. (4). First, it is lessarbitrary, as the long list of variables representing theconditioning information set at Tv+h+1 (e.g., all theindicators in Table A.2) is summarised by themodelling/forecasting activity, instead of being simplyreduced to m without any clear guidance. Second, it ismore efficient, since in Eq. (3) the measure of theeffect of the conditioning information set is based onthe estimate of the γ parameter, while in Eq. (4) itdepends on the estimate of the m parameters in the γvector. Third, it is less prone to breaks than Eq. (4), asthe forecast from method p is obtained by modellingthe information with adaptive methods over rolling


samples; in this way, the breaks affecting W (seeSwanson & van Djik, 2006) are partially filtered outbefore entering the test equation. Fourth, the rejectionof the null hypothesis γ=0 may be interpreted as asuggestion to combine the first release with the fore-cast obtained from p in order to improve the estimation

Table 4Fair–Shiller and rationality tests of the first release

Target variable 1 INT = intermediate GDP outturn

Eq. ( 3) estimates 2 p-value of

method p 4 αα ̂ β ̂̂ γ ̂ γ=0 β

γ

RW −0.0242 0.9358 0.1379 0.819 0(0.2777) (0.0919) (0.5998)

ARIMA 0.0048 0.9321 0.0997 0.274 0(0.0752) (0.1004) (0.0902)

LIA 0.0301 0.9083 0.0614 0.302 0(0.0628) (0.1068) (0.0589)

LIB 0.0658 0.9558 −0.0607 0.620 0(0.0897) (0.0756) (0.1215)

LIC 0.0896 0.9535 −0.1137 0.321 0(0.0890) (0.0860) (0.1135)

LID 0.0680 0.9597 −0.0618 0.621 0(0.0899) (0.0730) (0.1243)

LIM 0.0651 0.9469 −0.0441 0.727 0(0.0925) (0.0810) (0.1258)

BMPFA 0.0512 0.9328 0.0008 0.995 0(0.0997) (0.1027) (0.1236)

BMPFB 0.0687 0.9340 −0.0385 0.792 0(0.1224) (0.0938) (0.1456)

BMPFC 0.0667 0.9306 −0.0327 0.778 0(0.0718) (0.0973) (0.1152)

BMPFD 0.0845 0.9337 −0.0661 0.704 0(0.1400) (0.0937) (0.1731)

BMPFM 0.0713 0.9310 −0.0409 0.808 0(0.1218) (0.1016) (0.1671)

BMNOA −0.0076 0.7669 0.2393 0.053 0(0.0482) (0.1503) (0.1210)

BMNOB 0.0273 0.8119 0.1621 0.058 0(0.0693) (0.1105) (0.0838)

BMNOC 0.0166 0.7746 0.2116 0.197 0(0.0463) (0.1972) (0.1622)

BMNOD 0.0132 0.8065 0.1688 0.068 0(0.0743) (0.1093) (0.0905)

BMNOM 0.0009 0.7359 0.2703 0.057 0(0.0530) (0.1665) (0.1389)

1 Definitions are in Table 1; see also Appendix A. 2 HAC standard errors afirst release encompasses the forecasting method in this row (see Fair & Srational forecast of the target variable, conditional on the forecasting method

of the actual GDP. The null of the Fair-Shiller testimposes looser restrictions on Eq. (3) parameters thanthose entailing the rationality of the first release: thenon-rejection of γ=0 in Eq. (3) only implies the supe-riority of the information exploited by the statisticalagency with respect to that available to the modeller.

LA = latest available GDP vintage

3 Eq. ( 3) estimates 2 p-value of 3

=0 α=0=1 α ̂ β ̂̂ γ ̂ γ=0 β=1=0 γ=0

.766 0.4895 0.5732 −0.6074 0.350 0.008(0.3473) (0.1215) (0.6446)

.617 0.0791 0.5888 0.1758 0.319 0.018(0.1074) (0.1254) (0.1749)

.714 0.0891 0.5144 0.2072 0.017 0.003(0.0664) (0.1322) (0.0843)

.882 0.1232 0.5397 0.1506 0.196 0.001(0.0827) (0.1206) (0.1152)

.723 0.1285 0.5738 0.0964 0.471 0.015(0.0675) (0.1278) (0.1330)

.883 0.1074 0.5121 0.1928 0.075 0.001(0.0829) (0.1249) (0.1065)

.889 0.0743 0.5084 0.2810 0.028 0.000(0.0781) (0.1220) (0.1250)

.888 −0.0959 0.6438 0.4804 0.001 0.001(0.0782) (0.1096) (0.1341)

.804 0.0591 0.5899 0.2235 0.123 0.005(0.0979) (0.1274) (0.1430)

.827 0.0688 0.6030 0.1950 0.180 0.019(0.0913) (0.1246) (0.1438)

.853 −0.0013 0.5950 0.3279 0.047 0.000(0.1156) (0.1363) (0.1613)

.890 −0.0521 0.6158 0.4379 0.024 0.003(0.1065) (0.1233) (0.1893)

.195 0.0908 0.3994 0.2892 0.086 0.005(0.0684) (0.1873) (0.1659)

.246 0.1292 0.4582 0.1907 0.158 0.014(0.0680) (0.1849) (0.1334)

.569 0.1330 0.4770 0.1554 0.389 0.006(0.0676) (0.2300) (0.1791)

.242 0.1022 0.4107 0.2574 0.063 0.014(0.0684) (0.1824) (0.1358)

.240 0.1053 0.3922 0.2855 0.136 0.009(0.0668) (0.2180) (0.1887)

re in brackets (see Newey & West, 1987). 3 Under the null γ=0, thehiller, 1990); under the null α=0, β=1, γ=0, the first release is ain this row. 41-step-ahead forecasts from themethods listed in Table 1
.


Table 4 reports some OLS estimates of Eq. (3)parameters by measuring the growth rate of final GDPwith both the intermediate outturn and the latestavailable vintage. The reported test statistics for γ=0and the rationality test for α=0, β=1, γ=0 are allbased on heteroskedasticity and autocorrelation-con-sistent standard error estimators (see Newey & West,1987). The results are robust to alternative estimates ofthe variance-covariance matrix, based onWhite (1980)and bootstrapped standard errors.

The outcomes in the left hand columns never rejectthe null of rationality of the first release for theintermediate outturn. As would be expected in light ofthe discussion above, the results also suggest that nomodels except BMNO carry information that is usefulfor forecasting regular GDP revisions that is notalready in the first release. The results in the right handcolumns clearly reject the rationality of the first releasefor the latest available data. Moreover, the first releasedoes not fully encompass the information set ofindicators exploited by the automatic modeller: thenull is rejected in eight out of fifteen cases. TheBMNO no longer performs best, as the simultaneousindicator information is not as useful as leadingindicators in improving forecast benchmark revisions.

The Andrews (1993) sup-Wald tests for structuralchange never reject the null hypothesis for Eq. (3),supporting our intuition about the unlikeliness ofstructural breaks with this specification. RESET testoutcomes, also not reported, never reject the null of alinear functional form when second powers of fittedvalues are included, but they always reject it whenboth second and third powers are included, suggestingthe possible presence of cyclical relationships.

In general, regular revisions are events that cannot beexplained by the forecasts of the automatic modeller.Instead, benchmark revisions imply substantial defini-tional changes that are not fully accounted for by the firstrelease, but that can be further explained by the richdynamic structure of our automatic models. In thesecircumstances, it is advisable to combine the model-based predictions with the first GDP release as soon aspreliminary data are issued. For example, in preliminaryexperiments, simple averages of the first release and ofthe forecasts made using different methods can lead to10–20% RMSE improvements in explaining the latestavailable data. RESET test outcomes may entailrelationships evolving over the cycle. This point,

potentially useful for finding the optimal combinationof themodel-based forecasts and the first GDP release, isleft to future research.

5. Concluding remarks

We have analyzed the practical relevance of theliterature on evaluating the forecasting performance ofalternative models, using real-time data to mimic real-time activity. Through automatic general-to-specificadaptive model searches, we are able to obtainalternative GDP forecasts one and four steps aheadusing both the latest available and real-time datasets(corresponding to the best GDP measure and to theinformation truly available at the time when the forecastis made, respectively). There were three main findings.

First, the availability of indicators for the quarter tobe forecast significantly improves one-step-aheadGDP predictions in the case of both real-time andlatest available data. However, the relevance ofindicators tends to vanish at longer forecastinghorizons. These findings support the results of theliterature on using only the latest available data.

Second, the common practice of using fullyupdated datasets prior to each prediction is motivatedby the assumption that the latest data embodies the bestinformation. This assumption is not corroborated byour evidence; in fact, the forecasts of the first GDPrelease obtained from real-time data and from the latestavailable data do not differ significantly.

Third, the first GDPdata released appear to be rationalforecasts of the intermediate outturn (i.e., no longersubject to further statistical revisions), but not of the latestavailable data. In this case, forecasts may be improved bysimple combinations of the first release and thecorresponding model-based one-step-ahead predictions.

Acknowledgments

A preliminary draft was presented at the 27thInternational Symposium on Forecasting of theInternational Institute of Forecasters (New York City,June 24–27, 2007); we are grateful to Dean Croushore,Lars-Erik Oller, Adrian Pagan, Norman Swanson,Halbert White, and the participants for helpfulcomments. This paper has greatly benefited from thesuggestions of an associate editor of the International

Fig. A1. GDP growth rates across all data vintages.


Journal of Forecasting and two anonymous referees.The opinions it expresses are the authors' only, and donot necessarily reflect those of the institutions forwhich they work. PRIN financing is grateful acknowl-edged (R. Golinelli).

Appendix A

A1. Italian GDP vintages

From 1988 to 2006, Italian national accountsexperienced seven definitional changes:

(1) base-year change from 1980 to 1985, sincev=12;

(2) base-year change from 1985 to 1990, sincev=31;

(3) preliminary change from SEC79 to SEC95;base-year change from 1990 to 1995, sincev=43;

(4) complete retrospective SEC 95 data, from 1970Q1, since v=51;

(5) GDP unit proportionately transformed into theeuro, since v=55;

(6) GDP levels adjusted for trading days, sincev=60;

(7) base-year change from 1995 to chain index,since v=71.

We label themore relevant of the definitional revisionswhich the Istat published, together with large spans ofretrospective data, as benchmark revisions. The mainfeatures of the definitional changes are summarised inTable A.1. In order to visually inspect the relevance of therevisions, each line in Fig. A.1 denotes the quarterlygrowth rates for a given calendar date, as reported acrossall vintages.

A2. An overview of the Italian GDP revision process

In this section we focus only on the GDP vintagesfrom 12 to 75, i.e., the span used in all of the forecastingexercises: 63 vintages published from 1991Q2 to2007Q1. Amore general analysis of the revision processfor Italian quarterly national accounts can be found in DiFonzo, Pisani, and Savio (2002) and in Lupi andPeracchi (2003). In order to allow for comparisonsamong heterogeneous and non-stationary GDP vintages,

we transformed the levels by vintage in quarterly growthrates. The main features of the GDP revision process aresummarised by the following three definitions (orderedby the number of revisions they incorporate):

FRL The first release: i.e., data that have neverbeen revised;

INT The intermediate outturn: data that have beensubject to many regular revisions before theoccurrence of a benchmark revision. INTmerges 3 vintages: the 30th for the first blockof vintages (published between 1991Q2 and1995Q4), the 50th for the second block(1996Q1-2000Q4) and the 70th for the thirdblock (2001Q1-2005Q4), see also Table A.1;and

LA Latest available: i.e., fully revised data (the75th vintage).

The introduction of INT requires a shift from thestatistical and definitional classification of revisions toa regular and benchmark classification (based ontiming and relevance). Consequently, the completerevisions (i.e., the difference between LA and FRL)can be decomposed into two parts: the regularrevisions (the difference between INT and FRL) andthe benchmark revisions (the difference between LAand INT). A number of observations can be made on


the basis of the four plots of Fig. A.2 regarding themain features of FRL, INT and LA, and of thecorresponding regular and benchmark revisions.

On the basis of the first plot, comparing the quarterlyGDP growth of the first release with that of the latestavailable data, we define the statistical institute reportingthe first release as conservative when the grey LA areacompletely covers the black FRL area; this occurredmore frequently after 1996, during the second part of thesample.

In light of the box-plot on the right, we note that, overthe sample, the mean and median of the revisions arealways very close to each other, indicating symmetrical

Fig. A2. Main features

revisions; the rejection of the revision normality test istherefore due to the presence of outliers (mainly concen-trated in the regular revision case), rather than asymmetry.

The last two plots in Fig. A.2 suggest that thecomplete revision variance tends to smooth over time,in line with the tendency of both FRL and LA growthrate variability to decrease. This could mean animprovement in data collection and reporting methods.Further, on the basis of the plot on the left, where threesub-periods inside two consecutive benchmark revi-sions are shown, the statistical institute appears to havebeen systematically optimistic only during the secondhalf of the 90s.

of data revisions.

Table A.1Definitional revisions in Italy

Blocks ofvintages1

Classification Measurement Base Sample period Tradingdays

Start2 End3

From 1 to 11 SEC 79 Billion lire 1980 1970Q1 1988Q2 Not adjustedFrom 12 to 30 SEC 79 Billion lire 1985 1970Q1 1991Q1 Not adjustedFrom 31 to 42 SEC 79 Billion lire 1990 1970Q1 1995Q4 Not adjustedFrom 43 to 50 SEC 95 Billion lire 1995 1982Q1 1998Q4 Not adjustedFrom 51 to 54 SEC 95 Billion lire 1995 1970Q1 2000Q4 Not adjustedFrom 55 to 59 SEC 95 Million euro 1995 1970Q1 2001Q4 Not adjustedFrom 60 to 70 SEC 95 Million euro 1995 1980Q1 2003Q1 AdjustedFrom 71 to 754 SEC 95 Million euro Chain index 1981Q1 2005Q4 Adjusted1 Each block includes all of the GDP vintages between two consecutive definitional revisions. The most relevant revisions, which this paper labelsas benchmark revisions, are in bold. 2 Vintages starting later than 1970Q1 were “backcasted” by using the growth rates of the previous vintage;therefore all vintages used in the paper start in 1970Q1 (defined as t=1). 3 End-date of the first vintage in the block (defined in the first column);in the paper the different vintages' end dates are conventionally defined as t=Tv.

4 End-date of the latest available (75th) vintage is 2006Q4.

Table A.2Italy: the three subsets of indicators, I1, I2 and I3

The first subset of indicators (I1): first observation available before1970xpv Vintages of the industrial production index (logs of index

levels) 1

glav Working days (log-levels)dpe Stock prices growth rate (first-differences of log-prices)dpoil Oil price growth rate (first-differences of log-prices)R3m Nominal 3-month interest rate (levels)Q01 Rate of capacity utilisation (log-levels)X3 Orders level: total manufacturing 2

X118 Passenger cars registered (log-levels)X13 Finished goods stocks: total manufacturing 2

X18 Production actual tendency: intermediate goods 2

X19 Production actual tendency: investment goods 2

X20 Production actual tendency: consumption goods 2

X21 Orders tendency (3-4 months expected demand): totalmanufacturing 2

X26 Production future tendency (3–4 months): intermediategoods 2 3

X27 Production future tendency (3–4 months): investmentgoods 2 3

X28 Production future tendency (3–4 months): consumptiongoods 2 3

The second subset of indicators (I2): first observation availablebefore 1974 and after 1972

X6 Orders level: intermediate goods 2

X9 Orders level: investment goods 2

X12 Orders level: consumption goods 2

X14 Finished goods stocks: intermediate goods 2

X15 Finished goods stocks: investment goods 2

X16 Finished goods stocks: consumption goods 2

X22 Orders inflow/demand tendency (3–4 months):intermediategoods 2 3

(continued on next page)

Table A.2 (continued )

The first subset of indicators (I1): first observation available before1970X23 Orders inflow/demand tendency (3–4 months): investment

goods 2 3

X24 Orders inflow/demand tendency (3–4 months): consumptiongoods 2 3

Q10 Consumer confidence (logs of index levels)

The third subset of indicators (I3): first observation available before1980 and after 1978

X34 Future tendency of the economy, 3–4 months (Isae):intermediate goods 2

X35 Future tendency of the economy, 3–4 months (Isae):investment goods 2

X36 Future tendency of the economy, 3–4 months (Isae):consumption goods 2

X55 Production actual tendency (Isae): investment goods 2 3

X61 Production actual tendency (Isae): consumption goods 2 3

X67 Production actual tendency (Isae): intermediate goods 2 3

X50 Orders actual tendency (Isae): investment goods 2 3

X56 Orders actual tendency (Isae): consumption goods 2 3

X62 Orders actual tendency (Isae): intermediate goods 2 3

Q11 Manufacturing and construction confidence, weightedaverage (logs of index levels)

reale Real interest rate on bank loans (levels)delfs Electricity consumption of the Italian railway company

(first-differences of log-levels) 3

delt Total consumption of electricity (first-differences oflog-levels) 3

delsouth Electricity consumption of the southern Italian regions(first-differences of log-levels) 3

1Only the industrial production database is organised in vintagesand issued seasonally adjusted by ISTAT, while survey and financialdata are not subject to revisions. 2xi=log(1+IMi/100), where IMi are

X23 Orders inflow/demand tendency (3–4 months): investmentgoods 2 3

X24 Orders inflow/demand tendency (3–4 months): consumptiongoods 2 3

Q10 Consumer confidence (logs of index levels)

The third subset of indicators (I3): first observation available before1980 and after 1978X34 Future tendency of the economy, 3–4 months (Isae):

intermediate goods 2

X35 Future tendency of the economy, 3–4 months (Isae):investment goods 2

X36 Future tendency of the economy, 3–4 months (Isae):consumption goods 2

X55 Production actual tendency (Isae): investment goods 2 3

X61 Production actual tendency (Isae): consumption goods 2 3

X67 Production actual tendency (Isae): intermediate goods 2 3

X50 Orders actual tendency (Isae): investment goods 2 3

X56 Orders actual tendency (Isae): consumption goods 2 3

X62 Orders actual tendency (Isae): intermediate goods 2 3

Q11 Manufacturing and construction confidence, weightedaverage (logs of index levels)

reale Real interest rate on bank loans (levels)delfs Electricity consumption of the Italian railway company

(first-differences of log-levels) 3

delt Total consumption of electricity (first-differences of log-levels) 3

delsouth Electricity consumption of the southern Italian regions(first-differences of log-levels) 3

1Only the industrial production database is organised in vintagesand issued seasonally adjusted by ISTAT, while survey and financialdata are not subject to revisions. 2xi=log(1+IMi/100), where IMi arequarterly averages of monthly survey data, and i is the code reportedin the first column. 3Seasonally adjusted with X11 ARIMA, X12ARIMA or TRAMO-SEATS, if belonging to the I1, I2 or I3 subset,respectively.



A3. The data set of indicators

The variables belonging to the indicators' informa-tion set have been divided into three subsets (I1, I2 andI3) according to their availability, see Table A.2.

All indicators available before 1970 belong to thefirst subset (I1); the indicators with data availablebefore 1974 (and after 1972) belong to the secondsubset (I2); and the indicators with data available before1980 (and after 1978) belong to the third subset (I3).

The dataset of the potential predictors is not organisedby vintages, because Italian indicators are not revised,except for the industrial production index. All monthlyindicators are converted into quarterly averages and areseasonally adjusted using only the observations of thesample period spanned by each vintage. Seasonaladjustment therefore entails some indirect revision.

References

Andrews, D. W. K. (1993). Tests for parameter instability and structuralchange with unknown change point.Econometrica, 61(4), 821−856.

Baffigi, A., Golinelli, R., & Parigi, G. (2004). Bridge models toforecast the euro area GDP. International Journal of Forecasting,20(3), 447−460.

Bulligan, G., Golinelli, R., & Parigi, G. (2008). Forecasting monthlyindustrial production in real-time: From single-equation tofactor-based models. Paper presented at the EABCN-CEPRconference: Euro Area Data: Issues and Implications forEconomic Analysis. March; University of Cambridge.

Busetti, F. (2006). Preliminary data and econometric forecasting: Anapplication to the Bank of Italy quarterly model. Journal ofForecasting, 25, 1−23.

Campos, J., & Ericsson, N. R. (1999). Constructive data mining:Modelling consumers' expenditure in Venezuela. EconometricsJournal, 2, 226−240.

Campos, J., Hendry, D. F., & Krolzig, H. -M. (2003). Consistentmodel selection by an automatic Gets approach. Oxford Bulletinof Economics and Statistics, 65, 803−819.

Castle, J. L. (2005). Evaluating PcGets and RETINA as automaticmodel selection algorithms. Oxford Bulletin of Economics andStatistics, 67, 837−880.

Clark, T. E. (2004). Can out-of-sample forecast comparisons helpprevent overfitting? Journal of Forecasting, 23, 115−139.

Clark, T. E., & McCracken, M. W. (2007). Test of equal predictiveability with real-time data. Paper presented at the conference:Real-Time Analysis and Methods in Economics. April; FederalReserve Bank of PhiladelphiaApril.

Clements, M. P., & Galvao, A. B. (in press). Macroeconomicforecasting with mixed frequency data: forecasting output growthin the United States. Journal of Business and Economic Statistics.

Clements, M. P., & Hendry, D. F. (1998). Forecasting EconomicTime Series. Cambridge University Press.

Clements, M. P., & Hendry, D. F. (1999). Forecasting Non-Stationary Economic Time Series. MIT Press.

Corradi, V., Fernandez, A., & Swanson, N. R. (2007). Information inthe revision process of real-time datasets. Paper presented at theconference: Real-Time Analysis and Methods in Economics.April: Federal Reserve Bank of Philadelphia.

Croushore, D., & Stark, T. (2001). A real-time data set formacroeconomists. Journal of Econometrics, 105, 111−130.

Diebold, F. X., &Mariano, R. S. (1995). Comparing predictive accuracy.Journal of Business and Economic Statistics, 13, 253−263.

Diebold, F. X., & Rudebusch, G. D. (1991). Forecasting output withthe composite leading index: a real time analysis. Journal of theAmerican Statistical Association, 86, 603−610.

Di Fonzo, T., Pisani, S., & Savio, G. (2002). Revisions to Italianquarterly national accounts aggregates: Some empirical results.INSEE-Eurostat Workshop on Quarterly National Accounts,(pp. 7−31). Luxembourg: Eurostat.

Diron, M. (2006). Short-term forecasts of euro area real GDP growth.An assessment of real-time performance based on vintage data.ECB Working Paper, No. 622.

Faust, J. J., Rogers, H., &Wright, J. H. (2005). News and noise in G-7 announcements. Journal of Money, Credit and Banking, 37(3),403−420.

Fair, R. C., & Shiller, R. J. (1990). Comparing information in forecastsfrom econometric models. American Economic Review, 80(3),375−389.

Forni,M., Hallin,M., Lippi,M., &Reichlin, L. (2005). The generalizedfactor model: One sided estimation and forecasting. Journal of theAmerican Statistical Association, 100, 830−840.

Giacomini, R., & White, H. (2006). Tests of conditional predictiveability. Econometrica, 74(6), 1545−1578.

Golinelli, R., & Parigi, G. (2007). The use of monthly indicators toforecast quarterly GDP in the short run: an application to the G7countries. Journal of Forecasting, 26, 77−94.

Hannan, E. J., & Rissanen, J. (1982). Recursive estimation of mixedautoregressive-moving average order. Biometrika, 69(1), 81−94.

Hendry, D. F., & Clements, M. P. (2002). Pooling of forecasts. Eco-nometrics Journal, 5, 1−26.

Hendry, D. F., & Krolzig, H. -M. (1999). Improving on ‘Data miningreconsidered’ by K.D. Hoover and S.J. Perez. EconometricsJournal, 2, 202−219.

Hendry, D. F., & Krolzig, H. -M. (2003). New developments inautomatic general-to-specific modelling. In B. P. Stigum (Ed.),Econometrics and the Philosophy of Economics (pp. 379−419).Princeton University Press.

Hendry, D. F., &Krolzig, H. -M. (2005a).Resolving three ‘intractable’problems using Gets approach. Oxford University, July, mimeo.

Hendry, D. F., & Krolzig, H. -M. (2005b). The properties ofautomatic Gets modelling. Economic Journal, 115, C32−C61.

Hoover, K. D., & Perez, S. J. (1999). Data mining reconsidered:encompassing and the general-to-specific approach to specifica-tion search. Econometrics Journal, 2, 167−191.

Keane, M. P., & Runkle, D. E. (1990). Testing the rationality of priceforecasts: new evidence from panel data. American EconomicReview, 80(4), 714−735.

Kitchen, J., & Monaco, R. (2003, Octoberr). Real-time forecasting inpractice. Business Economics, 10−19.


Klein, L. R., & Ozmucur, S. (2005). The University of Pennsylvaniamodels for high-frequency macroeconomic modelling. Paperpresented at the Project LINK Spring Meeting, UNAM, MexicoCity, Mexico.

Klein, L. R., & Sojo, E. (1989). Combinations of high and lowfrequency data in macroeconometric models. In L. R. Klein, & J.Marquez (Eds.), Economics in Theory and Practice: An EclecticApproach (pp. 3−16). Kluwer Academic Publishers.

Lupi, C., & Peracchi, F. (2003). The limits of statistical information:how important are GDP revisions in Italy? In F. Kostoris (Ed.),Monitoring Italy, Chapter 2 (pp. 67−129). Rome: ISAE.

Muth, J. F. (1961). Rational expectations and the theory of pricemovements. Econometrica, 29(3), 315−335.

Newey, W. K., & West, K. D. (1987). A simple, positive semidefinite,heteroskedasticity and autocorrelation consistent covariancematrix. Econometrica, 55(3), 703−708.

Oller, L. -E., & Hansson, K. -G. (2004). Revision of national accounts:Swedish expenditure accounts andGDP. Journal of Business CycleMeasurement and Analysis, 1(3), 363−386.

Parigi, G., & Schlitzer, G. (1995). Quarterly forecasts of the Italianbusiness cycle by means of monthly economic indicators.Journal of Forecasting, 14, 117−141.

Perez-Amaral, T., Gallo, G. M., & White, H. (2005). A comparisonof complementary automatic modeling methods: RETINA andPcGets. Econometric Theory, 21, 262−277.

Pesaran, M. H., & Timmermann, A. (2005). Real-time econometrics.Econometric Theory, 21, 212−231.

Phillips, P. C. B. (2005). Automated inference and the future ofeconometrics. Econometric Theory, 21, 1−2.

Runstler, G., & Sedillot, F. (2003). Short-term estimates of euroarea real GDP by means of monthly data. ECB Working Paper,No. 276.

Schumacher, C., & Breitung, J. (2006). Real-time forecasting ofGDP based on a large factor model with monthly and quarterlydata. Deutsche Bundesbank Discussion Paper, No. 33.

Sedillot, F., & Pain, N. (2003). Indicator models of real GDP growthin selected OECD countries. OECD Working paper, No 364.

Stark, T. (2000). Does current-quarter information improvequarterly forecasts for the US economy? SSRN WorkingPaper, No. 00-2, http://ssrn.com/abstract=213814

Stark, T., & Croushore, D. (2002). Forecasting with a real-time dataset for macroeconomists. Journal of Macroeconomics, 24(4),507−568.

Stock, J. H., &Watson,M.W. (1996). Evidence on structural instabilityin macroeconomic time series relations. Journal of Business andEconomic Statistics, 14, 11−30.

Stock, J. H., & Watson, M. W. (2002a). Macroeconomic forecastingusing diffusion indexes. Journal of Business and EconomicStatistics, 20, 147−162.

Stock, J. H., & Watson, M. W. (2002b). Forecasting using principalcomponents from a large number of predictors. Journal of theAmerican Statistical Association, 97, 1167−1179.

Stock, J. H., & Watson, M. W. (2004). Combination forecasts ofoutput growth in a seven-country data set. Journal of Forecast-ing, 23, 405−430.

Swanson, N. R., & van Djik, D. (2006). Are statistical agenciesgetting it right? Data rationality and business cycle asymmetry.Journal of Business and Economic Statistics, 24(1), 24−42.

Swanson, N. R., & White, H. (1997). A model selection approach toreal-time macroeconomic forecasting using linear models andartificial neural networks. Review of Economics and Statistics,79(4), 540−550.

Tetlow, R. J., & Ironside, B. (2007). Real-time model uncertainty inthe United States: the Fed, 1996–2003. Journal of Money, Creditand Banking, 39(7), 1534−1561.

Trehan, B. (1989, Winterr). Forecasting growth in current quarterreal GNP. Economic Review, 39−52.

Trehan, B. (1992). Predicting contemporaneous output. EconomicReview, 2, 3−11.

White, H. (1980). A heteroskedasticity-consistent covariance matrixand a direct test for heteroskedasticity. Econometrica, 48,817−838.

Roberto Golinelli is Professor in Econometrics of the University ofBologna (Italy), and Fellow of the Euro Area Business CycleNetwork (EABCN). His main working experience was as head ofresearch projects at the Associazione Prometeia (macroeconometricforecasts), and at the European Commission (HERMES modelproject, indirect taxes harmonisation, the cost of non-Europe). Hisresearch interests include time series forecasting, transition andaccession countries analysis, poolability and heterogeneity.

Giuseppe Parigi is Director in the Research Department of theBank of Italy, and Fellow of the Euro Area Business Cycle Network(EABCN). He did his postgraduate studies at the Universities ofOxford and Warwick (UK), and has worked extensively with theBank of Italy's macroeconometric model of the Italian economy.His research interests include macroeconomics, econometrics andtime series methods and forecasting.

http://ssrn.com/abstract=213814

real-time squared: a real-time data set for real-time gdp forecasting

Documents