Census Bureau Seasonal Adjustment Software and Research

David.F.Findley@census.gov

U S C E N S U S B U R E A UU S C E N S U S B U R E A U

Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software

Genhol (holiday regressors)SAS Software:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses sets of series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)

Outline of Talk: Research

TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)– Filters and Filter Diagnostics– Automatic modeling: TRAMO vs. X-12’s “TRAMO”– Revisions

State-Space Models using Sampling Error Data

Non-Gaussian “Structural” State-Space Models for More Stable Resistance to “Outliers”

Statistical Research Division Time Series Group Research and “X-12-ARIMA+” Programming

Brian.C.MonsellKellie.C.WillsWilliam.R.Bell (honorary)David.F.Findley (honorary)Donald.E.Martin (Part-time, Howard University)Trang.Ta.Nguyen (1-year in-house visitor)John.Alexander.Aston (2-year Post-Doc from

Imperial College, London)S.J.M. Koopman (Fellow, Free Univ. of Amsterdam)

Economic and Statistical Programming DivisionTime Series Methods Branch

Research and SAS, Excel Programming

Catherine.C.HoodKathleen.M.McDonald.JohnsonGolam.FarooqueRoxanne.Feldpausch

Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software

Genhol (holiday regressors)SAS Software:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses sets of series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)

X-12-ARIMA

Improvements over StatsCan’s X-11-ARIMA• regARIMA models (including outliers, user-defined

regressors, etc.) vs. ARIMA models• Much more extensive automatic options for

modeling, including trading day, holiday est., additive vs. multiplicative adjustment

• More diagnostics (e.g. spectra, revisions)• Specialized output files, e.g. log files for users

favorite diagnostics, from many X-12-Graph (SAS, but for non-SAS-users)

X-11 Seasonal Adjustment

RegARIMA Models(Forecasts, Backcasts, and Preadjustments)

Modeling and Model Comparison Diagnostics and Graphs

Seasonal Adjustment Diagnostics and Graphs

REGARIMA Model

transformation ARIMA Process

Regressors for trading day and holiday or calendar effects, additive outliers,temporary changes, level shifts, rampsuser-defined effects

Leap-year adjustment, or“subjective” strike adjustment, etc.

ttt

t ZXDY

log

Xt

Dt

Types of Regression Variables Available in X-12-ARIMA

Outlier and Trend-Change EffectsAdditive (or Point) OutliersTemporary Change Outliers

Level shifts, Ramps Seasonal Effects

Calendar month indicators*Trigonometric Seasonal (Sines-Cosines)*

Calendar EffectsTrading Day (Flows or Stocks)*Leap-year February*, Length of Month*Shifting Holidays (e.g. Easter)

Constant Term User-Defined Effects

*Two-regime option availableNote: Regression coefficients can be fixed

X-12-ARIMA Releases

Ver. 0.2.10 July (Statistics Canada options)Ver. 0.3 Summer (TRAMO-type automatic

ARIMA model selection)-based on information gleaned from

TRAMO code provided by Victor Gomez

Ver. 1.0 End of year (Better organized output and manual, more testing etc.)

“X-12-ARIMA/SEATS”• Offers both x11{ } and seats{ } commands to provide “X-

11” or SEATS type seasonal adjustments with X-12-ARIMA diagnostics as well as SEATS diagnostics

• Is being updated from SEATS2000 to SEATS2001&2002 (with support from Agustin Maravall and Gianluca Caporello)

• Schizophenic (duplicate) output, currently• Distribution for research and testing to statistical

agencies and central banks in 2003

Diagnoses from X-12-ARIMA/SEATS

1. Spectrum diagnostic reveals source of “Invalid Decomposition” problem

X-12-A/SEATS COMMAND FILE

series {file="serie.txt"format="tramo"}

transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}check{}#x11{}seats{}

Message from seats{ } run:

• NOTE: Spectral plot for the seasonally adjusted series cannot be done when SEATS cannot perform a signal extraction.

Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.3846 0.12087

Seasonal MA Lag 12 -0.3665 0.12612

10*LOG(SPECTRUM) of the regARIMA model residuals Spectrum estimated from 1990.Jan to 1995.Oct.+++++++I+++++++++++++++++++++++++++++ -22.11I * I * I * I * * -23.34I * * I * * * I * * T I * * *T -24.57I * * *T I * * *T I * * * *T I * * * *T* -25.81I * * * *T* I * * * *T*

series {file="serie.txt"format="tramo"}

transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}check{}x11{}#seats{}

X-12-ARIMA/SEATS Seasonal Adjustment ProgramVersion Number 0.3s Build 24

WARNING: At least one visually significant trading day peak has been found in one or more of the estimated spectra.

G.1 10*LOG(SPECTRUM) of the differenced, transformed seasonally adjusted data. Spectrum estimated from 1990.Jan to 1995.Oct. ++++++++++I+++++++++++++++++++++++++++++++++++ I T I T I T -20.10I T I T I T I T -22.01I T I T I T

series {file="serie.txt"format="tramo"}

transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}regression{variables=td}check{}seats{}

X-12-ARIMA/SEATS Seasonal Adjustment ProgramVersion Number 0.3s Build 24

Reading input spec file from metalss.spc Storing any program output into metalss.out Storing any program error messages into metalss.err

WARNING: At least one visually significant seasonal peak has been found in one or more of the estimated spectra.

Standard Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.1995 0.12871

Seasonal MA Lag 12 0.3843 0.15795

X-12-ARIMA Diagnoses for SEATS

2. T/S Practice of adding outliers to improve kurtosis, etc. can substantially increase the size of revisions of the initial seasonal adjustments:

Example (from Catherine Hood) US Exports of Passenger Cars: History diagnostic shows cost to revisions of adding outlier regressors to reduce kurtosis

Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software

Genhol (holiday regressors)SAS Utilities:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses of many series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)

Genhol

• From holiday date file: Generates regressor matrices and associated command files to enable X-12-ARIMA estimation of complex moving holiday effects (e.g. for Easter, Ramadan, etc.).

• Regressors for up to three intervals:– before-the-holiday interval– surrounding-the-holiday interval– past-the-holiday interval (“recovery” interval)

Proportionality Regressors: An Example

• Assume :– An effect interval is 10 days long, and this year

2 of its days fall in January and 8 in February.The interval regressor’s values for this year will

be:– 0.2 in January– 0.8 in February– 0.0 for the rest of the year

Interface Program (SAS): for seasonal adjustment of sets of series

Example: Seasonally Adjusted Total U.S. Imports = sum of 140 component series, c. 80% of which are seasonally adjusted.

What is the effect on the month-to-month changes and quality diagnostics of the S. A. Total Imports if the seasonal adjustment options are changed for 5 of the component series?

Outline of Talk: Research

TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)– Filter Diagnostics – Automatic modeling: TRAMO vs. X-12’s “TRAMO”– Revisions

Filters and Filter Diagnostics

• Filter (spectral) diagnostics needed– To understand limitations/issues with short

series (finite filter diagnostics, also for concurrent adjustments, trends)

– To decide between closely competitive models

Paper by David Findley and Donald Martin.

0 1 2 3 4 5 6

Cycles per year

0.0

0.5

1.0

1.5

Squ

ared

gai

nSquared gain of symmetric SEATS filters

Parameter values -- 0.4,0.8

infinite109 months49 months

Outline of Talk: Research

TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)

• Automatic modeling: TRAMO vs. X-12’s “TRAMO”

• Accuracy– Results from simulated series

• Revisions– Results from Census Bureau series

ESMPD’s Automatic Modeling Study

• First presented at the International Forecasters Symposium, June 2001

• Continuation of this work to appear at the ASA meetings, August 2002, in a paper by Kathleen McDonald-Johnson, et al.

Series

306 time series from the US Census Bureau’s Import/Export series and Retail Sales

Results

• 88 series (29%) with same regARIMA model• 27 series (9%) with same differencing and same

regressors but different ARMA choices• 123 series (40%) with same differencing, but

different regressors• 32 series (10%) with different nonseasonal

differencing order (but sometimes offset by a constant)

• 36 series (12%) with different seasonal differencing order

Conclusions

• TRAMO’s weakness is the procedure for deciding about trading day modeling– TRAMO developers are aware of our results

• X-12-ARIMA has a problem with choosing nonparsimonious models– Monsell has already implemented some

changes, including a unit root test.

Why Are Different Models Chosen?

• Model estimation method is different – TRAMO : Hannan-Rissanen and m.l.e conditional on

AR part of model– X-12-ARIMA : Exact MLE

• Model residuals are different, which can lead to different choices of outliers

• Outlier procedure itself is different – TRAMO removes insignificant outliers after each

iteration• TRAMO uses approximate BIC

Accuracy: X-12-ARIMA vs T/S (ESMPD)

• Results from 54 simulated series were first presented at the ASA meetings, August 2000– Continuation of the first SEATS studies,

beginning in 1997

The Simulated Series

• Fifty-four series– Six different trends – three from SEATS and

three from X-12– Six different seasonal factors – three from

SEATS and three from X-12– Irregular sampled from three sets of irregular

factors combined from SEATS and X-12

Results of Accuracy Study

• SEATS performed better on the majority of series with large irregulars if the series are 9+ years long, but most adjustments were not acceptable.

• Both programs did better than expected on the short series, but X-12-ARIMA adjustments were usually better than SEATS adjustments on series 4-7 years long

Revisions: X-12-ARIMA vs T\S

• New ESMPD study using X-12-SEATS on Census series. “Final” results will be presented at the ASA meetings, August 2002.– Can we identify characteristics in the series

that will indicate if its “linearized” series will be a better candidate for a model-based adjustment than for an X-11 filter adjustment or vice versa?

Methods

• Use X-12-SEATS to get revision diagnostics from both an X-11/X-12-type adjustment and a SEATS adjustment– Used TRAMO to get the ARIMA model, and

then used either an x11 or a seats “spec”

Very Preliminary Results

• 260 US Import/Export series• Only a very small subset (18 series) where

we can see definite differences in the revision diagnostics for the seasonal adjustment

An Observation: Series with

– Large revisions in X-12 and smaller revisions in SEATS had generally large values for 12 (most greater than 0.95) and values for X-12’s I/S ratio < 5.

– Large revisions in SEATS and smaller revisions in X-12 had generally 0.4 < 12 < 0.6 and values for I/S > 6.

In both cases, smaller revisions are associated with more constant seasonal factor estimates

Next Steps

• Look at more series• Look at more diagnostics/characteristics of

the series to try to find patterns, not just revisions

Outline of Talk: Research

Projects almost ready to yield results:

State-Space Models using Sampling Error Data Non-Gaussian “Structural” State-Space Models

for More Stable Resistance to “Outliers”

State-Space Models with Sampling Error Statistics: Bell and Nguyen

100+ Disaggregate Construction series with “high” sampling error variancesConsider model-based adjustment with

regARIMA+observation errormodels that incorporate sampling error variance and autocovariance estimates to achieve acceptable or better seasonal adj’s.

(Need state-space for model est. & seas adj.)

Non-Gaussian “Structural” State-Space Models for More Stable Resistance to

“Outliers” Koopman and Aston X-12-ARIMA and T\S use outlier regressors

identified by t-statistics and critical values. Identifications can change as new data arrive, causing seasonal adjustment revisions.

Use heavy tailed non-Gaussian models instead of critical values. (Hard to estimate such models, simplest for “Harvey’s structural models”)

More Information

WWW site for X-12-ARIMA (papers and software):

www.census.gov/srd/www/x12a

Thanks to Catherine Hood for supplying some of these slides.