Eurostat

Enhancing dissemination: aspects related to sets of

policy indicators

ESTP course - MIPLuxembourg 1-3 December 2015

R. Ruggeri Cannata

Eurostat

Outline

• Release policy• Revision analysis• Data coverage

Eurostat

Looking at the set of indicators: release policies

ESS guidelines on revisions policy for PEEIs, Annex: Principles for a common revision policy for European StatisticsPrinciple 4:Routine and annual revisionsRoutine and annual revisions should be published in the framework of well defined, synchronised and regularly updated release/revision calendars at national and European level. Releases of European and national data aggregates should be synchronized as far as possible.

Eurostat

Looking at the set of indicators: data from different domains

Impact on the MIP set of indicators• No aggregate (EU and EA)• Different domains: different release policies• FA: always up to date • GGGD: as published in the EDP notification

Eurostat

Looking at the set of indicators: release policies – the issue of ratios

• For most of the MIP indicators: numerator and denominator stemming from different domains (% GDP, deflator)

• Denominator value is taken at the date of the release of the numerator (Numerator policy)

• Ratio updated whenever numerator or denominator is updated (Freshest information policy)

• Values are "fixed" at the moment of the official release (EDP policy)

Numerator's domain policy

Eurostat

Revisions

Eurostat

Harmonization of revision policies: CoP

• Principle 6 "Impartiality and objectivity" • 6.6: Advance notice is given on major revisions or

changes in methodologies• Principle 8 "Appropriate Statistical Procedures" • 8.6: Revisions follow standard, well-established and

transparent procedures • Principle 12 "Accuracy and Reliability" • 12.3: Revisions are regularly analysed in order to

improve statistical processes

Eurostat

ESS guidelines on revision policy for PEEIsThe definition of a revision policy at ESS level is an essential step towards further harmonisation of infra-annual statistics, especially Principal European Economic Indicators (PEEIs) It follows the vision of an integrated European statistical system where the releases and the revisions of European statistics are coordinated and synchronised as far as possible, and transparency for the user is maximised via clear communication of revision policies and practices

Eurostat

Are revision good or bad?Revisions have to be considered a normal phenomenon to increase progressively the quality and in particular the accuracy of the data. Revision policy should be recognized as an important aspect of good governance in statisticsGood governance in statistics, in turn, is part of public sector transparency and accountability more broadly

Eurostat

Definition of revisions• Revisions are broadly defined as any change in a

value of a statistic released to the public. They can occur either when new observations become available and some past values are modified or when the current and/or some past values are modified

• Data are generally revised in order to incorporate new, improved information. Therefore, revisions are inevitable whenever statistics are produced that report promptly on economic developments despite the fact that some relevant information is still outstanding

Eurostat

Should a revision policy be publicly available?• A public revision policy gives users confidence,

providing in advance explanations for the reasons for revisions and the nature of them• An ESS revision policy facilitates the comparison

of practices adopted among domains and countries

• It promotes a common language when discussing revisions, also improves, as a matter of fact, the quality of the documentation

• Common policy action among member countries promotes consistency of data at the national and international levels

Eurostat

Revision analysis for data users

Aspects important for the evaluation of revision impact on the overall data quality:•Accuracy•Reliability•Stability

• number of revisions within a given unit of time

Eurostat

Is revision analysis a tool for the users only?

• Producers use revision analysis (size and direction) to identify the presence of problems in estimations and, where possible, to modify the statistical production process

• Producers use revision studies to identify particular methods that could be improved or in order to achieve greater accuracy or timeliness

Eurostat

Alternative views on vintages• Compare a vintage to another and summarize the

differences between the two columns (“vertical” or “classical” view)

• Look to the revisions of a specific reference period (“horizontal view”)

• Focus on the diagonals looking for example to the differences between the first release and the second release

Eurostat

Descriptive revision analysis• MAR: measures the average size of the revisions

without providing an indication of directional bias• Associated with a 90% range

• Arithmetic average (or revisions mean) reveals whether revisions are systematic or not and gives an indication on the average level of revision being close to zero• Positive mean may indicate that on average earlier

releases have been underestimated• Test for significance

Eurostat

Descriptive revision analysis - cont.

• RMAR useful when making comparisons in the size of revision across different indicators

• Revisions Variability: • standard deviation of revisions as indicator of

revisions volatility• minimum and maximum revision

Eurostat

Descriptive revision analysis - cont.• News or noise:

• Revisions add new available information (news)• Revisions arise because of measurement errors and

inefficiencies in the preliminary estimates (noise)• Testing the correlation between

Rt = Lt - Pt and the estimates Lt and Pt

• Noise: Rt are significantly correlated with Pt and uncorrelated with Lt

• News: Rt are significantly correlated with Lt and uncorrelated with Pt

Eurostat

Revision analysis to improve the production process

• Hoven (2008): Dutch estimates of GDP volume growth

• Ciammola et al. (2008): Italian index of industrial production

• Both propose a top-down approach to identify the specific area(s)/domain(s) “responsible” for large or biased revisions

Eurostat

Looking at set of indicators: revision analysis

• Basic/ core measures: targeting users that require quick, easy to understand information

• Additional/ advanced measures: targeting users that require more in-depth analysis

• Sophisticated / special user measures: information for detailed research purposes

Eurostat

Summary of statistics for analysis of revisions (1)1. Mean absolute revision2. Median absolute revision3. Arithmetic average or mean revision4. Statistical significance of the mean revision5. Median revision:6. % of positive revisions:7. % of negative revisions:8. % of zero revisions:9. Adjusted t-statistic for significance of mean revision10.Critical values of t statistic for significance of mean revision11.Standard deviation of revision12.Root mean square revision13.Quartile deviation

21

Eurostat

Summary of statistics for analysis of revisions (2)14.Minimum revision15.Maximum revision16.Range of revision17.Skewness18.% sign (later) = sign (earlier):19.Acceleration / deceleration20.Relative mean absolute revision:21.Average absolute value of first published estimate22.Correlation between revision and earlier estimate (test if revisions are

„noise‟)23.Correlation between revision and later estimate (test if revisions are „news‟)24.Serial correlation of revisions:25.Decomposition of the Mean Squared Revision

22

Eurostat

Data coverage

Eurostat

Looking at set of indicators: data coverage• Policy makers need an as complete as possible

picture of the economy

• Statistics are subject to events which could disrupt time series length• adoption of new classifications (ESA 2010, BPM6)• changes in the production process

Eurostat

Approval process

Eurostat

Characterising Back-calculation• Horizon of backcalculation • Choice of the model and of the method

• Robustness• In official statistics: high degree of automation needed to

minimise personal judgment• Validation of results• Backcalculation applications• House price index: MT, ES, LT, LV, CY• Disaggregation• Unemployment rate: HR

Eurostat

Application - HPI• Aim: 10 years coverage for the Eurostat

2010=100 annual series using quarterly ones1. Check on correlation between official and

auxiliary series on the overlapping period2. Choose the most correlated series as proxy3. Data transformation: log-transformed and first

order differentiated 4. OLS regression run on the overlapping period on

delta logs5. Estimated parameters are used to backcalculate6. Calculate the annual series (AVG of 4 quarters)

Eurostat

Latvian quarterly House price index (2010=100)

Eurostat

Disaggregation: HR Unemployment rate • HR data included in the MIP scoreboard since

accession in 7/2013• Unemployment yearly values derived from

monthly ones• Monthly values produced by Eurostat from

quarterly LFS + monthly number of unemployed persons

• HR LFS available twice a year for 2000-2006• Eurostat disaggregated the quarterly data to

produce monthly values by components after using a SARIMA model to produce missing quarters

Eurostat

HR Unemployment rate (%)

Eurostat

Useful Links• ESS guidelines on revisions

http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-RA-13-016/EN/KS-RA-13-016-EN.PDF

• OECD/Eurostat Task Force on “Performing Revisions Analysis for Sub-Annual Economic Statistics”

• http://www.oecd.org/std/40315546.pdf