1

Joint UNECE/EUROSTAT/OECD METIS Work Session(Geneva, 10-12 March 2010)

The On-Going Review of the SDMX Technical Specifications

Marco Pellegrino, Håkan Linden (Eurostat) Arofan Gregory (Open Data Foundation)

Benefits of SDMX

XML-based (“open” format) neutral in terms of underlying technologies

Can be used by national and international agencies as a key building block in internal IT systems used for collecting, compiling and searching statistical information

Avoids duplication of efforts in developing and maintaining standards and IT tools for processing statistical information: pools the expertise and resources working on data/metadata issues (community of developers)

Can improve quality and efficiency in the exchange and dissemination of data and metadata (harmonisation and coherence of data, focus on meaning)

Can reduce the reporting burden

It is an international standard anchored in recognised bodies such as ISO

SMDX important dates

Version 1.0 GESMES/TS

Version 2.0SDMX-EDISDMX-ML SDMX Registry 2008

Acceptance of SDMX at UN levelISO/TS 17369

September 2004

Version 1.0accepted as an ISO technical specification

Version 2.0 approved and published

November 2005 February 2008

SDMX recognised and supported as the preferred standard

The SDMX main components

SDMX Information

model

SDMX Information

model

Content-oriented

guidelines

Content-oriented

guidelines

IT infrastructure for exchange and sharing

IT infrastructure for exchange and sharing

...not just a data transmission format

5 years after the SDMX version 2.0…

Proposed amendments:

1. Bugs in the published SDMX technical standards and inconsistencies to fix;

2. Missing information in order to broadly implement the information model;

3. New features (including convergence with other standards, e.g. DDI and semantic web)

Attention: only the changes really needed should be made Attention: only the changes really needed should be made to the SDMX technical standards to the SDMX technical standards

DDI and SDMX

SDMXAggregated data

Indicators, Time SeriesAcross time

Across geographyOpen AccessEasy to use

DDIMicrodata

Low level observationsSingle time period Single geographyControlled accessExpert Audience

Crucial overlap and mappings, normally not well documented Interoperability would provide users with a full picture of the production

process

Main issues at stake

Backwards compatibility

Analysis of the most common use cases behind each change (e.g. need of adapting some of the data formats)

Impact assessment: costs Vs benefits expected

Use of a common terminology: SDMX technical vocabulary

Technical Workshop on the review of the SDMX technical standards held

Luxembourg in February 2010

Workshop attended by more than 30 participants coming from 11 national agencies (statistical institutes or central banks) and from 6 SDMX sponsoring organisations

Main issues discussed: analysis of the change requests for the technical standard, use cases, impact assessment, backward compatibility, ……….

8

Agreement on the following points

The new version of the SDMX technical standards must fix bugs and inconsistencies, while providing solutions to well-identified use cases;

The improved SDMX technical standard 2.1 should not get too complex; this could block its further use and implementation and cause higher costs…..

There is a need for a proper documentation.

Technical subgroups installed

Technical subgroups working on the following issues:

1. Web services (work already started)

2. Structure Query And Registry Interface Reconciliation

3. Code-lists

4. Cross-sectional format

5. Compact format

6. Query format

Additional issues to be tackled in a second step: Business registers, Additional issues to be tackled in a second step: Business registers, transformations and expressionstransformations and expressions

Timeline and planning

1. March-April 2010: Comments, follow-up, working of the sub-groups

2. May 2010: Technical report on the SDMX Technical Standards 2.1

3. June 2010: Submission to the SDMX Secretariat and SDMX sponsors for defining the further process