CountryData DevelopmentImproving the collation, availability and dissemination of development indicators (including the MDGs)

Nairobi, 27 November 2013

National Systems

National Database• Implemented in all project countries• Store and expose national development

indicators• Provide a searchable Web interface over

the data• Most project countries use the DevInfo

software as the platform• Thailand and Morocco use native systems

(StatXChange and BDS)

SDMX Registry• Facilitates “pull model” for efficient data

dissemination• Sends update notifications to subscribers• Enables automated data and metadata

transfers• Implemented in 10 project countries• Most project countries use DevInfo 7

Registry– Thailand uses Metadata Technology’s Fusion

Registry

DevInfo 7• Released by DevInfo Support Group in

November 2012.

• Much improved over di 6: better user interface, visualizations, search

• A number of tools developed for the project facilitate SDMX exchange– SDMX Registry– MDG DSD Mapping Tool

DevInfo 7: Recent Developments• A number of improvements recently

implemented for the project by DSG– Advanced mappings (many-to-one and one-

to-many)– Advanced publication filters– Simplified and improved menu structure– Improved SDMX functionality

Thailand: StatXChange to CountryData

• StatXChange is an advanced data sharing system developed by the Thailand NSO– Relies on Web services that supply data in an

XML format (StatXML)

• Application was developed with UNSD assistance that converts StatXML to MDG SDMX.

Thailand: SDMX Registry• Once converted, SDMX data is placed on

a Web server and registered with Fusion Registry, installed with UNSD assistance.

• Fusion Registry is an advanced implementation of the SDMX Registry specification, developed by Metadata Technology.

Morocco: SDMX Export• Morocco uses its own indicator database,

BDS, developed in-house.

• Morocco NSO built SDMX export module, which produced SDMX data that has been published at CountryData.

• Fusion Registry to be installed in December 2013 to complete the data exchange system.

CountryData

CountryData Platform• Developed by UNSD for the project• Automatically receives development data

indicators from project countries• Enables comparisons between national

data and those obtained from international agencies

• Provides re-dissemination and data visualization

• Part of, and linked to, UNdata

CountryData SDMX Data Exchange

National Registry

National Repository DB

1. Publish

2. Register

CountryData Web Service

3. Notify

4. Download

CountryData DB

5. Upload

SDMX Dissemination Site

International MDG DB

6. Publish and Visualize

National Statistical Office

CountryData Web Site

Discrepancy Analysis• Once in the CountryData database,

national development indicators are matched with international MDG series.

• Any discrepancies are investigated and reported on the Web site.

• Metadata crucial in explaining discrepancies.

Visualization and Re-dissemination• National and international development

indicators are presented at CountryData in charts as well as a tabular format.

• Data transferred to CountryData can be downloaded in a variety of formats including CSV, Excel, and SDMX.

Application Programming Interface• API was developed for CountryData and

some other UNdata datasets in 2012

• Based Eurostat’s SDMX Reference Infrastructure software

• Uses SDMX to query for structures and data, and to receive results

• Support for SDMX 2.1, as well as REST Web service, to be introduced soon

4. Retrieve data

API: How It Workswww.mydataportal.com

1. Request chart on a Web page

2. Call SDMX Web Service

3. Query database

5. Return SDMX data

6. Create and return chart

Plans for the Future

Further DevInfo Development• Work is underway on new di features

– Installer for the di 7 software– Further improvements to SDMX publishing

Expansion of Data Exchange• Mexico has supplied MDG indicators in the

SDMX format, which has been published at CountryData.

• Other countries have expressed interest in joining data exchange.

Migration of Legacy Data• Plans are in place to migrate the MDG

Labs database into CountryData– MDG Labs was developed for the pilot phase

of the UNSD-DfID project.– Shows discrepancy analysis for several dozen

countries.– Does not support SDMX or data exchange.

After migration, all its data will be downloadable as SDMX.

CountryData Development• New home page to accommodate non-

project countries

• New visualizations as part of UNdata redesign

DSD Development• MDG DSD constantly updated, primarily

with geographical information.– CountryData DSD v 0.9 in sync with latest

MDG DSD

• CountryData DSD to be updated with new indicators requested by countries.

• Both DSDs to be updated for improved maintenance and compatibility with SDMX Reference Infrastructure.

Links• MDG Labs (current version)http://unstats.un.org/unsd/mdglabs

• UNdata Web Servicehttp://data.un.org/WS

• UNdata SDMX Browserhttp://data.un.org/SdmxBrowser