© metadata technology escwa sdmx workshop session: data formats
TRANSCRIPT
© Metadata Technology
ESCWA SDMX Workshop
Session: Data Formats
© Metadata Technology
What is Data?
• Results of Measurements
Country: UK
Unemployed: 13million
Year: 2010
Country: France
Unemployed: 5million
Year: 1982
Country: Australia
State : Perth
Coastline (Km): 12500
© Metadata Technology
What is a DataSet?
• An organised collection of Data
KEY
Values
KEY
Values
Data Set
Think of a Dataset as an organised container for exchanging Data
© Metadata Technology© Metadata Technology
How is the Data organised?
Use case: House Price Data
© Metadata Technology
Average House Price
• House Type– Terrace– Semi Detached– Detached
• County (location)– Surrey– Hampshire
• Number Bedrooms • Time• Average Price (measure)
Code List
Dimensions (Concepts)
© Metadata Technology
DataHouse Type County Bedrooms Price Time
Terrace Surrey 2 130,000 2008
Terrace Surrey 2 150,000 2009
Terrace Surrey 2 120,000 2010
Terrace Hampshire 3 170,000 2009
Terrace Hampshire 2 200,000 2008
Detached Hampshire 3 250,000 2008
Detached Hampshire 2 210,000 2009
Detached Surrey 2 300,000 2008
Semi-Det Surrey 3 250,000 2008
Semi-Det Surrey 2 220,000 2009
Semi-Det Hampshire 3 250,000 2009
© Metadata Technology
House TypeTerrace
CountySurrey
Bedrooms2
KEY
2008
130,000
2009
150,000
2010
220,000
VALUES
Time Series View
House TypeDetached
CountyHampshire
Bedrooms2
2008
200,000
2009
210,000
VALUES
House TypeSemi-Det
CountySurrey
Bedrooms3
2008
210,000
2009
220,000
VALUES
© Metadata Technology
Pivot on County
House TypeTerrace
Surrey
130,000
Bedrooms2
KEY
Time2008
Hampshire
200,000
VALUES
© Metadata Technology
Pivot on House Type
CountySurrey
Terrace
130,000
Bedrooms2
KEY
Time2008
Detached
300,000
VALUES
Semi
220,000
© Metadata Technology
Q: What is the Key? A: Anything except the Observation Value and a pivot dimension
House Type County Bedrooms Time Price
Terrace Surrey 2 2008 130,000
Terrace Surrey 2 2009 150,000
Terrace Surrey 2 2010 120,000
Terrace Hampshire 2 2008 200,000
Terrace Hampshire 3 2008 250,000
Detached Hampshire 3 2009 170,000
Detached Hampshire 2 2009 210,000
Detached Surrey 2 2008 300,000
Semi-Det Surrey 3 2008 250,000
Semi-Det Surrey 2 2009 220,000
Semi-Det Hampshire 3 2009 250,000
© Metadata Technology
Pivot on House Type
House Type County Bedrooms Time Price
Terrace Surrey 2 2008 130,000
Terrace Surrey 2 2009 150,000
Terrace Surrey 2 2010 120,000
Terrace Hampshire 2 2008 200,000
Terrace Hampshire 3 2008 250,000
Detached Hampshire 3 2009 170,000
Detached Hampshire 2 2009 210,000
Detached Surrey 2 2008 300,000
Semi-Det Surrey 3 2008 250,000
Semi-Det Surrey 2 2009 220,000
Semi-Det Hampshire 3 2009 250,000
ValuesKey
© Metadata Technology
Pivot on Country
House Type County Bedrooms Time Price
Terrace Surrey 2 2008 130,000
Terrace Surrey 2 2009 150,000
Terrace Surrey 2 2010 120,000
Terrace Hampshire 2 2008 200,000
Terrace Hampshire 3 2008 250,000
Detached Hampshire 3 2009 170,000
Detached Hampshire 2 2009 210,000
Detached Surrey 2 2008 300,000
Semi-Det Surrey 3 2008 250,000
Semi-Det Surrey 2 2009 220,000
Semi-Det Hampshire 3 2009 250,000
ValuesKey
© Metadata Technology
Add new Dimension
House Type
County Bed Bath Time Price
Terrace Surrey 2 2 2008 130,000
Terrace Surrey 2 2 2009 150,000
Terrace Surrey 2 2 2010 120,000
Terrace Hampshire 2 2 2008 200,000
Terrace Hampshire 3 3 2008 250,000
Detached Hampshire 3 3 2009 170,000
ValuesKey
© Metadata Technology© Metadata Technology
DataSet and SDMX Schema
© Metadata Technology
SDMX Schema for Data
• SDMX Has 2 Widely Used Formats Data
The way the XML is structured
<Person name=“Matt” age=“28” /> <Person>
<Name>Matt</Name>
<Age>28</Age>
</Person>
2. COMPACT
1. GENERIC
© Metadata Technology
Generic Data
KEY
VALUES
© Metadata Technology
Compact Data
KEY
VALUES
DATASET
KEY
VALUE
VALUE
KEY
© Metadata Technology
GENERIC DATA – ONE SERIES
COMPACT DATA – ONE SERIES
© Metadata Technology
COMPACT DATA – 1 Series
GENERIC DATA – 1 Series
SERIES ATTRIBUTES
© Metadata Technology
OBSERVATION ATTRIBUTES
GENERIC DATA – 1 Observation
COMPACT DATA – 3 Observations
© Metadata Technology
Exercise Generate Schema
• Open Up the SDMX Tool Suite
• Drag DSD File onto Tool Suite
• Generate a Compact Schema
© Metadata Technology© Metadata Technology
Web Services