metadata for the s-dwh ‒ an overview lars-göran lundell statistics sweden
TRANSCRIPT
Metadata for the S-DWH ‒
an overview
Lars-Göran Lundell
Statistics Sweden
Why do we need metadata?
DON’T
PANIC42
Metadata and the S-DWH
• “Metadata is the DNA of the data warehouse”— Ralph Kimball
• “Metadata for the data warehouse environment is one of the most important aspects”
— Bill Inmon
• “Statistical metadata systems play a fundamental role in statistical organizations”— METIS
The Metadata Framework
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
Metadata Categories
Active
Passive
ReferenceStructural
FormalisedFree-form
Metadata subsets• Statistical metadata• Process metadata• Quality metadata• Technical metadata• Authorisation metadata• Data models
• Deliverable 1.1
Quality
Process
Statistical
Technical
Authorisation
Data models
Metadata Functionalities
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
Metadata Functionalities
• Metadata functionalities needed to facilitate and support the operation of the S-DWH
• Descriptions, definitions• Standards: GSBPM, Neuchâtel, etc.• Examples• Case study
• Deliverable 1.4
S-DWH Layer Create Use Maintain Evaluate
Data access √ √ √ √
Interpretation √ √ √ √
Integration √ √ √ √
Source √ √ √ √
Metadata Models
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
Metadata Models
• Overview of available models and standards• Relevant models for the S-DWH
• Topicality, support, usage, usability
• Suitability for metadata subsets• No super model
• Recommendations• Keep it simple• Use only one model per subset• Make models/standards known
• Deliverable 1.3
Metadata Quality
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
Metadata Quality
• International Standards• ISO 9000, ISO 11179
• Quality dimensions to assess S-DWH metadataRelevance Comparability Stability
Accuracy Coherence Completeness
Accessibility Uniqueness Interpretability
• Metadata quality characteristics by S-DWH layer
• Quality management for S-DWH metadata • Customer focus• Process approach• System approach to management
Metadata Quality
• Recommendations:• Adopt quality dimensions• Decide on quality indicators and quality levels• Adopt naming standards• Decide on compulsory attributes• Set up governance and assessment rules
• ... all “as appropriate”
• Deliverable 1.2
Metadata for the layered architecture
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
Metadata for the layered architecture
• Mapping:• Functionalities of the SDWH metadata system
• (deliverable 1.4)
• Functional architecture of the SDWH• (deliverable 3.3)
• Examples
• Metadata subsets and functionalities by S-DWH layer
• Metadata in the functional architecture data flows
• Deliverable 1.6
The Metadata Layer
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
The Metadata Layer
• One logical metadata store• The user has one place to search• Matches all S-DWH layers
• Several physical stores (possibly)
• Deliverable 1.1
Quality
ProcessActive
Passive
ReferenceStructural
FormalisedFree-form
A metadata item The metadata layerThe data store
Statistical
Metadata Governance
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards
Metadata Governance
• Governance and management• Roles and functionalities• Governance roles: Who does what?• Governance functions• Governance and metadata subsets
• Deliverable 1.5
Deliverables
1.1 Metadata FrameworkLars-Göran Lundell
1.2 Recommendations on the impact of Metadata Quality
Colin Bowler, Michel Lindelauf, Jos Dressen
1.3 Overview of the use of Metadata Models
Jos Dressen, Michel Lindelauf, Harry Goossens
1.4 Definitions of the Functionalities of a Metadata System
Ennok, Lundell, Bowler, De Giorgi, Kulla
1.5 Governance of Metadata Management
Viviana De Giorgi, Michel Lindelauf
1.6 Mapping deliverable 1.4 to the Ideal Architecture
Maia Ennok
Metadata Framework
General metadata considerationsDefinitions and terminologyCollecting and using metadataAvailable standardsMetadata in the S-DWHGeneral requirementsAdapting to the layered S-DWHOrganising metadataGoverning metadataConsidering standards