Big Data Quality Framework

21/10/2019 United Nations Statistics Division Slide 1

Ronald JansenAssistant Director

Chief of Data Innovation and Capacity BranchUnited Nations Statistics Division

21/10/2019 United Nations Statistics Division Slide 2

Overview

Big Data Quality FrameworkInputThroughputOutput

Examples

Change Management

Big Data Quality FrameworkCES

UNECE Task Force 1994

21/10/2019 United Nations Statistics Division Slide 5

SOURCEInstitutional environment

Sustainability of the data providerReliability of data providerTransparency of data provider

Privacy and SecurityLegislationData Keeper vs. Data ProviderRestrictionsPerception

21/10/2019 United Nations Statistics Division 9

21/10/2019 United Nations Statistics Division Slide 11

Examples

United Nations Statistics Division 21/10/2019 12

Examples

United Nations Statistics Division 21/10/2019 13

United Nations Statistics Division 21/10/2019 14

Change ManagementMaking changes – Maintaining quality

ExampleStatistics Canada

Improving Agricultural crop statisticsUse of SurveysUse of Satellite dataJust Month September

Reduce response burdenMaintain quality

ExampleAustralia Bureau of Statistics

Calculating CPIUse of Price SurveysUse of Scanner dataReplace 25% of Basket

Reduce costGain more detailMaintain quality

United Nations Statistics Division 21/10/2019 15

Change ManagementMaking changes – Maintaining quality

Border SurveysPositives

Controlled sampleTargeted questions

NegativesNon-responseLong processing timeAggregated data

Mobile Phone DataPositives

DetailShort processing time

NegativesSelectivityNo info on purpose

Tourism Statistics

Thank you

United Nations Statistics Division 21/10/2019 16