![Page 1: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/1.jpg)
Irene Salemink1 oktober 2019
Future Advanced Data Collection
The Future is NOW
![Page 2: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/2.jpg)
![Page 3: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/3.jpg)
Future data collectionStrategic assignments for NSO’s
Shaping the future by being smart
Trends
Challenges and constraints
Future data collection
ACTION
Conclusions
![Page 4: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/4.jpg)
4
Strategic assignments NSO’s
![Page 5: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/5.jpg)
5
• Demand driven and user centric• Broad range statistics/information• Fit-for-purpose• Accurate and timely• Policymakers
• Private sector• Society
• Fact based policy making• Actionable Intelligence
Strategic assignments NSO’s
![Page 6: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/6.jpg)
6
• Dealing with complexity• Complex societal and economic phenomena• Real time statistics• Tailored to all aggregation levels• More detail, regional, local• Fit –for-purpose quality
Strategic assignments NSO’s
![Page 7: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/7.jpg)
7
New information products
![Page 8: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/8.jpg)
Complex phenomena
Transport
GezondheidMotieven achter Mobiliteit
GDP
Arbeidsmarkt
Energie
Innovatieve bedrijven
Migratie
NationaalCyber security Woningmarkt
Luchtkwaliteit
Toerisme
Energy transition
Support local government with insights to support sustainable development goals
Production Installation Register (solar panels)VAT registers from Tax authorityBasic register Addresses and buildings
![Page 9: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/9.jpg)
Complex phenomena
Transport
GezondheidMotieven achter Mobiliteit
GDP
Arbeidsmarkt
Energie
Innovatieve bedrijven
Migratie
NationaalCyber security Woningmarkt
Luchtkwaliteit
Toerisme
Innovative enterprises
Web scraping and text mining to identify small innovative enterprises
ClassificationLinking to background characteristics
![Page 10: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/10.jpg)
10
Shaping the future by being smart
![Page 11: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/11.jpg)
11
• Official statistics become “smart statistics”• Smart technologies • Smart data
• Guaranteed confidentiality
• Privacy by design
Trusted Smart Statistics
Shaping the future by being smart
![Page 12: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/12.jpg)
12
• Facts for evidence based policy making• Quantitative monitoring of development and progress
of policy• Society oriented• Reliable and innovative • Protect confidentiality and privacy
Nature of data collection is bound to change
Trusted Smart Statistics
![Page 13: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/13.jpg)
13
Trends
![Page 14: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/14.jpg)
14
Trends
• Datafication
• Data storage and access
• Computing power and analytical capabilities
![Page 15: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/15.jpg)
15
Administrative data use at CBS
![Page 16: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/16.jpg)
16
Sensors and IoT
![Page 17: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/17.jpg)
• Need for :Fast, easy, and free access to all relevant data
• Reality:Datasets; too big to copy, not allowed legally to “leave the building”, need matching between multiple (different) sources, require knowledge, only the proportion that is needed can be accessed…
• Solution:Dependent on type of data sharing
17
Data storage and access
![Page 18: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/18.jpg)
18
Data architecture patterns
Data virtually linked
Data or agorithm to
the other data(secure MPC)
External data to CBS
CBS data to External
Distributed data
Data management at CBS
Data (partially) managed by third party
Data are kept in environment of origin, data are matched by virtual connectivity
External data is brought to the CBS environment and
processed on CBS-site
Data or algorithm are encrypted and sent to other data sources for
matching without exposing the inputs of
others
CBS data are brought to an allocated highly
secure environment to be linked with other
data 1
3
2
4
![Page 19: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/19.jpg)
19
• These 4 patterns come with capabilities that need further investigation
• Privacy preserving analytic techniques• Secure multi party computation
• Data virtualisation and data abstraction• Metadata management
Data storage and access
![Page 20: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/20.jpg)
Data virtualisation architectureDATA CONSUMERS
VARIOUS DATA SOURCES
The image part with relationship ID rId3 was not found in the file.
The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file.SQL Queries(JDBC, ODBC, ADO.NET)
Web Services(SOAP, REST, OData)
Web-based catalog & search
Secure delivery(SSL/TLS)
DATA CONSUMERS
The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file.The image part with relationship ID rId3 was not found in the file.
The image part with relationship ID rId5 was not found in the file. The image part with relationship ID rId8 was not found in the file.The image part with relationship ID rId10 was not found in the file. The image part with relationship ID rId12 was not found in the file. The image part with relationship ID rId14 was not found in the file. The image part with relationship ID rId16 was not found in the file. The image part with relationship ID rId18 was not found in the file.
The image part with relationship ID rId20 was not found in the file. The image part with relationship ID rId26 was not found in the file. The image part with relationship ID rId28 was not found in the file. The image part with relationship ID rId30 was not found in the file. The image part with relationship ID rId32 was not found in the file. The image part with relationship ID rId34 was not found in the file. The image part with relationship ID rId36 was not found in the file. The image part with relationship ID rId38 was not found in the file. The image part with relationship ID rId40 was not found in the file.
MPP Processing*
Relational Cache
Corporate Security
Monitoring & AuditingThe image part with relationship ID rId43 was not found in the file.
MetadataRepository
Execution Engine & Optimizer
DATA VIRTUALISATION
{ }JSON
Data
Abs
trac
tion
* Massive Parallel Processing 20
![Page 21: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/21.jpg)
21
Metadata modelling
Person
Petra
Person
Peter
Is_married_with
Living_together
Name….: PeterBorn: 1 sep 1991Twitter…: @peter
Label Relations Node Properties
![Page 22: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/22.jpg)
22
Computing power and analytics
• Explore and Confirm• Deductive; data analysis to explain, check or validate ideas
• Inductive; data analysis to generate new ideas
• Edge analytics• Analysis and data quality framework are brought to the data
gathering devices instead of moving the data to the (centralized) analytics and quality frameworks
![Page 23: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/23.jpg)
23
Challenges and constraints
![Page 24: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/24.jpg)
24
• Data gap
• Burden to society and response rates
• Privacy protection and difficult acces tot data tot data
• Methodology
Challenges
![Page 25: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/25.jpg)
25
Future data collection
![Page 26: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/26.jpg)
26
• Signalling trends and potential future needs• Meet the public demand for information• Data hub• CBS fulfils role of a platform• Enabling broad cooperation between
• Governmental bodies• Municipalities• Companies• Scientific institutes
CBS in relation to data owners & end users
![Page 27: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/27.jpg)
27
Data ecosystemsDepartments
Municipalities
Scientific Institutes
Provinces
CBS
Science
Compani
es
Data infrastructure
CBS
Science
Compani
es
CBS
Science
Compani
es
CBS
Science
Compani
es
(OPEN) DATA
People
CBS
ScientificinstitutesCompanies
![Page 28: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/28.jpg)
28
• Datascouting• Awareness of data sources being there• Usability and availability of the data source• Organization and facilitation of the acquisition, tapping into and
unlocking of new data sources
• Data scout en community• Link between internal and external stakeholders
Tapping into and unlocking new data sources
![Page 29: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/29.jpg)
29
• CAWI hybrid mode• Multi mode and multi source• Custom-fit data collection• Experimenting
• Crowdsourcing• Interactive Voice Response
• Validation of alternative sources / data
Surveys – primary data collection
![Page 30: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/30.jpg)
30
Call to ACTION
![Page 31: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/31.jpg)
31
• Methodology• Collect – Connect – Link• Metadata• Proprietary sensor networks• Legal frameworks and social acceptability
Call to action
![Page 32: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/32.jpg)
32
Methodology on combining data• Multiple collection modes and different data sources
• units to be matched do not equal source units• sources do not contain overlapping units• matching errors• variables in multiple sources with different measurement errors
• Complex mixed mode designs
• Extent and/or combine existing techniques• Probabilistic matching• Matching with supervised machine learning• Synthetic matching
• Integration by design
![Page 33: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/33.jpg)
33
Integration by design
![Page 34: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/34.jpg)
34
Sensor networks - precision agriculture cycle
• Draw parcels• Yield potential• Tractor lanes
• Additional fertilization, pesticides & water
• Based on sensordata
• Fertilization• Variable
planting
• Harvest• Storage
Winter Spring
AutumnSummer
![Page 35: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/35.jpg)
35
…and also….
CDL
• Hotspots• Portal• Reference Ledger Scheme
• DNB – e-recognition• Bd – Digi coupling• UWV – Apache NiFi• Bol.com - Cloud
• Post Doc
• Primary data collection 2.0
• Redesign HBS• Essnet smart surveys
![Page 36: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/36.jpg)
36
Summary
Collect
Connect
Process
Integrate
Analyse
Discover
Disseminate
Interact
Protect
Scale-up
UDC, PDC, ADC, DDC…
Cloud
Better: new sources
Better: new insights
Faster: “time to market”
Nicer: user experience
Cheaper: flexible options (Agility)
![Page 37: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/37.jpg)
37
Conclusions
• Influence of NSI on content of captured data diminishes• Availability and technology greatly determine the data to
be collected• New sources increase complexity
• Validation, measuring errors & biases, mixed modes and multiple sources are a challenge
• AND possibilities• New – More – Cheaper – Faster
![Page 38: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/38.jpg)
38
Conclusions
• Data collection becomes Advanced
• Data collection through data connection
• Statistical production process has to follow
• Integration by design
![Page 39: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/39.jpg)
39
![Page 40: Future Advanced Data Collection - UNECE...Future data collection. Strategic assignments for NSO’s. Shaping the future by being smart. Trends. Challenges and constraints. Future data](https://reader033.vdocuments.site/reader033/viewer/2022042414/5f2f4c82876d0928801f975b/html5/thumbnails/40.jpg)