the future of statistical data collection? challenges and opportunities johan erikson (statistics...
Post on 26-Dec-2015
219 Views
Preview:
TRANSCRIPT
The Future of Statistical Data Collection?Challenges and Opportunities
Johan Erikson (Statistics Sweden)
Gustav Haraldsen (Statistics Norway)
Ger Snijkers (Statistics Netherlands)
Seminar on New Frontiers for Statistical Data Collection31 October – 2 November 2012, Geneva, Switzerland
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 2
Outline
1. Fundamental challenges
2. How do we meet these challenges?
3. Are we prepared? • Traditional survey perspectives:
- Management- Methodology
• Beyond the traditional survey framework
4. Conclusions
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 3
1. Fundamental challenges
Official Statistics in a changing world!
Three main challenges:
1. A shift in the balance of power between survey organisations and respondents
2. New competitors may make Official Statistics redundant
3. Globalisation of the economy
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 4
A shift in the balance of power
• Dillman et al. (2009):
“Surveys are now respondent driven, rather than driven primarily by the needs of
survey organizations.”
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 5
A shift in the balance of power• Traditionally – data collectors in control - Responding to survey was seen as a duty – not any more - The change can be seen clearly by studying the attitude of
respondents in different age groups
• Business organisations focus more on production and competiveness
• The means of communicating are changing• Respondents are getting harder to reach, and can
control who they want to be reached by
Conclusion: We need to adapt to respondents more and more in orderto get their cooperation using all kinds of comm. means.
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 6
New competitors
• Groves (2011) - “Organic vs. designed data”:
“We’re entering a world where data will be the cheapest commodity around, simply because
society has created systems that automatically track transactions of all sorts.”
• E.g. transactions on social media:3.2 million new entries/day, in the Netherlands
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 7
New competitors
• Data are everywhere: “Big Data” Conferences• New competitors use these data to produce new
statistics/information in real time:- E.g. Google’s stock prices index (GOOG)
• This challenges the way traditional (NSI) statistics are produced:- founded in survey methodology, with a focus on quality
• The new information maybe directed at different users
Conclusion: • What is our position in the information market?• Can we produce timely, relevant and competitive stats?
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 8
Globalisation
• UNECE (2011) - The Impact on globalization on National Accounts:
“The increasingly global nature of economic transactions and arrangements presents a
challenge to the application of national accounts concepts and the use of data collection and compilation systems for
measuring developments in the domestic economy.”
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 9
Globalisation
• Traditional indicators that show a society’s development may no longer be valid
• The concepts and units (needed to compile these indicators) are becoming irrelevant to globalised businesses:
- If the concepts are irrelevant to providers, data will be hard to get - Offices in any part of the world may have to be contacted
Conclusion: • The definitions, measurement and compilation of
traditional indicators needs to be reviewed• The concepts and units need to be revised, • The data collection methods and procedures need to be
adapted in international perspective (relates 1st challenge)
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 10
Data Collection Strategy - 3 steps:1. Re-use of available data
• Data sharing & data warehousing
2. Use of new registers and other secondary sources • Traditional government-based registers • Information from private businesses
• “Organic/Big Data” sources
3. Primary data collection:1. EDI technologies, like XBRL
2. Web surveys 3. traditional modes: paper, CATI, CAPI
• Using new communication technologies • Reciprocity: report back to respondents
Multi-source designs
Mixed-mode designs
2. What is done? The answer to these challenges so far
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 11
3. Is this sufficient? Are we prepared re: challenges?
• To answer this question we looked at the internal processes in our Statistical Offices
• What are we actually doing …?• What is our short-term focus?
• The Quality Diamond: (Haraldsen; in: Snijkers, Haraldsen, Jones & Willimack, 2013, Wiley)
Integrates four quality perspectives on statistics:
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 12
SpecificationsCoverageSamplingNonresponseMeasurementProcessing
RelevanceAccuracyTimelinessPunctualityAccessibilityClarityComparabilityCoherence
(Eurostat Quality Dimensions, 2011)
Question order effectsMode effectsCross-cultural effects
CostsTechnologyEthicsTimelinessPRB
The Quality Diamond• Management perspective• Methodology perspective
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 13
Non-response
6 Managing customer demands and expectations
Mixed-mode effectsData Collection Strategy effects
1 Costs2 Timeliness3 Flexibility of process planning4 Innovation of process5 Response Burden7 Planning staff8 Systems and tools9 Culture
The Quality Diamond:The Management Perspective
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 14
Non-responseMeasurementCoverageSampling
Accuracy
Mixed-mode effectsQuestionnaire effectsData Collection Strategy effects
Response BurdenInnovation of data collection methods
The Quality Diamond:The Methodology Perspective
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 15
Differences in focus:
• Management (mostly) interested in becoming better (more efficient) within traditional survey framework.
Short-term challenges, while running the surveys: - Standardisation and integration of processes - Innovating the current processes within the Data
Collection Strategy, like planning mixed-mode designs
• Methodology driven by quality considerations: - Reducing survey design effects and Total Survey Error - Survey process constraints not much considered
• The two perspectives hardly overlap
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 16
Beyond the traditional survey framework• Short-term focus – based on traditional
framework
We need to go beyond that framework: Considering challenge 1: balance of power
• The end of probability sampling? - High non-response levels and low representativeness:
responsive designs, web panels, post-survey adjustments - Use of registers: no sampling involved; non-sampling errors?
Towards multi-source/mixed-mode designs• The end of standardised questionnaires/data collection?
- From collecting data to collecting {data & metadata} usingdata capture methods (like XBRL)
- From standardised interviewing to conversational interviewingand data mining, using new technologies (smart phones, Skype)
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 17
Beyond the traditional survey framework
• Official data requirements and concepts – based on stove-pipe approach: traditional indicators, separate surveys
We need to go beyond that:• The end of traditional indicators to measure societal
developments?Considering challenge 3: Globalisation
- What information do we need? - How do we measure that information?
• The end of official statistics? Considering challenge 2: New competitors - What will be our position in the information market?
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 18
4. So, are we prepared?
• Standardisation and integration of data collection processes are a step in the right direction… but not enough
• Data Collection Strategies towards multi-source/mixed-mode designs are a step in the right direction… but not enough
Managers and methodologists need to collaborate, having common short-term and long-term focuses: a communication perspective!
We need to go beyond the traditional survey framework… and act now! The future is already here!
top related