the future of statistical data collection? challenges and opportunities johan erikson (statistics...

The Future of Statistical Data Collection?Challenges and Opportunities

Johan Erikson (Statistics Sweden)

Gustav Haraldsen (Statistics Norway)

Ger Snijkers (Statistics Netherlands)

Seminar on New Frontiers for Statistical Data Collection31 October – 2 November 2012, Geneva, Switzerland

New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 2

Outline

1. Fundamental challenges

2. How do we meet these challenges?

3. Are we prepared? • Traditional survey perspectives:

- Management- Methodology

• Beyond the traditional survey framework

4. Conclusions


1. Fundamental challenges

Official Statistics in a changing world!

Three main challenges:

1. A shift in the balance of power between survey organisations and respondents

2. New competitors may make Official Statistics redundant

3. Globalisation of the economy


A shift in the balance of power

• Dillman et al. (2009):

“Surveys are now respondent driven, rather than driven primarily by the needs of

survey organizations.”


A shift in the balance of power• Traditionally – data collectors in control - Responding to survey was seen as a duty – not any more - The change can be seen clearly by studying the attitude of

respondents in different age groups

• Business organisations focus more on production and competiveness

• The means of communicating are changing• Respondents are getting harder to reach, and can

control who they want to be reached by

Conclusion: We need to adapt to respondents more and more in orderto get their cooperation using all kinds of comm. means.


New competitors

• Groves (2011) - “Organic vs. designed data”:

“We’re entering a world where data will be the cheapest commodity around, simply because

society has created systems that automatically track transactions of all sorts.”

• E.g. transactions on social media:3.2 million new entries/day, in the Netherlands


New competitors

• Data are everywhere: “Big Data” Conferences• New competitors use these data to produce new

statistics/information in real time:- E.g. Google’s stock prices index (GOOG)

• This challenges the way traditional (NSI) statistics are produced:- founded in survey methodology, with a focus on quality

• The new information maybe directed at different users

Conclusion: • What is our position in the information market?• Can we produce timely, relevant and competitive stats?


Globalisation

• UNECE (2011) - The Impact on globalization on National Accounts:

“The increasingly global nature of economic transactions and arrangements presents a

challenge to the application of national accounts concepts and the use of data collection and compilation systems for

measuring developments in the domestic economy.”


Globalisation

• Traditional indicators that show a society’s development may no longer be valid

• The concepts and units (needed to compile these indicators) are becoming irrelevant to globalised businesses:

- If the concepts are irrelevant to providers, data will be hard to get - Offices in any part of the world may have to be contacted

Conclusion: • The definitions, measurement and compilation of

traditional indicators needs to be reviewed• The concepts and units need to be revised, • The data collection methods and procedures need to be

adapted in international perspective (relates 1st challenge)


Data Collection Strategy - 3 steps:1. Re-use of available data

• Data sharing & data warehousing

2. Use of new registers and other secondary sources • Traditional government-based registers • Information from private businesses

• “Organic/Big Data” sources

3. Primary data collection:1. EDI technologies, like XBRL

2. Web surveys 3. traditional modes: paper, CATI, CAPI

• Using new communication technologies • Reciprocity: report back to respondents

Multi-source designs

Mixed-mode designs

2. What is done? The answer to these challenges so far


3. Is this sufficient? Are we prepared re: challenges?

• To answer this question we looked at the internal processes in our Statistical Offices

• What are we actually doing …?• What is our short-term focus?

• The Quality Diamond: (Haraldsen; in: Snijkers, Haraldsen, Jones & Willimack, 2013, Wiley)

Integrates four quality perspectives on statistics:


SpecificationsCoverageSamplingNonresponseMeasurementProcessing

RelevanceAccuracyTimelinessPunctualityAccessibilityClarityComparabilityCoherence

(Eurostat Quality Dimensions, 2011)

Question order effectsMode effectsCross-cultural effects

CostsTechnologyEthicsTimelinessPRB

The Quality Diamond• Management perspective• Methodology perspective


Non-response

6 Managing customer demands and expectations

Mixed-mode effectsData Collection Strategy effects

1 Costs2 Timeliness3 Flexibility of process planning4 Innovation of process5 Response Burden7 Planning staff8 Systems and tools9 Culture

The Quality Diamond:The Management Perspective


Non-responseMeasurementCoverageSampling

Accuracy

Mixed-mode effectsQuestionnaire effectsData Collection Strategy effects

Response BurdenInnovation of data collection methods

The Quality Diamond:The Methodology Perspective


Differences in focus:

• Management (mostly) interested in becoming better (more efficient) within traditional survey framework.

Short-term challenges, while running the surveys: - Standardisation and integration of processes - Innovating the current processes within the Data

Collection Strategy, like planning mixed-mode designs

• Methodology driven by quality considerations: - Reducing survey design effects and Total Survey Error - Survey process constraints not much considered

• The two perspectives hardly overlap


Beyond the traditional survey framework• Short-term focus – based on traditional

framework

We need to go beyond that framework: Considering challenge 1: balance of power

• The end of probability sampling? - High non-response levels and low representativeness:

responsive designs, web panels, post-survey adjustments - Use of registers: no sampling involved; non-sampling errors?

Towards multi-source/mixed-mode designs• The end of standardised questionnaires/data collection?

- From collecting data to collecting {data & metadata} usingdata capture methods (like XBRL)

- From standardised interviewing to conversational interviewingand data mining, using new technologies (smart phones, Skype)


Beyond the traditional survey framework

• Official data requirements and concepts – based on stove-pipe approach: traditional indicators, separate surveys

We need to go beyond that:• The end of traditional indicators to measure societal

developments?Considering challenge 3: Globalisation

- What information do we need? - How do we measure that information?

• The end of official statistics? Considering challenge 2: New competitors - What will be our position in the information market?


4. So, are we prepared?

• Standardisation and integration of data collection processes are a step in the right direction… but not enough

• Data Collection Strategies towards multi-source/mixed-mode designs are a step in the right direction… but not enough

Managers and methodologists need to collaborate, having common short-term and long-term focuses: a communication perspective!

We need to go beyond the traditional survey framework… and act now! The future is already here!

the future of statistical data collection? challenges and opportunities johan erikson (statistics...

Documents

new frontiers

new competitors data

stastictal data collection

use of data collection

new information

designed data

data collectors

new entriesday