overview of european data from official statistics

69
Overview of European Data from Official Statistics Roxane Silberman CNRS/Réseau Quetelet and DwB coordinator With the support of Cyril Jayet, Marie Cros, Raphaëlle Fleureux, Alexandre Kych, Benoît Tudoux (CNRS-RQ) DwB WP3 and WP5 - CESSDA PPP WP10 DwB training course: Working with data from Official Statistics particularly the European Labour Force Survey Bucharest, 2 nd DwB Training Course, February 13-15, 2013

Upload: noe

Post on 03-Feb-2016

44 views

Category:

Documents


0 download

DESCRIPTION

Overview of European Data from Official Statistics. Roxane Silberman CNRS/Réseau Quetelet and DwB coordinator With the support of Cyril Jayet, Marie Cros, Raphaëlle Fleureux, Alexandre Kych, Benoît Tudoux (CNRS-RQ) DwB WP3 and WP5 - CESSDA PPP WP10 - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Overview of European Data from Official Statistics

Overview of European Data from Official Statistics

Roxane SilbermanCNRS/Réseau Quetelet and DwB coordinator

With the support of Cyril Jayet, Marie Cros, Raphaëlle Fleureux, Alexandre Kych, Benoît Tudoux (CNRS-RQ)DwB WP3 and WP5 - CESSDA PPP WP10

DwB training course: Working with data from Official Statistics particularly the European Labour Force Survey

Bucharest, 2nd DwB Training Course, February 13-15, 2013

Page 2: Overview of European Data from Official Statistics

Introduction • Main course on SILC, Eurostat microdata

• Yet it is important to have a larger overview of official microdata in Europe European integrated microdata (Eurostat) come from national microdata that are quite diverse

Raises a lot of methodological issues about comparability: ex. SILC May be complemented

by related microdata available at national level more detailed, other variables, and sometimes on a longer period

by other microdata available at European and national levels

• Provide a short and comprehensive overview of the rich resources of official microdata in Europe What is available at national and European level Relations between the different levels How and where to access these microdata

While helping locating in this landscape the SILC, related national microdata and complementary sources on similar topics

Page 3: Overview of European Data from Official Statistics

Outline• Historical background for national and European microdata and consequences• Overview of official microdata in Europe

Integrated European microdata (Eurostat) SILC

Other European microdata National microdata collected and harmonized in European databases

Related topics to SILC Other national microdata

Related topics to SILC• How to to find information and metadata ?• How access microdata across borders ?

Eurostat access and other European microdata access Transnational access to national microdata

• Support by DwB Transnational access calls CIMES

Page 4: Overview of European Data from Official Statistics

I. Historical backgrounds for national and European microdata and consequences

- A vast, complex and moving perimeter- Historical backgrounds for national microdata- Historical backgrounds for European microdata- Some consequences for research

Page 5: Overview of European Data from Official Statistics

Official microdata: a vast perimeter• Different words (official, government, national etc..)

In any cases, statistics provided by government bodies A larger perimeter than NSIs and Eurostat

• Microdata provided by : National level

National Statistical Institute National Statistical administrations coordinated by the NSI + fiscal data Central banks Government agencies particularly those in charge of Social security, health, pensions … Local authoritiesNumber of government bodies producing and providing data vary according to the organization

of the each statistical system and degree of centralization European level

Eurostat European Central Bank European Commission and agencies Databases held by universities

Page 6: Overview of European Data from Official Statistics

… complex and moving• Perimeter may change according to the decision of governments/NSIs

• See France: • Customs statistical department went out and in • Fiscal data recently included as a Statistical department coordinated by INSEE • Cereq microdata on transition from school to work went out …

• Household finance and consumption surveys (HFCS) move to central banks coordinated by the BCE • Mixed status for some government agencies (social security, unemployment …)

• What about data to be certified by NSIs ? • Under different legal framework (surveys, administrative data, business data, fiscal data, health data,

financial data …) with consequences for access • Perimeter does not necessarily cover similar data in the different countries depending on role of other

producers (universities …) and historical changes (Eastern countries recently joining the EU)

Page 7: Overview of European Data from Official Statistics

Different types of microdata

Censuses or registers + longitudinal samples from censuses Individual and households surveys + birth cohorts and panels Business surveys Administrative data (frequently longitudinal databases) Combined datasets

Administrative datasets – More common in countries based on registers– Now increasingly the case in all countries (yet requiring a common

identifier) Administrative data and surveys

Page 8: Overview of European Data from Official Statistics

Historical backgrounds for national microdata

• Statistical systems initially built at national level

• From diverse sources and bodies, piece by piece, Toward an increasing coordination role of the NSI Yet keeping traces from the construction even in centralized statistical system where some

bodies still remain apart.

• Important differences Surveys and registers countries Centralized vs decentralized/coordinated Political systems

Regional autonomy (Spain) Federal system (Germany and the landers) UK and Scotland France and overseas departments National and local authorities gaining importance

Page 9: Overview of European Data from Official Statistics

Main topics

• Similarities Households: Demography, migration ,family, employment, education, budget and income

• More variety Housing, transportation, health, social networks, opinions, social mobility …

• Business Wages, work organization, innovation… Increasing number of administrative data combined to lower the respondent burden (firms more

reluctant to answer)

• Number of official datasets may vary depending on: Registers or surveys systems Degree of centralisation Political changes: Eastern countries Role of the universities (France vs Germany) Examples from first results of DwB WP5 : short list in Eastern countries in line with Eurostat

microdata, longer list in France, medium in Germany

Page 10: Overview of European Data from Official Statistics

Historical backgroundEuropean level

• Increasing harmonization process at international level since WW2 led by international organisations (UN, OECD…) mostly by encouragement and persuasion

• European framework specific as there is a political and legal framework European Statistical System development started with the CECA during the 50ies, progressively developed

during 30 years, new start and developments since the 90ies towards more integration Harmonization process for variables, classifications, methodology Yet relying frequently on pre-existing national microdata and on the subsidiarity principle

• As a consequence, European integrated microdata are based on national microdata from various sources (surveys, registers,

administrative basis) depending on countries Methodological problems (see Training course part 3 on SILC)

Dissemination under European bodies yet access to national part under decision of countries Not all integrated microdata include all countries

• Access still burdensome even if progress expected in the future• Level of anonymization high due to differences in countries size and in countries requirements

Yet access to national part may be possible at national level. May be more detailed and accessible in some countries May include other questions and variables May offer a wider historical perspective if pre-existing in the countries (yet not for all countries) Yet the harmonization process may impact the series and determine breaks in the series at national

level

Page 11: Overview of European Data from Official Statistics

Some consequences for working with data

• At European level European microdata may gather data coming from surveys or from registers Some harmonised databases may include government and non government

microdata• At national level

Depending on the degree of centralisation for production and/or dissemination information and access are more or less fragmented and may have move from one body to another one

NSI unique or main provider (Netherlands) vs NSI and Statistical departments and other government bodies (France)

Recently the French Income survey moved from the NSI to the Central Bank Depending on the legal status access may be more or less accessible for

researchers For comparative research projects requiring access to national microdata

Researchers face “silos” for information and access

Page 12: Overview of European Data from Official Statistics

II. European and national microdata

Three subsets from a European perspective

European integrated microdataNational microdata harmonized at European level Other national microdata for comparative research

Page 13: Overview of European Data from Official Statistics

A. European integrated microdata

Produced and provided at national level and integrated and provided at European level by European government bodies To remind : national part may differ at national level and at European levelA part are pre-existing surveys integrating harmonization requirements

Eurostat European Central Bank, European Commission and other European government bodiesOthers under EU regulations or recommendations of Eurostat

Page 14: Overview of European Data from Official Statistics

A.1 Eurostat microdata

Labour Force Survey (LFS) European Community Household Panel (ECHP) Statistics on Income and Living Conditions (SILC) Adult Education Survey (AES) Community Innovation Survey (CIS) Structure of Earnings Survey (SES)

Page 15: Overview of European Data from Official Statistics

Related national microdata for LFS and SILC

• LFS : a wider historical perspective in some cases• SILC: different sources

More variables in some cases Surveys and registers

Page 16: Overview of European Data from Official Statistics

Countries Years CUF PUF SUF On-site access

Remote Execution

Remote Access

Czech 1995-2011

Estonia 1995-2011 X X X X

Finland 1969-2011 X X

France 1962-2010 X X X

Hungary 1992-2011 X X

Italy 2004-2011 X

Latvia 1996-2011 X

Lithuania 1998-2011 X X

Norway 1972-2011 X X

Poland 2000-2011

Portugal 1992-2011

Slovakia 2002-2011 X

Slovenia 1997-2011 X X

Spain 1964-2011 X X

Switzerland 1991-2011 X

UK 2002-2010 X X X X

Labour Force Survey (LFS) at national level: a wider historical perspective

Page 17: Overview of European Data from Official Statistics

SILC and the Estonian Social Survey

• “ESS is the Estonian branch of a pan-European survey of income and living conditions called the EU-SILC (…). Statistics Estonia, however, has added questions, which are of interest to the domestic consumers of Estonia, to the EU-commissioned survey, and attempts to have the survey be a combination of Estonian and European data requirements.”

• In 2004, four modules were added (…). They were all commissioned by Estonian domestic consumers. The topics of the four modules concerned social contacts; family attitudes and political views; crime, violence and feeling of security; and ethnic integration.

• In 2005, there were three modules in ESS: one by order of Eurostat and the other two by domestic consumers. The topic of the Eurostat module was “Social origin”(…) Estonian domestic modules were entitled “From school to work” and “Trade unions and collective agreements”.

Page 18: Overview of European Data from Official Statistics

SILC and French EPCV and SRCV

• SILC (SRCV) starts in 2004, but… Living conditions 1978-1979, 86-87; 93-94 European Community Household Panel (1994-2001) Permanent Living Conditions Survey EPCV (1996-2004)

• The current French SRCV system took over from the former permanent survey of living conditions (EPCV) system in 2004: it reprises some questions from the EPCV to produce social indicators considered indispensable for the specific needs of the French statistical institute but not required within the European framework.

Page 19: Overview of European Data from Official Statistics

A diversity of situations

• The Belgian datasets are much more detailed than the delivered files to Eurostat • SILC datasets from Eurostat do not contain Swiss data (2007 - 2010).• The Great Britain component of the EU-SILC dataset is collected by the Office for

National Statistics (ONS) as part of the General Lifestyle Survey (GLF) (held at the Archive under Special Licence access conditions - see GN 33403).

• The Northern Ireland component is collected by the Northern Ireland Statistics and Research Agency (NISRA) as part of the Living Conditions Survey (LCS) (not currently held at the Archive).

Page 20: Overview of European Data from Official Statistics

A Danish example of register :THE REGISTER FOR HEALTH AND SOCIAL CONDITIONS

• The purpose of the register is to provide Statistics Denmark, the Ministry of Health and the National Board of Health with readily available information from public registers on the state of health of the Danish population and relevant socio-economic background conditions. Another purpose is to provide data for research in social medicine.

• The register was established in cooperation between the three authorities• The collection of data for the register began in 1994, starting with data from registers with

information available from 1977 onwards: The National Register of Patients, the Register of Population Statistics and the Employment Classification Module.

• This was followed in 1995 by the inclusion of data from the following registers: the Register of Causes of Death, the Register of Health Insurance Statistics and the

Register of Coherent Social Statistics.• In 1997 a final agreement was made defining the contents of the register, including data from

three more registers: The Integrated Database for Labour Market Research, the Educational Classification

Module and the Register of Building and Dwelling Statistics. At the time, specification was made for the future inclusion of data on medical births,

voluntary abortions and deformations in birth.

Page 21: Overview of European Data from Official Statistics

A.2 European Central Bank microdata

• Household Finance and Consumption Survey• Every 3 years• First deliverable in 2013• No pre-existing national survey in some countries while older waves in others

France: Every 6 years and oldest waves Questionnaire 112 pages in France vs 65 pages for the European survey, yet some

variables collected at European level not in the French survey Adaptation of the survey was needed (break in the serie) Moved from the NSI to the Central Bank (with consequence for researcher access)

Page 22: Overview of European Data from Official Statistics

A.3 Other European government microdata

• The European Commission, the Directorate-General for Economic and Financial Affairs (DG ECFIN) Business and Consumer Surveys

• EUROFOUND Surveys on working conditions

Page 23: Overview of European Data from Official Statistics

The Business and Consumer Surveys

•The European Commission, the Directorate-General for Economic and Financial Affairs (DG ECFIN)• manages a network of national institutes

to conduct a harmonised EU programmeof 6 business and consumer tendency surveys

(quarterly or monthly from 1985, 1995 for services).IndustryServicesConsumersRetail tradeBuildingInvestment

and some others

Page 24: Overview of European Data from Official Statistics

EUROFOUND and surveys on working conditions The European Foundation for the Improvement of Living and Working Conditions• An autonomous EU agency• Set up by the Council Regulation (EEC) No. 1365/75 of 26 May 1975• Contributes to the planning and design of better living and working conditionsGoverning board:• 3 representatives per country, for 28 countries (EU27 and Norway)• The ministry in charge of labour• The employers• The unions• 3 representatives for the European Commission• The DG Employment• The DG Health and consumer protection• The DG Research end innovation• Based in Dublin with a Brussels Liaison Office

Page 25: Overview of European Data from Official Statistics

3 surveys combining companies and employees surveysThe European Working Conditions Survey (EWCS) : 1990, 1995, 2000, 2005, 2010• Working conditions and the quality of work and employment

The European Quality of Life Survey (EQLS) : 2003, 2007, 2010• A broad range of indicators of quality of life, both objective and subjective

The European Company Survey (ECS) : 2004, 2009, 2013• Workplace practices based on the views of both managers and employee representatives The exemple of the EWCS surveys• Year 1990 1995 2000 2005 2010• Number of workers 12500 15800 21500 30000 44000• Number of countries EC12 EU15 EU15+12 EU27+4 EU27+7• A questionnaire development expert group: Eurofound's Governing Board, other EU bodies

(EU Commission, Eurostat, European Agency for Safety and Health at Work), international organisations (OECD, ILO), national statistical institutes and leading European experts in the field

• Implemented by INRA Europe then Gallup Europe, with national fieldwork partners• Datasets available for download at the ESDS (UKDA)

Page 26: Overview of European Data from Official Statistics

A. 4 Others microdata under EU regulations or Eurostat recommendations for harmonization

• Household budget surveys (HBS) Eurostat recommendations about methodology and harmonization

• Information and Communication technologies surveys (TIC) Regulation n°808/2004 and updated regulation 1006/2009 linked with the

European roadmap for the TIC

Page 27: Overview of European Data from Official Statistics

B. National microdata harmonized at European level

• IECM (IPUMs international) Censuses

• MTUS Time Use Survey

• LIS (Luxembourg Income Study) Household Budget Survey

Collected and a posteriori harmonized by Universities Archives (MTUS, LIS, IECM)More detailed microdata often accessible at national level

Page 28: Overview of European Data from Official Statistics

IECM + national disseminationIECM soon + national dissemination

Only national disseminationNo information

B.1 IECM/IPUMS and national dissemination of European censuses

Page 29: Overview of European Data from Official Statistics

More censuses microdata available at national level

• The case of the 2001 census

• The example of 2 European countries :United Kingdom France (new census, yearly since 2006, the example of 2008)

Page 30: Overview of European Data from Official Statistics

United KingdomSamples of anonymized records (SARs)• The Individual Licensed SAR

3 % (1 800 000 people) UK : 9 government office regions for England, and Wales, Scotland and Northern IrelandAvailable from CCSR (a charge may apply), also available from IPUMS

• The Special Licence Household SAR (SL-HSAR) 1 % (245 000 households) England and Wales

Available from the UK Data Archive• The Small Area Microdata (SAM)

5 % (2 900 000 people) ±500 Local authorities

Available from CCSR (a charge may apply)

The Controlled Access Microdata Sample(s) (CAMS) Both the individual and household SARs files Available in much greater detail

Accessible in safe settings in all ONS sites, for approved research projects.Only for England and Wales

Page 31: Overview of European Data from Official Statistics

France : Available as PUF from Insee website The standard files (PUF)

• The dwelling file• The individual file by regions

Available from IECM/IPUMS• The individual file by constituencies• The individual file by place of work

3 standard (PUF) individual files for residential mobility They differ for confidentiality reason, • Place of residence at the census time and previous residence (also abroad).

For departments, great communes or all communes• The individual file for travel to work mobility

From commune to commune)• The individual file for school mobility

From commune to commune

Page 32: Overview of European Data from Official Statistics

France: Available for researchers via Data ArchiveThe SUF files • For the last Census available for the end of 2012.• 2 to 3 files:

An individual file with a complete set of variables, at the region levelA family file.An individual file of the annual census survey, at the region level.Available from the Réseau Quetelet

The Secure Access Center to Data (CASD, Centre d’Accès Sécurisé aux Données, GENES-Réseau Quetelet)

• All the microdata census files (and others …)Available in the most detailed formSecure remote access

Page 33: Overview of European Data from Official Statistics

Pays Enquête Année Income Unit Data Collection

Cyprus LWS 2001 Primary Economy Unit Central Bank of Cyprus

and University of Cyprus

Finland LWS 1998 Household Wealth

Survey

Statistics Finland

Germany LWS 2001 German Socio Economic

Panel

German Institute for

Economic Research, DIW

Italy LWS 2002 Survey of Household

Income and Wealth

Bank of Italy

UK LWS 2000 British Household Panel

Survey + cross national

equivalent files

Institute for Social and

Economics Research

LIS 1999 Family Resource Survey Department for Work

and Pension , ONS,

National Centre for Social

Research

B.2 LIS is a cross-national data center, located in Luxembourg. LIS is home to the Luxembourg Income Study Database (LIS) and the Luxembourg Wealth Study (LWS)Database.

Page 34: Overview of European Data from Official Statistics

Countries Years CUF PUF SUF On-site access

Remote Execution

Remote Access

Czech 1999-2010          

Estonia 2000-2010 X        

France 1972-2006   X     x 

Hungary 1993-2011   X X    

Italy 1953-2011          

Latvia 1996-2011   X     X

Lithuania 2003-2008   X      

Norway 1988   X X    

Poland 1986-2010          

Slovakia 1994-2011   X      

Slovenia 2001-2012     X    

Spain 1998-2010 X   X    

Switzerland 2000-2005   X      

Household Budget Survey

Page 35: Overview of European Data from Official Statistics

France Time Use Survey INSEE

BELGIUM

1966

The Multinational Comparative

Time-Budget Research Project

Pierre Feldheim and Claude Javeau,

Sociological Institute, Free University of

Brussels

BULGARIA

1988

The 1988 Bulgarian National

Time Use Survey

Central Statistical Office, Institute of

Sociology at the Bulgarian Academy of

Sciences

FINLAND

1987-88

Time Use Survey Statistics Finland

HUNGARY

1965

The Multinational Comparative

Time-Budget Research Project

Sociological Research Group, Hungarian

Academy of Sciences

ITALY 1979/80 Il Tempo della Citta. Una Ricerca

Sull'uso del Tempo Quotidiano in

una Metropoli

University of Turin

UK 1961 The People's Activities BBC

UK 2005 Omnibus Survey, One Day Diary

of Time Use Module

The Office for National Statistics coordinated the

study and collected the data. The Institute for

Social and Economic Research at the University of

Essex transferred the diaries into coded electronic

data.

The Centre for Time Use Research collects Time Use Surveys

Page 36: Overview of European Data from Official Statistics

Countries Years CUF PUF SUF On-site access

Remote Execution

Remote Access

Estonia 1999-2010 X X

France 1966-1999 X x

Italy 1988-2009 X X

Norway 1971-2010 X X

Portugal 1999

Spain 2002-2009 X X

Time Use Survey

Page 37: Overview of European Data from Official Statistics

Household Budget Surveys, Information and Communication Technology Surveys and Time Use Surveys

HBS + TIC + TUSHBS + TICTIC + TUSHBS + TUSTICNo information

Page 38: Overview of European Data from Official Statistics

C. A lot of other national OS usable for comparative research

• Various surveys• Related surveys on living conditions (other than those related to the SILC) • Other examples :

• Social mobility• School leavers and transition from school to work

• Administrative microdata increasingly combined and used by researchers• Examples

• Employers and employees • Social security, pensions

Page 39: Overview of European Data from Official Statistics

Other national surveys on living conditions conditionsCount

ry

Denmark

The register for health and social conditions 1977-2012

Family allowance and child benefits 1957-2012

Estonia

Estonian Social Survey 2004-2010

Household Budget Survey 2010

France Living conditions 78-79; 86-87; 93-94

Permanent Living Conditions Survey (EPCV) 1996-2004

The Statistical survey on income and living conditions (SRCV)

2004-2009

Survey of users of accommodation and hot meal distribution services (Homeless people)

2001

United Kingdom

General Lifestyle Survey (GLF) 2000-2008

Living Standards During Unemployment 1983-1984

English Housing Survey 2008-2011

Norway

Norwegian Level of Living Study 1973-2007

Study on housing conditions among low-income families

1995

Welfare and level of living among the very frail elderly

2000

Page 40: Overview of European Data from Official Statistics

Social mobilityRichard Breen ed., 2004. Social Mobility in Europe. Oxford University Press, Oxford.

Page 41: Overview of European Data from Official Statistics

Country Sources of data Years for which data are

included

Data Producer

Germany Zumabus 1976-7, 1979(2), 1980, 1982 ZUMA-GESIS : data come from collections

firms that carry out most of the academic

social research in Germany

German socio-economic panel 1986, 1999 DIW, German Institute for Economic

Research

France Formation qualification

professionnelle Insee surveys

1970, 1977, 1985, 1993 INSEE

Italy National survey on social

mobility

1985 Universities of Bologna, Trento, and

Trieste

Italian Houselhold longitudinal

survey

1997 Universities of Trento, Milano, and

Bologna, Istituto Trentino di Cultura, Istat

Great Britain General household survey 1973, 1975-6, 1979-84, 1987-

92

ONS

Poland Slomczyski 1989 1988 Polish Academy of Sciences

Hungary Social mobility and life history

survey

1973, 1983, 1992 Hungarian Central Statistical Office

Sources: Richard Breen. 2004. Social Mobility in Europe. Oxford University Press, Oxford.

Page 42: Overview of European Data from Official Statistics

Youth cohorts and transition from school to work surveys

• Numerous surveys conducted in European countries Statistical Offices, France, Italy, Norway … Ministry of education, employment, statistical departments Denmark,

Germany, France, Ireland, Scotland … Government agencies (Unemployment, others…) Belgium, France … Universities, UK…

Source CEDEFOP, Christine Mainguet, La transition du système éducatif vers la vie active (The transition from school to work) Exploitation des données statistiques nationales (Statistical sources).

Page 43: Overview of European Data from Official Statistics

Employers-Employees A guide to Linked Employer-Employee Data Sources in the EU and Beyond

See Tanvi Desai, London School of Economics, 2008

Page 44: Overview of European Data from Official Statistics

Linked Employer-Employee Data sources, examples of national surveysCountry National surveys Data producer Access notesCzech Republic

Information System on Average Earnings (ISAE)

The Czech Ministry of Labour commissions the private agency TREXIMA

Direct access is only available on-site at TREXIMA

The Structure of Earnings Survey for the Czech Republic is derived from the ISAE there is no access to the Czech SES data at the national level.

Spain INE does not provide access to any other linked employer-employee or firm panel data resources than SES. The Banco de España conducts an annual survey of non-financial firms, the Central Balance Sheet Data. However, the microdata for this survey are only available to researchers affiliated with the Banco de España

Germany LIAB (Linked Employer-Employee Data of the IAB)

IAB (Establishment Panel )

Data access is possible via on-site use and afterwards via remote data access also.

The LIAB data is a linked employer-employee dataset constructed from the IAB Establishment Panel and the Federal Employment Agencies employment statistics.

DESTATIS (employment statistics)

France REPONSE (Relations Professionnelles et Negociations d’Entreprise)

DARES DARES

COI (Changement Organisationnels et l’Informatisation)

DARES Réseau Quételet ,(SUF) The COI is used for France’s contribution to Eurostat’s ICT survey

Enquête Familles at Employeurs

INED

DADS INSEE Réseau Quételet, (SUF)Labour cost and structure of earnings survey (Ecmoss)

INSEE Réseau Quételet, (SUF) The Structure of Earning Survey for France is a part of this survey

Page 45: Overview of European Data from Official Statistics

The Impact of Social Security Contributions on Earnings: Evidence from administrative data in France, Germany, Netherlands an UK. A. Bozio, J. Grenet Research project 2011

Page 46: Overview of European Data from Official Statistics

Country Data sources Years Data collection ProviderFrance DADS 1993- Fiscal and social

administration, INSEEReseau Quételet (CMH, GENES)

Germany Sample of Integrated Labour Market Biographies (SIAB)

1975-2008 IAB IAB

Lohn- und Einkommensteuerstatstik – faktisch anonymisierte Daten (FAST)

1992-2004 Fiscal administration; DESTATIS

DESTATIS

Verdienststrukturerhebung (VSE) 1990-2006 DESTATIS DESTATIS

United Kingdom New Earnings Survey (NES) 1975-2003 ONS Secure data service, UKDA

Annual Survey of Hours and Earnings (ASHE) 2004-

Annual Business Inquiry

Bozio Antoine. The Impact of Social Security Contributions on Earnings: Evidence from administrative data in France, Germany, Netherlands an UK. Research proposal submitted in September 2011 to the Open Research Area (ORA) call

Page 47: Overview of European Data from Official Statistics

V. National and European microdata: where can I find the information and

metadataSources

National microdata disseminationNesstar

Microdata dissemination at European level

Page 48: Overview of European Data from Official Statistics

Sources

• Sources of information : NSI websites and CessdaPPP

• 1 National Statistical Institute selected per country

• These are general trends, there could be little variations due to lack of information available on the web or very recent changes

Page 49: Overview of European Data from Official Statistics

Metadata dissemination for national microdata: NSIs and Archives

Page 50: Overview of European Data from Official Statistics

Metadata dissemination at European level• Each European body• CESSDA only for some national microdata (depending on Archives members)• No single point of access even at national level

Some countries are opening portal for access to official data (open data initiatives) yet mostly for aggregate data

In some countries, archives gather metadata from different government producers (yet not all)

Page 51: Overview of European Data from Official Statistics

Looking for data: Nesstar

• Nesstar: A widely used tool

• Examples: Australia, Canada, Denmark, Finland, France, Germany, Hungary, Ireland, Lithuania, Netherlands, Norway, Sweden, Slovenia, United-States, etc.

• A unified way to look for data and metadata (documentation describing the data)

• Allows to browse into variables (instead of looking into the questionnaire)

Page 52: Overview of European Data from Official Statistics
Page 53: Overview of European Data from Official Statistics

VI. National microdata: where and how can I access the data

AccessTypes of access

Data providers for official statistics in the European countries Transnational access

Eurostat access and other European microdata accessTransnational access to national microdata

Page 54: Overview of European Data from Official Statistics

Types of accessFrom highly anonymized to highly detailed microdata•Campus files•Public use files•Scientific Use files•Confidential, highly detailed, sensitive microdata•On site, remote execution, remote access

Terminology issuesDifferent types of accreditation procedures, application forms, criteria for each type and for each country/producer/providerTransnational access not always possible due to differences in legal frameworks and/or interpretation of the laws

Page 55: Overview of European Data from Official Statistics

Eurostat and other European bodies• Eurostat still burdensome

Yet now free of charge Network contracts

• New regulation expected for access to confidential microdata, Yet new procedures for accreditation (2 steps) Currently access on site Remote access network in project (DARA ESSnet)

• European Central Bank in progress • LIS remote execution• IPUMS and IECM free and easy, yet highly anonymized

Page 56: Overview of European Data from Official Statistics

Transnational access to OS in Europe• Some European mapping about type of access

Transnational access for PUFs, SUFs, and confidential data available in safe centers and/or via remote access.

Sources of information : NSI websites + DwB Bucharest Eastern countries workshop + information collected within DwB Wp3 and CessdaPPP)

1 National Statistical Institute selected per country (does not include the NSAs and other government bodies)

These are general trends, there could be little variations due to lack of information available on the web or very recent changes

Page 57: Overview of European Data from Official Statistics

Transnational access to Public Use Files

In some countries the number of PUF is (very) limited.

Page 58: Overview of European Data from Official Statistics

Transnational access to Scientific Use Files

Page 59: Overview of European Data from Official Statistics

Data archives / dissemination

Page 60: Overview of European Data from Official Statistics

Transnational access to confidential data

Page 61: Overview of European Data from Official Statistics

Countries Years CUF PUF SUF On-site access

Remote Execution

Remote Access

Austria 2003-2007   X X X  

Czech 2005-2010          

Estonia 2004-2010 X X X   X

Finland 1967-2011   X X    

France 2004-2009   X     X 

Germany 2005-2008     X X  

Ireland          

Italy 2005-2010   X X    

Latvia 2005-2011   X     X

Lithuania 2005-2010 X X      

Poland 2005-2010          

Portugal 2004-2009          

Slovakia 2006-2011   X      

Slovenia     X    

Spain 2004-2011   X X    

Switzerland 2007-2009   X      

Statistics on Income and Living Conditions (SILC)

Page 62: Overview of European Data from Official Statistics

Structure of Earning Survey, Eurostat access

Made with Philcarto * 13/07/2012 17:20:15 * http://philcarto.free.fr

SES - EurostatEurostat CDRom N= 2Eurostat Safe Centre N= 3Eurostat Safe Centre + Eurostat CDRom N= 15No Eurostat Access N= 6No information

Heights of the rectangles of the bar chartare proportional to the count of spatial unitsfor each modality of the variable: 'SES - Eurostat'maximum = 15 for the modality 'Eurostat Safe Centre + Eurostat CDRom'

Page 63: Overview of European Data from Official Statistics

Structure of Earning Survey, National availability

Made with Philcarto * 13/07/2012 17:21:24 * http://philcarto.free.fr

National direct accessNational access N= 9No national access N= 4No information

Heights of the rectangles of the bar chartare proportional to the count of spatial unitsfor each modality of the variable: 'National direct access'maximum = 9 for the modality 'National access'

Page 64: Overview of European Data from Official Statistics

Countries Years CUF PUF SUF On-site access

Remote Execution

Remote Access

Finland 1995-2009     X X X  

France 1966-2006     X     X 

Germany 1990-2006 X X X X X  

Italy 2001-2009     X X    

Poland 1996-2010            

Slovenia       X    

Switzerland 1994-2008     X      

UK 1997-2011     X X    

Structure of Earnings Survey (SES)

Page 65: Overview of European Data from Official Statistics

• “Combining datasets from different countries to perform micro-based cross-country statistical analysis is not currently possible (would require the different datasets to be stored in a single location and accessed within a common user interface).”

• “ Cross-country analysis can only be performed using cell aggregated values from the original datasets (e.g. by gender, year of birth, year of observation, etc.). The level of aggregation depends on each country's disclosure rules”

• International comparison projects have to be carefully planned

• Currently, the researcher team should include researchers from each country covered by the study

Page 66: Overview of European Data from Official Statistics

VII. Finding and accessing national microdata in Europe: DwB support

Support by DwB and future ESC

Page 67: Overview of European Data from Official Statistics

DwB Future perspective and current activities• Future perspective

A European Service Centre for Official Statistics ESC-OS as a single point of access linked to the CESSDA Portal

That could offer a range of services: metadata, training, support for accreditation…

A pilot for a European Remote Access Network for access to confidential OS A standard for a European research accreditation

• Current activities Support transnational access to highly detailed microdata

DwB regular calls) CIMES Centralising and Integrating Metadata from European Statistics

Page 68: Overview of European Data from Official Statistics

CIMES Centralising and Integrating Metadata from European Statistics

Page 69: Overview of European Data from Official Statistics

Thanks for Listening

Contact: [email protected]

Website: http://www.dwbproject.org/