Research into an alternative sampling frame for the FRS

Antonia Simon, Development Team, DWP

2

Background to this research

• Research to use administrative data as an FRS sample frame undertaken as placement fellowship: funded by Economic Social Research Council (ESRC) and Department for Work and Pensions (DWP)

• DWP is interested in better using its admin data - what potential for use as a sampling frame for the FRS?

• Are there likely to be legal and ethical issues with using admin data for sampling?

• This presentation focuses on the availability and potential of DWP-held admin data sources for sampling

Growing interest in administrative data

• What possible advantages do administrative data have?

- Potential to save cost and time

- Potential to improve accuracy e.g. linking back survey responses to admin data which hold information on income

- Reduce respondent burden- Can provide information on survey non-responders

• Little use of admin. data for sampling frames in UK compared with other European countries

3

4

Research objectives• Assess main features of the current sampling frame for FRS

• Identify suitable DWP-held income-based administrative data for future sampling of the FRS

• Compare current and future extent of population coverage of current FRS sampling frame with the identified administrative sources

• Identify any key ethical and technical considerations

5

Assessing the current FRS sampling frame

• Small-users Postcode Address File (PAF) (approximates private households)

• FRS frame using PAF:

Is a list of all addresses in Great Britain receiving < 50 items of post per day

May include some small businesses

Current FRS sampling

2 stage sample drawn from the Postcode Address File (PAF):

stratified sample of primary sampling units (PSUs) - postcode sectors

systematic random sample of 24 addresses within each selected PSU

6

7

An alternative sampling frame for FRS would need to…

• Provide similar coverage to the current FRS frame

• For practicality be clustered into postcode areas

• Provide full address information to enable a precise household to be drawn

8

2 DWP-held admin sources identified

Address Hierarchy File (AH file) Part of the ‘Work and Pensions Longitudinal Study’ Updated monthly Historical record of addresses DWP customers only

Customer Information System (CIS) Updated nightly Everyone with NINO, i.e. not just DWP customers Includes people with NINO now living abroad

9

Assessing population coverage

• Aimed to assess current and future extent of population coverage of DWP-held administrative data

• Matched PAF frame for FRS to both CIS and AH file on:

- Postcodes

- Addresses

• Postcodes assessed to be ‘valid’ or ‘invalid’ formats using British Standards ‘BS 7666’

10

Postcode match-rate: PAF to CIS & to AH file

11

Address match-rate: PAF to CIS & AH file

12

Why differences in match-rates between postcodes and addresses?

• AH is historical record of claimants’ receipt of state benefits / pension

• Mixed quality of address information on CIS & AH including:

Information for flats being dropped Suffixes on addresses being dropped flat and house numbers becoming confused Addresses have no street / house number

Non-matches: CIS to PAF

• 330,000 postcodes (19% of all postcodes) on CIS failed to match to the PAF

• 3 hypothesised reasons for this non-match:

- Deceased individuals on CIS admin data

- Deleted postcodes - no longer in use by Royal Mail e.g. demolished buildings

- Foreign addresses

13

14

Analysing the non-matches: CIS to PAF

• Foreign addresses – 0.5% non-match cases

• Rest, ‘valid’ RM formats but non-match due to:

- Deleted postcodes - no longer in use by RM

- Dormant postcodes – postcodes formerly in circulation but not currently used by RM

- Large-user’ postcodes/addresses – excluded from small-users PAF as they receive > 50 items post daily

15

Key ethical & technical considerations of using administrative data as sampling frame

Ethical List of addresses would be same as currently

available in public arena (e.g. PAF, electoral register)

Technical Dealing with addresses with poor data quality – e.g.

‘invalid’ addresses Dealing with multiple entries of the same address Excluding foreign addresses (on CIS)

16

Some conclusions• Overlap of current DWP-held admin data with PAF is high – better than anticipated at outset of this research

• However, further work needed e.g. cleaning address information and an assessment of bias (if house names are always excluded)

• Potential for FRS and other household surveys

• More guided sampling if fuller use is made of admin data

• Improved correction of non-response

17

Further info, please contact:a.simon@ioe.ac.uk or valerie.christian@dwp.gsi.gov.uk

Publication:Simon, A. (April 2011). Using administrative data held by

the Department for Work & Pensions as an alternative sampling frame for the Family Resources Survey.

Economic & Labour Market Review. Vol. 5, 4, pp.15-21.http://www.statistics.gov.uk/articles/elmr/elmr-apr11.pdf

Any questions???