the understanding society covid-19 web survey · 5 85.6 84.2 84.0 67.3 35.8 27.1 616.8 547.3 notes:...

Introduction Development, Design, Implementation Population Inferences Some Wave 1 Results Conclusion

The Understanding Society COVID-19 Web SurveyFFSUC, 8th July 2020

Michaela Benzeval Jon Burton Thomas F. Crossley PaulFisher* Annette Jäckle Hamish Low Brendan Read

*Institute for Social and Economic Research, University of [email protected]

1 / 37


Outline

1. Introduction

2. Development, Design, Implementation

3. Population Inferences

4. Some Wave 1 Results

5. Conclusion

2 / 37


Introduction

� Understanding Society is the UK Household LongitudinalStudy (UKHLS)

� Large panel survey, based on probability samples, annualinterviews. Mixed mode.

� From April 2020, participants asked to participate in short,frequent web interviews: The Understanding SocietyCOVID-19 Web Survey.

� This survey covers the changing impact of the pandemic onthe welfare of UK individuals and households.

� Core content captures change as pandemic and policyresponses evolve.

� Variable content responds to changing situation and researcherinput.

3 / 37


Advantages of Building on Understanding Society

� Content:� Past waves of the main study provide important context for

responses.� Future waves will allow study of long-run impacts.

� Methodological:� (Initial) probability samples.� Past waves provide detailed information on non-respondents.

4 / 37


Outline

1. Introduction




5. Conclusion

5 / 37


Understanding Society (UKHLS)

� Follows the same people across time within the UK.� Initial probability samples, carefully modeling of non-response

and attrition.� Began 2009, but incorporates BHPS sample (1991).� Wave 11 (2019-20): approx. 22,400 households.� 40 min annual interview of all HH members aged 16+, +12

min HH interview.� Self-completion questionnaire for 10-15 year olds.� Mixed mode since 2016: F2F and web, in about 70%

"web-first" in 2019.� Employment, Education, Health, Housing, Income, Social and

family networks, civic engagement.

6 / 37


Normal Wave Time Line

� Autumn t − 2: agree priorities for content (re) development.� January t − 1 begin content (re) development work.� July t − 1: questionnaire consultation� Autumn t − 1 : scripting� January t: fieldwork begins� 8 quarterly batches, each in the field for 5.5 months (total

28.5 months of fieldwork)� May t + 2: final data received.� Summer-Autumn t + 2: cleaning, checking, weights, derived

variables.� November t + 2: Data release.� More than 4 years from initial planning

7 / 37


Understanding Society COVID-19 Web Survey

� Builds on an existing Event-Triggered Data Collection projectwith Ipsos.

� Asks all adult participants to answer a short (20 minute) websurveys on experiences during pandemic.

� First survey (wave 1) April 24-30.� Then May, June, July, September....� 2 Telephone follow-up surveys.� Planned mail surveys for youth (hoping for SDQ in July).

8 / 37


COVID-19 Web Survey: Challenges

� Speed: funding, procurement, ethics, design, testing andfielding < 6 weeks.

� Choosing content for short survey.� Rapid processing and release.� (Differential) non response rates (about 50 % retention on

Wave 9).

9 / 37


Wave 1 (April)

� Fieldwork April 24-30.� Content: health, economic impacts, home schooling.� Collects pre-pandemic (February) economics baselines� 42,330 sample members were sent the pre-notification letter

inviting them to the study.� 32,596 had completed the wave 9 annual interview.

� The response rate for the full sample was 38.7 %, (41.2 %including partial respondents.)

� Among wave 9 respondents, the response rate was 46.0%,(48.6% including partials.)

10 / 37


Data Release

� Wave 1 data were released (SN 8644) by the UK Data Serviceon 29th May.

� As of 6th July, 405 downloads.� The release included:

� all fieldwork documents (letters, questionnaire, etc).� a user guide.� beta weights, derived variables.� a child file.

11 / 37


Wave 2 (May)

� Into the field on 27th May.� Repeat/enhanced modules on economics and health.� New modules on housing and family.� Core modules were translated into Welsh.� 14,811 responses.� Data released 3rd July.

12 / 37


Wave 3 (June)

� New content� community engagement, home working, training, job search

� In the field.� Data release by end of July.

13 / 37


Open Question Call

� We invited researchers to submit content proposals via a webform.

� 80 submissions for the May questionnaire, and a further 36 theJune call.

� Have been able to include half of these suggestions (in full orin part); will include more in July.

14 / 37


Telephone Survey

� For adults in households where no-one is a regular internetuser.

� Fielded end of May with version of the April web questionnaire.� Capturing older respondents.� Interview time almost double.

15 / 37


July and Beyond

� Web survey every second month.� Hoping for short youth questionnaire by mail in July (SDQ).� Longer youth questionnaire by mail in September.� Second telephone followup survey in autumn.

16 / 37


Outline

1. Introduction




5. Conclusion

17 / 37


Probability Samples

� Probability Sample: is every unit in the target population has aknowable, nonzero probability of selection

� Contrast: volunteer, convenience samples

� Non-response and attrition mean Understanding Society (andother real surveys) are not pure probability samples.

� Very careful modeling of non-response and attrition.� Continuous evaluation of ability to estimate population

quantities ("representativeness")

18 / 37


Advantages of Starting from Probability Samples

� All "types" present initially.

� Detailed information available on non-respondents.� Especially attritors.� Can estimate f (Reponse|X ) directly,� Rather than infer it from f (X |R = 1) and information on f (X )

from external sources.

19 / 37


Non-response to the COVID-19 Web survey

� Model as attrition from Wave 9 of main survey.� Inverse probability (IP) weights.

� 49% retention.� Very good for voluntary web survey.� Comparable to response rate to many large (official) surveys.

� But much below typical Understanding Society wave-on-waveretention rate.

20 / 37


Testing for Attrition Bias

Wave 9 Covid Test StatisticWeighted Unweighted Calibration weight Full IP weight Calibration Full

In neither weighting model:

Poverty 0.15 0.11 0.12 0.14 0.03*** 0.01(0.000) (0.320)

Receives core benefit 0.05 0.03 0.03 0.05 0.02*** -0.00(0.000) (0.755)

Behind with housing 0.09 0.06 0.06 0.09 0.03*** 0.00(0.000) (0.730)

Smoker 0.15 0.09 0.10 0.13 0.05*** 0.02**(0.000) (0.002)

Long-standing illness 0.38 0.34 0.34 0.36 0.04 0.02(0.000) (0.029)

21 / 37


Other Methodological Issues

� Past web response strong predictor of response (ME = 0.22).� Invitation by email strong predictor of response (ME = 0.29).

� In weighting models but also useful for design of future datacollection.

� How to best use data from telephone follow up?

22 / 37


Outline

1. Introduction




5. Conclusion

23 / 37


Some Wave 1 Results

� Weighted analysis

� Sample� Wave 9 respondents� Age 20-65� Full response to COVID-19 Wave 1� Complete hours and employment information at Wave 1

24 / 37


Employment Changes for Men and Women

.2.4

.6.8

1pr

obab

ility

empl

oyed

male: G

CSE or lo

wer

female

: GCSE or

lower

male: a

-leve

l

female

: a-le

vel

male: d

egree

female

: deg

ree

Employed February Employed AprilPositive hours February Positive hours April

Notes: N=10708.

25 / 37


Labour Market Changes

Employed(Feb)

Employed(April)

Positivehours(Feb)

Positivehours(April)

Hours(Feb)

Hours(April)

Earnings(Feb)

Earnings(April)

All 79.0 76.5 78.0 54.0 34.5 22.7 424.3 375.1Gender:Men 83.4 80.4 82.5 57.5 38.3 25.0 494.6 426.3Women 75.1 73.0 74.0 50.9 30.8 20.4 354.3 324.1Ethnicity:Not BAME 79.9 77.9 79.1 54.7 34.8 22.8 425.9 377.7BAME 70.3 62.3 67.1 47.1 31.7 21.8 406.5 344.6Age:Age 20-29 78.3 71.4 76.0 45.4 33.2 19.8 327.4 283.1Age 30-39 85.3 83.3 84.2 58.9 35.4 23.3 442.6 388.6Age 40-49 84.6 84.0 84.0 63.3 35.8 25.5 480.1 435.6Age 50-59 80.2 78.7 79.7 56.9 35.0 23.1 448.8 397.3Age 60-65 58.4 56.4 57.6 37.6 31.1 19.2 364.9 310.9Household type:Single adult, no children 69.3 68.7 68.8 50.1 35.7 24.9 420.2 381.7Single adult, children 70.1 68.5 69.7 44.2 31.5 17.7 387.3 302.6Multiple adult, no children 77.2 73.8 76.1 52.3 34.7 22.9 409.5 363.5Multiple adult, children 84.4 82.6 83.4 58.1 34.4 22.4 445.7 393.1

26 / 37


Labour Market Changes

Employed(Feb)

Employed(April)

Positivehours(Feb)

Positivehours(April)

Hours(Feb)

Hours(April)

Earnings(Feb)

Earnings(April)

All 79.0 76.5 78.0 54.0 34.5 22.7 424.3 375.1Long-run income quintile:1 59.6 54.3 57.8 34.7 30.7 17.3 288.0 224.62 76.9 74.0 76.4 47.0 34.1 19.8 326.6 282.53 84.6 83.2 84.1 55.4 34.9 21.5 368.1 342.94 87.0 85.4 86.3 64.4 35.8 25.6 453.8 405.95 85.6 84.2 84.0 67.3 35.8 27.1 616.8 547.3Notes: The columns refer to both employees and self-employees. Columns 1-4 are populationpercentages, columns 4-8 are weekly means. Columns 1-2 refer to the full sample, columns 3-4 excludesthose with missing work hours, columns 4-5 refer to those employed and reporting non-negative workhours in February, and columns 7-8 refer to those reporting earnings in February.Sample sizes (full sample) are: 10803 (col 1-2), 10803 (col 3-4), 8747 (col 5-6), 8015 (col 7), 8015 (col 8).

27 / 37


Reasons for Hours Change

Employercuts Furloughed

Loss ofself-employment

business Unemployed Health CaringAll reporting a decline in hours 10.0 43.7 12.7 9.3 7.2 6.9Gender:Men 10.3 44.4 15.3 9.9 7.0 5.1Women 9.8 43.1 10.2 8.6 7.5 8.6Ethnicity:Not BAME 10.1 45.2 12.6 7.5 6.9 6.6BAME 9.0 27.8 14.2 29.3 11.3 10.3Age:Age 20-29 9.0 53.1 5.1 19.0 4.2 2.2Age 30-39 9.1 45.8 12.2 7.9 4.8 15.7Age 40-49 9.1 40.6 15.8 4.6 8.3 11.7Age 50-59 13.2 40.3 14.6 6.0 8.3 2.8Age 60-65 7.2 35.7 18.3 9.9 13.4 1.5Education:GCSE or lower 12.9 47.7 11.5 11.4 6.9 4.1A-level 7.2 51.8 9.9 9.5 9.2 4.3Degree 9.9 35.2 15.5 7.6 6.1 10.6Household type:Single adult, no children 10.0 43.8 14.8 7.5 9.6 0.9Single adult, children 10.0 44.3 12.1 8.2 2.5 11.1Multiple adult, no children 9.0 45.9 11.5 11.5 7.9 1.5Multiple adult, children 11.2 41.2 13.7 7.2 6.5 13.8

28 / 37


Reasons for Hours Change

Employercuts Furloughed

Loss ofself-employment

business Unemployed Health CaringAll reporting a decline in hours 10.0 43.7 12.7 9.3 7.2 6.9Long-run income quintile:1 11.9 38.4 14.1 18.8 9.7 5.82 11.4 50.8 12.1 8.4 7.2 6.73 9.4 54.6 9.8 5.1 5.9 6.14 8.3 40.2 10.7 7.4 8.0 7.05 9.5 31.1 17.7 8.7 6.0 9.2Worker type:Fixed hours 11.3 54.5 0.0 8.4 6.0 5.4Flexible hours 9.1 43.0 0.0 8.7 3.1 14.3Employer sets (sure min.) 18.7 55.3 0.0 12.2 6.4 3.0Employer sets (no sure min.) 13.7 57.7 0.0 25.6 5.5 3.1Self-employed 2.1 7.3 58.8 7.0 12.6 11.3Works at home (Feb):Sometimes or always 8.4 25.9 24.8 7.0 5.7 13.7Never 10.7 50.3 8.3 10.1 7.8 4.4Notes: Each cell refers to a percentage of the population experiencing a decline in weekly work hours.Sample reporting a decline in weekly work hours.Respondents can report multiple reasons. N=3993 for the all sample.

29 / 37


Household Earnings

Figure: Household Earnings across the Distribution20

040

060

080

010

00H

ouse

hold

wee

kly

earn

ings

1 2 3 4 5Long-run income quintile

February AprilNotes: N=9341.

30 / 37


Mitigation by Long-Run Income Quintile0

.05

.1.1

5.2

.25

.3.3

5sh

are


Used saving Applied for mortgage holiday

New borrowing New work

(a) Self-Insurance0

.05

.1.1

5.2

.25

.3.3

5sh

are


Received a transfer from family or friends Applied for Universal Credit

Used foodbank

(b) External

31 / 37


Arrears by Long-Run Income Quintile

0.1

.2.3

shar

e


Behind with some or all bills (2017-2018) Behind with some or all bills (April)

Behind with housing payments (2017-2018) Behind with housing payments (April)

Notes: Reported on bills: 10505 individuals. Reported on housing payments: 7870 individuals.

32 / 37


Arrears, Hunger, Spending Cuts

Behind with bills Behind with housing

2017-18 April 2020 2017-18 April 2020 HungerReducedspending

All 6.3 8.2 8.5 8.0 5.1 37.4Gender:Men 5.0 7.0 8.5 7.9 5.1 37.1Women 7.4 9.3 8.6 8.0 5.1 37.8Ethnicity:Not BAME 5.7 7.1 8.4 6.8 5.0 36.6BAME 11.7 19.1 10.0 18.6 5.7 45.4Household type:Single adult, no children 8.6 10.4 16.0 10.1 4.4 27.2Single adult, children 14.0 24.2 18.1 15.7 5.9 38.2Multiple adult, no children 5.2 5.8 7.8 7.4 4.6 36.2Multiple adult, children 6.3 9.3 6.8 7.4 5.8 41.5

33 / 37


Outline

1. Introduction




5. Conclusion

34 / 37


Summary and Concluding Remarks

� Understanding Society COVID-19 Web Survey� Tracking people over this year.

� Linked back to previous 11 years of data (and future!)

� Know characteristics of those who responded and those whodid not.

� Range of content (and taking suggestions).

� Methods:� Carefully modeling of attrition and testing for biases.

� Ongoing....

35 / 37


Summary and Concluding Remarks

� Substantive results:� Losses across the distribution. Concentrated at the bottom,

single-parents, BAME individuals.

� Mitigation: CJRS seems to have had large effect, but uneven.

� Much early reliance on self-insurance, and family transfers.

36 / 37


Acknowledgments

� Fieldwork agencies: Ipsos MORI, Kantar� Funders: ESRC, Heath Foundation

37 / 37

the understanding society covid-19 web survey · 5 85.6 84.2 84.0 67.3 35.8 27.1 616.8 547.3 notes:...

Documents