Assessing the quality impacts of Assessing the quality impacts of posting Census Questionnairesposting Census Questionnaires

Garnett ComptonGarnett Compton

Q2008 Conference, 8-11 July 2008Q2008 Conference, 8-11 July 2008

1. Setting the scene – Post-out and the 2007 Census Test

2. Differences in response

3. Quality of the address register

4. Costs

5. Other quality and operational impacts

6. Conclusions and questions

Overview

Why consider Post-out?

• To reduce serious risks experienced in 2001, in particular the failure to recruit a large number of enumerators.

• To provide savings to invest in improving response.

• Because of the limited success of making contact at delivery.

Household response rates by delivery method and ETC

ETC Hand delivery

Post-out Difference P value

Total 53.4% 50.6% 2.8% <1%

1 66.9% 63.4% 3.6% 1%

2 55.7% 51.2% 4.5% 4%

3 47.8% 44.7% 3.1% 9%

4 36.8% 37.0% -0.2% 54%

5 33.8% 29.3% 4.5% 1%

Success rates at follow-up by delivery method by ETC

ETC Hand delivery

Post-out

1 37.1% 35.4%

2 27.0% 26.9%

3 23.6% 22.2%

4 16.5% 17.6%

5 14.9% 13.6%

Overall 26.0% 25.8%

Address register coverage

• 680 (1.3%) new addresses found during enumeration in hand delivery areas

– Nearly 70% of new addresses were sub-premise addresses – suggest existed at time of AC.

– About 20% found already existing/available latest update.

– About 1/6 found in hand delivery areas during follow-up

Estimated Cost Savings

Estimated cost savings between 100% post-out and 100% hand delivery

Initial return* rate difference

(%’age points)

Estimated savings

5 £28m - £35m

6 £25m – £33m

10 £6m - £21m

15 -£18m - £1m

* At the start of follow-up – 23 May

Quality and operational impacts

• No difference in under/over count between two methods

• No difference in number failing 2 of 4 rule• No large difference in age/sex distributions

between delivery methods• 50% more calls to the contact centre in post-

out areas• CTES:

– No difference in views on “junk” mail– Small difference recognised as “official” mail

Conclusions

• Post-out has an impact on return rates, but not success at follow-up. Post-out requires more follow-up to obtain same overall response rate.

• Differences in return rates are not affected by the hard to count characteristics of an area (i.e. the ETC).

• No significant differences in response quality• A post-out methodology will allow savings to invest in

targeted follow-up and community liaison.• The levels of AR undercoverage will be small with

minimal, but manageable, impact on the overall quality.

• Some operational impacts but manageable through design and development

Thank you

Any Questions?