Respondent Acceptance of Web and E-mail Data Reporting for an Establishment SurveyRichard Rosen, Louis Harrell, Hong Yu

Division of Current Employment StatisticsBureau of Labor Statistics

Third International Conference on Establishment SurveysMontreal, Quebec, Canada

June 18-21, 2007

Current Employment Statistics (CES)

• Monthly survey of employment, payroll, and hours• A Federal-State cooperative system • A sample of 300,000 business establishments• CES data are published after only 10-14 days of

collection• Provides key economic indicators

Employment by industry, state, and area Average hourly earnings Average weekly hours

Goals/Objectives

Explore different methods of distributing and collecting CES reports

Utilize existing technology Minimize respondent burden

Software

Used off-the-shelf software SurveyTracker from Training Technologies Inc.

http://www.surveytracker.com/ The software package costs around $3800

Phases of e-mail collection

Form design Distribution Respondent’s e-mail box Retrieval Export

Survey Tracker

Design

CES Form Design

Distribution

Respondent’s E-mail Box

Confirmation Screen

Response

Retrieval

Retrieval

Export

The software exports by comma separated value format

Read into SAS program and export into the format of choice

Advantages

Ubiquitousness of e-mail• The prevalence of e-mail in everyday life is such that almost

anybody with a phone line and a minimal cost to sign up for internet service allow individuals to get an e-mail address

Using off-the-shelf software• By utilizing off-the-shelf software to design and distribute the

software, it saves the effort of re-inventing the wheel Low marginal cost

• Sending e-mails costs close to nothing Utilization of pre-existing infrastructure

• Utilization of pre-existing e-mail server as well as web server saves cost and time

Disadvantages

No data edits• Due to the security issues, no data edits will be performed

by the e-mail SPAM blockers

• ISPs with aggressive SPAM blockers may flag the survey as SPAM

Ill-formed data• No easy way to account for respondent mistakes

Reliance on software vendor• Necessary to rely on the software vendor initiative to

address the problems

Current Status

Incorporated as one of the CES collection methods starting February 2007

Comparable to pre-existing collection methods Survey Tracker deemed to be not suitable, looking

into another software package• Inability to handle large database• Compatibility issues

Minimum requirement: Outlook and Internet Explorer Not all respondents who meet the minimum requirement can

respond

Collection Rate Comparison

All TDE Web E-mail

February 73% 73% 66% 74%

March 68% 73% 64% 67%

April 69% 69% *60% 70%

*CES experienced web server problems during April 2007

E-Mail Collection Data Item Response Rate

WW AEPR AEHR GME PW PR HR

90% 76% 70% 74% 80% 60% 59%

CES and Web

CES has been collecting data via Internet since 1996 (first Federal Agency)

BLS Developed an Enterprise level Internet Facility to promote: Security and continuity of operations Uniform standards for Web pages Cross-survey participation

BLS Model

Initial BLS Model Included: Digital Certificates Option to use an account number and password Password requirements are stringent

Min. 8 chars. Including letters, numbers, upper/lower case CES has experienced declining Web response,

looking for something new …

TDE vs Web Response Rates

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

TDE 1stCl

Web 1stCl

Issues with Current Web Site

Transition to Web was complicated Temporary Account/password Permanent account/password or digital certificate

Ongoing Reporting Forgot account/password Non-renewal of certificate/not portable

Contact change/turnover Lose continuity of knowledge on reporting Need to re-issue account/certificate

Web-Lite Features

No temporary account/password No permanent account/password No digital certificate Access gained through CES Report ID already

on the CES form More-”TDE Like”

Web-Lite Design Compromises

No respondent identifying information displayed

Cannot update contact information on-line No historical data shown

Reduces visual continuity of reporting Limits edit capability

Cannot directly link to other BLS Web surveys

Security

Site remains secure: SSL 128 bit encryption Data stored on separate server Firewall protects other BLS resources Use of CAPTCHA

What is CAPTCHA?

Completely Automated Public Turing test to tell Computers and Humans Apart.

A trademarked acronym owned by Carnegie Mellon University.

Used to determine whether a user is human or a robot.

First developed in 1997, by Andrei Broder at Altavista.

Comparison of TDE and Web Response Rates, January 2004-March 2007TDE vs. Web 1st Closing Response Rates

Jan. 2004 - Mar. 2007

40%

50%

60%

70%

80%

90%

100%

Jan04

Feb04

Mar

04

Apr04

May

04

Jun04

Jul0

4

Aug04

Sep04

Oct

04

Nov04

Dec04

Jan05

Feb05

Mar

05

Apr05

May

05

Jun05

Jul0

5

Aug05

Sep05

Oct

05

Nov05

Dec05

Jan06

Feb06

Mar

06

Apr06

May

06

Jun06

Jul0

6

Aug06

Sep06

Oct

06

Nov06

Dec06

Jan07

Feb07

Mar

07

TDE 1stCl

Web 1stCl

Sources of New Units Converted to Web Lite

Web Adds by Active FlagJan06 - Jan07

0

200

400

600

800

1000

1200

1400

1600

1800

2000

2200

2400

200601 200602 200603 200604 200605 200606 200607 200608 200609 200610 200611 200612 200701

Old Web

Old Web to Lite

TDE to Lite

State to Lite

DCC to Lite

Response Rate Comparison for TDE, Old Web, and Web Lite

Web Lite, TDE, Old Web and Total Web 1st Closing Response RatesApr06 - Jan07

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Apr06 May06 Jun06 Jul06 Aug06 Sep06 Oct06 Nov06 Dec06 Jan07

TDE to Lite

DCC to Lite

TDE

Old Web to Lite

Total Web

Old Web

Data Item Response Rates, Selected Items, March 2007

Women Workers

Gross Month Earnings

Total % Received % Received

DCC to Lite 4,611 52% 45%

State to Lite 51 53% 27%

TDE to Lite 2,331 62% 47%

Old Web to Lite 349 45% 28%

Old Web 1,760 44% 25%

Conclusions

If technical issues can be overcome, e-mail appears to be a viable collection method. Response rates are favorable Offers visual interface of web collection

Research will continue on possible candidates for e-mail data collection software.

Web offers on-line editing, and recent months had performance superior to TDE

Web Lite offers improved item response rates