International Conference on Establishment Surveys

1

Everything you wanted to knowabout Electronic Data Reporting

(EDR) at Statistics Canada

June 2007

International Conference on Establishment Surveys

2

Outline of Presentation

Introduction Background Original approach for EDR Services What happened? Consequences Electronic Collection Framework Summary/Conclusions

International Conference on Establishment Surveys

3

Introduction

Statistics Canada (STC) began using Electronic Data Reporting 10 years ago by mailing questionnaires on diskettes.

Six years ago a Government On-line initiative was launched; STC awarded funding to develop a comprehensive Internet collection platform.

International Conference on Establishment Surveys

4

Introduction

Project originally focused on developing an EDR solution for 50 business and agriculture surveys, since these respondents complete large numbers of surveys on an ongoing basis.

Since then, the 2006 Census provided a customized electronic data reporting option, and achieved a 20% take-up rate on the household side.

Today, Statistics Canada has a new plan to provide electronic collection service to respondents, based on last 5 years of somewhat painful experience.

International Conference on Establishment Surveys

5

Background

STC administers over 400 surveys across four business lines: social, institutional, agricultural and business.

Computer assisted telephone interviewing (CATI) and personal interviewing (CAPI) are the mode of collection for social surveys; paper as well as CATI are used for agriculture, business and institutional surveys.

International Conference on Establishment Surveys

6

Background

Strict confidentiality and privacy policies require a sophisticated security model for EDR, along with a complex technical infrastructure.

The wide diversity in STC’s surveys (complexity, length, frequency, sample size) added to the challenges for developing EDR applications.

International Conference on Establishment Surveys

7

Original Approach for EDR Services

Introduced EDR to small surveys or sub samples of larger business/agriculture surveys to minimize potential reduction in response rates. Original mode (usually paper) kept as an option for respondents.

Selected sub-annual surveys whenever possible.

Created separate EDR applications and integrated later with other collection streams (e.g. CATI and paper).

International Conference on Establishment Surveys

8

Original Approach

Web-based and Excel-based options developed to handle different requirements for different surveys.

Web-based option used HTML, Java Script.

Both options used PKI-Entrust (which required respondents to download a Java Applet on a JVM).

International Conference on Establishment Surveys

9

Original Approach

Created secure staging area physically separate from the Internet and STC’s internal network for maximum security.

Built authentication and encryption services, along with virus and detection capability.

Implemented a “Thick client” model (edits executed on client-side).

International Conference on Establishment Surveys

10

What Happened?

Unable to assess cost/benefit of EDR as a mode of collection, given small EDR sample sizes.

Response rates varied (0% - 60%).

Technical infrastructure not stable due to high level of complexity.

Respondents had technical difficulties accessing site and completing surveys.

International Conference on Establishment Surveys

11

What Happened? Costs to develop and support 50 unique applications

too high.

EDR with a CATI follow-up (i.e. multiple, independent systems) does not provide significant benefits (time or money).

Surveys adopting EDR need to develop their collection approach from a single instrument point of view.

Overall - minimum success achieved with majority of surveys originally selected for EDR.

International Conference on Establishment Surveys

12

Consequences

Undertook a complete review of Statistics Canada’s approach to EDR services.

Statistics Canada moving towards a simpler, more technically integrated collection environment.

Proceeded to adopt an enterprise wide view of collection with a more generic/integrated approach across modes.

Adopted new standards for surveys wanting to offer an EDR option (e.g. minimum sample of 6,000).

Developed an Electronic Collection Framework outlining short to medium term strategy for EDR.

International Conference on Establishment Surveys

13

Electronic Collection Framework

Stabilize current EDR environment

Gather Business Requirements

Conduct Options Analysis

Conduct Pilots

The Way Forward

International Conference on Establishment Surveys

14

Electronic Collection Framework Stabilize Current EDR Environment (Completed)

Documented all outstanding issues

Created lessons learned document

Reduced existing suite of EDR surveys to only those with a good business case

Implemented upgrades to environment based on maximum improvement for minimum cost

International Conference on Establishment Surveys

15

Electronic Collection Framework Gather Business Requirements (Completed)

Conducted an Agency wide analysis of the business requirements for EDR. Involved 40 divisions and all types of surveys

Produced the “STC Electronic Collection Services”, a summary of requirements

Identified the need for 2 core electronic services e-file transfer service (urgent) e-questionnaire service

International Conference on Establishment Surveys

16

Electronic Collection Framework

e-file transfer service a secure service to collect (& share) survey related administrative files bi-directional: serves STC and its partners based on the “Managed File Transfer” Technology

e-questionnaire service a self-administered questionnaire service Enables respondents to securely fill and submit surveys online (internet)

International Conference on Establishment Surveys

17

Electronic Collection Framework

Conduct Options Analysis (Completed)

Assessed buy (using COTS) vs build software for new infrastructure

Identified 10-12 potential COTS products aligned with each of the 2 core services

Undertook detailed analysis of key products

International Conference on Establishment Surveys

18

Electronic Collection Framework Conduct Pilots (in progress)

Obtained free access to 2 COTS products Record STC hands-on experience Gather end-user feedback e-file transfer pilot (complete)

9 Divisions, 5 External Partners Very positive feedback

e-questionnaire pilot (underway) 2 social surveys participating in pilot as well as

business survey

International Conference on Establishment Surveys

19

Electronic Collection Framework

Way Forward

Obtain corporate approval for Electronic Collection Framework Strategy/Business Case

Complete e-questionnaire Pilot

Undertake competitive processes for both e-file transfer and e-form COTS software

Launch Agency wide e-file transfer service spring 2008

Offer limited e-questionnaire service summer 2008

International Conference on Establishment Surveys

20

Summary/Conclusions Difficult start up for EDR at Statistics Canada New approach of integration and simplification of

collection processes helping to redefine the role of EDR in future

Consultation exercise established priorities for e-services.

Re-engineering of current infrastructure under way (subject to funding approval)

Increasing interest by Social Surveys Future looks promising