Collection of Paradata in a CAPI System with Wireless TelecommunicationsVesa Kuusela and Kai Vikki

Social Survey Unit

Statistics Finland

6.5.2010 2Q2010, Helsinki 2010

Background Paradata did not exist in paper-and-pencil era Computer Assisted Interviewing (CAI) systems brought along many

possibilities to collect data about the process CATI call schedule systems collect (automatically) data that can be

used for quality control or for process development In CAPI system paradata collection has to be arranged separately

Problem has been the limited telecommunications facilities At Statistics Finland, newly redesigned CAPI information system is

based on 3G telecommunications Fast data transmission has opened new perspective for paradata

collection

6.5.2010 3Q2010, Helsinki 2010

Wireless Wide Area Network (WWAN)

WWAN uses mobile telecommunications network for transfering data Nearly the same as mobile phone network (GSM)

Based on third generation (3G) of telecommunication hardware standards

Universal Mobile Telecommunications System (UMTS) is most common of the 3G technologies (in Europe)

In theory, UMTS supports up to 21 Mbit/s data transfer rates - in practice much less.

General packet radio service (GPRS) is a mobile data service for the 2G mobile communication systems (GSM).

GPRS provides data transfer rates up to 114 kbit/s Pricing is based either on amount of transferred data or on fixed

monthly fee

6.5.2010 4Q2010, Helsinki 2010

Current status of 3G

WWAN is originally designed for the new mobile telephones but its use for wireless connection to the Internet increases rapidly

In Finland, 3G/UMTS networks cover nearly all parts of country and GPRS covers the whole country

Currently fastest available transfer rates are 5 Mbit/s downstream and 2Mbit/s upstream

For computer use, a slot for a SIM card is needed either on mother board, on PCMCI card, or in a connector to UBS port.

6.5.2010 5Q2010, Helsinki 2010

Building blocks of new CAPI System

Interviewers connect using 3G over the InternetFixed monthly fee, independent of amount of transmitted

data Data transmission with Secure FTP (SFTP) Firewall protected SSH-Server on Demilitarized Zone SSH-server has Inbox and Outbox for each interviewer At the main office, a network computer called ”Separator”

polls SSH-server once in 10 minutes and delivers objects to appropriate places

6.5.2010 6Q2010, Helsinki 2010

Schematic display of technical arrangement

6.5.2010 7Q2010, Helsinki 2010

Features of new system

Interviewers have Laptop computer “Net stick” for (high-speed) Internet access through WWAN

While conducting interview, interviewer works off-line All telecommunication (for CAPI) is arranged through the Internet and

SFTP Data transmission once a day

In addition, through VPN interviewers have access to Statistics Finland intranet are clients to Statistics Finland email system

6.5.2010 8Q2010, Helsinki 2010

Paradata from interviews

All Statistics Finland questionnaires include a standard set of data fields for automatic recording of

interviewer codedate and time of interviewend result (interview or non-response)

reason of non-response with standard codingthree start and end times to record the total processing

time of each case At the office, paradata from interviews are processed and

printed every day with a standard reporting scheme For specific needs it is possible to add automatic fields or

questions for interviewers to collect paradata

6.5.2010 9Q2010, Helsinki 2010

Example of a standard report of a survey (LFS)

Labour Force Survey 2010-04-16, printed at 10:49:18 Week 12Field period 29.3.2010 - 15.4.2010

Interviewer All cases Returned Returned Interviews Non-resp. Over coverage Non-resp. Mean time Total time% % Minutes Minutes

0707, Mnnn 25 25 100 23 2 0 8,0 2,74 632147, Rrrrr 10 10 100 8 2 0 20,0 4,63 372668, Nnnnn 20 20 100 19 1 0 5,0 4,79 91…. …. …. … …. …. …. …. …. ….Total 2139 2139 100 1867 268 8 12,29 4,36 8136

Same table can be produced also for all regions or

based on repondents’ characteristics (e.g. gender or age)

6.5.2010 10Q2010, Helsinki 2010

Recording of contact history of cases

In interviewers’ user interface is application for recording of all contact attempts with respondents

Time stamps are automatically attached to each transaction

Agreed appointments are added to the calendar application

6.5.2010 11Q2010, Helsinki 2010

Example of recorded contact history

Each transaction is on one row showing

Date and time Contact media Contacted

person Time of agreed

appointment Remarks

Contact history is returned automatically with other case data

6.5.2010 12Q2010, Helsinki 2010

Possibilities provided by new technology

Technically, the amount of collected paradata is no problem anymore

Very large paradata files can be sent, e.g. audit trail files, audio files and video files

Many features of CATI case management can be installed in the new CAPI system

Some new possibilities for CAPI system design also emerge

New approach to paradata collection The difficult question is what to do with the paradata

6.5.2010 13Q2010, Helsinki 2010

Questions in paradata collection

For what purpose is paradata collected? Quality control and quality assurance? To increase cost-efficiency of data collection?

How is paradata collection arranged? Automatically

Requires some standardisation in data collection Interviewers enter appropriate data

Causes extra cost if interviewers are paid by hour By supervisors from other process or administrative information

How is the reporting arranged? Automatic reporting Ad hoc –type reporting

What about feedback for process control?

6.5.2010 14Q2010, Helsinki 2010

Future scenarios Technically, CAPI case management with WWAN could be

designed similarly as in a CATI systemStoring data real-time in a central data base is possible

already (on-line interviewing)Case assignment to interviewers is more problematic Connection between interviewer and office can be

arranged using e.g. VPN or Citrix For example, it is possible to build a system where

supervisors see in real time when an interviewer is conducting an interview and for what survey

6.5.2010 15Q2010, Helsinki 2010

The end

Thank you for

your attention