QUEST 2007 1

Standards for Questionnaire Design and Layout in Business Surveys

Birgit Henningsson

QUEST 2007 2

Why?

• Questionnaires potential source of errors

• Reduce respondent burden

• Improve economic statistics

• Reduce the amount of editing

• This means improved questionnaires are necessary

QUEST 2007 3

Tasks

• Guidelines for design of questions• Visual guidelines for questionnaire layout• Enterprise characteristics that affect measurement• Editing staff debriefings• Process data in web questionnaires• Studies of response process• Model for studying data availability• Create cyclic model for systematic development and review• Checklist for evaluation - expert review• Use nine surveys that were chosen by the editing project

QUEST 2007 4

Cyclic Model

• Define the Survey

• Questionnaire Design

• Testing

• Data collection

• Evaluation

• New round

QUEST 2007 5

Questionnaire Design

• Gathering of knowledge– Types of questions– Response alternatives– Languages– Definitions and instructions– Order of questions– General advice and guidelines for writing

questions

QUEST 2007 6

Enterprise characteristics

• Differences to household surveys

• Response process

• The right person

• Statistical units

• Terms and definitions

QUEST 2007 7

Checklist for expert review

• General things• Covering letter and instructions• Questionnaire

• Structure in the questionnaire• Formalities• Visual layout• Tables• Questions• Response categories

QUEST 2007 8

Visual layout

• Large effect on the data process

• Application of existing principles

• Questionnaire flow

• Standardized survey information

• Adaptation to scanning possibilities

• Cover letters

• Visual examples of standard questions and extracts of the nine surveys

QUEST 2007 9

LägenhetstypOBS! Ändra inte på indelningen nedan. ½ rum räknas som ett rum

a) Hur många lägenheter av varje typ finns i huset/husen?

b) Hur fördelas antalet kvadratmeter på respektive lägenhetstyp?

c) Hur stor är summan av alla årshyror/avgifter för respektive lägenhetstyp? **

d) Hur stor är summan av insatser för bostadsrätter/ kooperativa hyresrätter?

e) Hur många lägenheter var outhyrda/ osålda den 31 december 2005?

Ange antal Ange hela kvadratmeter

Svara i kronor Svara i kronor Ange antal

1 rum och kokvrå.011 012 013 014 015

1 rum och kök021 022 023 024 025

2 rum och kök031 032 033 034 035

3 rum och kök041 042 043 044 045

4 rum och kök051 052 053 054 055

5 rum och kök.061 062 063 064 065

6 eller fler rum och kök071 072 073 074 075

Övriga lägenhetstyper (ex. kokvrå med fler än 1 rum)

081 082 083 084 085

Summa samtliga lägenheter091 092 093 094 095

QUEST 2007 10

QUEST 2007 11

Editing Staff Debriefings

• Qualitative method to use experiences from the production process

• Four debriefings made within project

• Summary of results and findings, place method in cyclic model

QUEST 2007 12

Topic Guide

• Start with the aims of the debriefing• Ask the supervisor to describe the editing process • In what different ways does the respondent answer the

questions? (web, paper, a file)• What reactions do you have from the respondent?• Are there any problems with the variables in the survey?• What are the reasons to the problems?• Does the respondent use the instructions?• What kind of business has problems?• Do you need further improvements in your system?

QUEST 2007 13

Results

• You will have a lot of information

• Can point out in the questionnaire where the problems are

• Often know the reason for errors

• You can use the method as a complement to cognitive test

• A complement to quantitative methods – process data

QUEST 2007 14

What's left after the project?

• Tests with actual respondents

• Create finished guidelines, templates

• Use results from this and other projects to finalize a complete cyclic model