online survey design guide guidelines

8/10/2019 Online Survey Design Guide Guidelines

1/12

http://lap.umd.edu/survey_design/index.html

2006-03-06

This site provides design guidelines and principles for e!-!ased survey instruments!ased on a "uman-#omputer $nteraction approach. $t is hoped to give the e! surveypractitioners% especially the !eginners% practical design guidelines and an opportunity tovie& e! surveys as a multidisciplinary field.

'y e!-!ased surveys% &e mean surveys that are displayed on the orld ide e!() via a standard format.

The *+uestionnaire design* in this site is only for e!-!ased surveys and explicitly deals&ith design features of the e!site% such as colors% sound effects% video clips% screenlayouts% navigational issues% etc.% due to the special nature of e!-!ased surveys. Thus%the traditional components of +uestionnaire design% such as &ording and ordering% are out

of the scope of this pro,ect.

or those &ho are familiar &ith the survey error frame&or% it should !e noted that this siteis dedicated to reducing the easurement and onresponse error arising from surveyinstruments. #overage and 1ampling errors and easurement and onresponse errorarising from other sources are not discussed.

or any comments% suggestions or +uestions% please contact &e!masterlap.umd.edu.
http://lap.umd.edu/survey_design/index.htmlmailto:[email protected]:[email protected]://lap.umd.edu/survey_design/index.html


2/12

This section provides tips and guidelines resulting from several research efforts% as ameans to promote the !est practices regarding the design and implementation of &e!-!ased surveys.

The presented guidelines are integrated from orman (200)% #ouper et al. (200)%Tourangeau et al.(2000)% 4illman(555)% 4illman et al. (55)and orman(550).

1 Navigation

.7utomation

2 Usability / Accessibility

2. 8sa!ility2.27ccessi!ility2.3 #onsistency

2.9 8se of color2. 8se of ultimedia eatures

3 Web-based questionnaires

3. e!-!ased +uestionnaire design3.2 ;uestion formatting

3.2. 8se of color3.2.2 8se of ultimedia eatures3.2.3 #hoice of


3/12

1 Navigation

4efinition: avigation

7llo& respondents to easily navigate the survey.

1treamline for&ard movement through the +uestionnaire &hile allo&ing!actracing and changing of ans&ers.

#onstruct &e!-!ased +uestionnaires so that they scroll from +uestion to +uestionunless order effects are a ma,or concern% large num!ers of +uestions must !esipped% and/or a mixed-mode survey is !eing done for &hich telephone intervie&and &e! results &ill !e com!ined.

hen the num!er of ans&er choices exceeds the num!er that can !e displayed onthe screen% consider dou!le-clicing &ith appropriate navigational instructions !eingadded.

hen respondents complete the survey in a for&ard% linear manner% either the&hole form or the single item implementations can !e used as long as navigational

functions are clear and easy to use. hen correction and editing tass re+uire the respondent to find items% the &hole

form implementation can !e used to aid in the searching for items.

>aged surveys that are not congruent &ith sections are to !e avoided.

1ectioned surveys that re+uire scrolling should clearly indicate that additional itemsmust !e accessed !y scrolling to them.

$ndexes to sections and pages are of marginal !enefit and may sometimes lead toconfusion.

1.1 Automation 4efinition:7utomation

'ranching: 7utomate conditional !ranching &hen possi!le% !ut allo& therespondent to override !ranching if there is a need or desire to do so on thepart of the respondent.

7utomate sip patterns: to eliminate errors and to simplify the process oftaing the survey from the respondent@s point of vie&% mae the program%rather than the respondent% manage sip patterns.

7utomatically validate input% if possi!le: input validation improves data +uality

and saves time in data preparation. "o&ever% such validation should !e userfriendly and simply identify the mistae of the user.

Tae advantage of the media@s a!ility to trac respondent !ehavior: a &e!-!ased survey can !e used to collect more than ,ust respondent@s ans&ers.$nformation on ho& much time a respondent spends on each +uestion or onthe &hole survey% the num!er of visits a respondent maes to a &e! site inorder to complete a survey% the se+uence in &hich a respondent completesthe survey% and other such !ehaviors could also !e collected.

Tae into account the costs of automation: incorporation of logic checingand automatic +uestion sipping may re+uire more-extensive soft&are

programs and programming.
http://lap.umd.edu/survey_design/defs.html#Navigationhttp://lap.umd.edu/survey_design/defs.html#Automationhttp://lap.umd.edu/survey_design/defs.html#Navigationhttp://lap.umd.edu/survey_design/defs.html#Automation


4/12

2 Usability / Accessibility

2.1 Usability

4efinition: 8sa!ility

'e a&are that survey respondents !ring various !acgrounds% experienceand motivations to the survey.

7ssume that respondents are all novice to your +uestionnaire !ut they havesome or minimal experience &ith &e! page !ro&sing.

The purpose of the design strategy is training respondents not over time !utin the same survey session.

Aet respondents &ith minimal computer literacy to overcome their limitedno&ledge and perhaps fear of the computer in order to respond.

ae error/&arning messages as specific as possi!le: ideally% an errormessage should !e placed directly a!ove or !elo& the unans&ered item% and

it should !e specific a!out &here the error occurred and% if possi!le% thenature of the pro!lem.

>rovide some indication of survey progress: a graphical progress indicatorcan !e especially useful as a means to inform respondents ho& much of thesurvey has !een done. $t is desira!le to give respondents an approximatemeasure of their progress at least a fe& times during the survey.

7llo& respondents to interrupt and then reenter the survey: respondents &hocannot complete a survey in one sitting should !e offered the option ofstopping at some point and resuming later.

>rovide feed!ac: 8se graphical sym!ols or &ords that convey a sense of

&here the respondent is in the completion progress% !ut avoid ones thatre+uire advanced programming.

Brror messages: >hrase *error* messages in a positive% helpful manner thatsuggests ho& to complete a +uestion.

7dditional feed!ac: $f there is a routine flo& or !urst of automatic entry ofdata% notify the user at the end of that flo& of any errors.

Calidating data: hen the respondent is allo&ed to ans&er all +uestions%implement logic and consistency checs on conditional !ranches.

hen context matters% provide form-!ased vie&s of sections to help to clarifythe meaning of items and the interrelationships among items.

$t must !e remem!ered that although good design seems intuitive% it re+uiresempirical verification !efore final implementation.

2.2 Accessibility

4efinition:7cessi!ility

'e consistent across a variety of hard&are and soft&are systems.

7void differences in the visual appearance of +uestions that result fromdifferent screen configurations% operating systems% !ro&sers% partial screen

displays% and &rap-around text. $f necessary% instruct respondents to maximiDe the screen.
http://lap.umd.edu/survey_design/defs.html#Usabilityhttp://lap.umd.edu/survey_design/defs.html#Accessibilityhttp://lap.umd.edu/survey_design/defs.html#Usabilityhttp://lap.umd.edu/survey_design/defs.html#Accessibility


5/12

2.3 Consistency

4efinition: #onsistency

'e consistent: sho& information in the same places.

$nconsistency in response format may lead to user error. EBxampleF

ollo& the same format% alignment and color scheme. 7ccommodate the respondents@ anticipation.

4esign an interface consistent &ith users@ understanding of the tas.

8sing a system encourages formation of !ehavior patterns.

hen appropriate% eep response options consistent.

hen response options change% dra& attention to the change in some &ay%for example% change the layout of the responses or re+uire navigation to ane& page that optionally instructs the user that the response options havechanged.

Bducate the respondent to !e familiar &ith the +uestionnaire format as+uicly as possi!le.

2.4 Use o color


6/12

and pictures.

2.$ "a%ing Corrections

$f possi!le% implement edits immediately after the entry of a field or set offields to maintain the context of the item.

$f an error is detected on a later page% provide a +uic path !ac to correct

the error% or provide a pop-up field to re-enter the information.

$f an error is the com!ined result of several fields% guide the usersuccessively to those fields or% if possi!le% provide a pop-up screen sho&ingall of the relevant information and providing fields to correct the information.

hen correction and editing tass re+uire the respondent to find items% the&hole form implementation can !e used to aid in searching for items.

3 Web-based questionnaires

3.1 Web-based questionnaire design

e!-!ased survey +uestionnaire design is =T &e! design. e!-!ased survey +uestionnaire design does not solve all pro!lems related

to the &e! surveysH e!-!ased +uestionnaire design can decrease orincrease parts of measurement% nonresponse and coverage error.

e!-!ased surveys assume that respondents can read. 1uch an assumptionmay exclude populations &here literacy is a pro!lem: according to theational 7dult Iiteracy 1urvey% nearly 2J of 81 adult population hasextremely limited literacy in Bnglish. 7!out 20J of that category have visualdifficulties that interfere &ith their a!ility to read printed materialH almost 2Jare immigrants.

e!-!ased surveys assume that respondents have access to a computer%&hich may exclude economically disadvantaged populations.

e!-!ased surveys re+uire that responses !e entered electronically% &hichmay exclude physically disa!led populations.

7lthough fe& su!stantive differences !et&een computer-!ased and paper-!ased surveys have !een found% those differences that have !een foundoften reflect changes in the design of the instrument. ost of these changesinvolve apparently innocuous modification of the layoutH still% these changesmay mae it harder or easier to complete the +uestions% and they may

convey different expectations a!out the inds of information re+uired.3.2 &uestion ormatting

3.2.1 Use o color

Tae advantage of graphic design theory: 7 proper use of color couldprovide visual cues that may simplify the &e!-!ased survey process.

3.2.2 Use o "ultimedia #eatures

Tae advantage of the media@s presentation capa!ilities: e!-!asedsurveys could include hypertext lins to help and additional resourcesas part of the survey instrument &ithout increasing its apparent length.

'e a&are of ho& respondents may interpret +uestions in light ofaccompanying graphics: hen graphics or pictures are used% !ear in


7/12

mind that respondents tend to interpret +uestions in the context ofthose graphics and pictures. EBxampleF

7l&ays as &hether the multimedia material is an enhancement thatsolves an information pro!lem or &hether it is a distraction (*eyecandy*).

3.2.3 C'oice o (es)onse #ormat 8se matrix +uestions sparingly: atrix +uestions place an extra

!urden on the respondent !ecause they re+uire a lot of &or to !edone &ithin a single screen. $n addition% it is impossi!le to predictexactly ho& a matrix +uestion &ill loo on a respondent@s &e!!ro&ser. Therefore% they should !e used &ith caution.

Bxample: 1imple atrix Bxample: #omplex atrix

3.2.3.1 (adio button

The siDe does not change even &hen font siDe changes.


8/12

7void multiple selections on drop-do&n !oxes: use chec!oxes.

4epending on the place on the screen% use drop-up !oxes.

3.2.3.4 e*t in)ut

The 1iDe of the !ox should meet the amount of the re+uired

information.

>rovide sufficient space for text input.

>rovide concise and clear input guidance. (eg. /44/KKKKfor the !irth date)

7void scrolling around the input !ox. $f necessary% eep itvertical.

3.2.4 (es)onse Categories

>reprogram to chec the error in input (eg. the input do not sum to

00JH chec some response options and *none of a!ove* option) andprovide feed!ac if erroneous input is caught.

$mplement category randomiDation &hen needed.

>rovide an open-ended field &ith the *others* option.

hen order matters (-point scale)% pay attention to the responsela!els and spacing.


9/12

8se graphics sparingly: $n a &e!-!ased survey% graphics cansignificantly slo& the do&nloading of a &e! page% especially if usersare liely to connect to the $nternet using a modem as opposed to ahigh-speed connection.

'e a&are of ho& respondents may interpret +uestions in light ofaccompanying graphics: hen graphics or pictures are used% !ear inmind that respondents tend to interpret +uestions in the context ofthose graphics and pictures.

Bxample: Araphics

7l&ays as &hether the multimedia material is an enhancement thatsolves an information pro!lem or &hether it is a distraction (*eyecandy*).

3.3.3 verall structure

>lace logo on the top left and menus on the left vertically.

Ieft alignment of the +uestions and ans&ers.

>lace the response format on the left of each response categories.

Certical alignment has more &hite space thus it is visually appealingand results in less pro!lems &ith alignment and formatting

'e cautious a!out the sense of scale (eg. column space for Iiertscale) in horiDontal alignment.

Geep the ans&er la!els visi!le for the grid design.

>redict the visual flo& and align the +uestions% ans&ers andproceeding !uttons accordingly.

4ecrease the movement time !y decreasing the distance !et&een theselected response chec !ox (or radio !utton% etc) and the@ext/1u!mit@ !utton.

>resent +uestions in hierarchically ordered structure.

>rovide cognitive comfortsH respondents do not need to go !ac andforth in their memory to retrieve.

>reprogram the sip pattern (!ranching).

4ecrease item nonresponse error

4ecisions a!out &hether to display +uestions on the same pagetogether or on separate screens can impact the data collected.

7 group of +uestions considered together may convey informationa!out the purpose of the +uestions and may elicit different responsesthan if the +uestions &ere presented one at a time across multiplepages.

4esigners can help guide the user@s mental model !y providingappropriate +uestion context. 1uch context may !e conveyed in may&ays% including +uestion &ording% response &ording% introductoryexplanation% or as a!ove% may !e implied !y surrounding +uestions.

#onsider several design options: all +uestions in one page% related+uestions grouped on the same page% or one +uestion per page.
http://lap.umd.edu/survey_design/ex_s_multi.htmlhttp://lap.umd.edu/survey_design/ex_s_multi.htmlhttp://lap.umd.edu/survey_design/ex_s_multi.html


10/12

The grouping of response options may impact the responsesrecorded.

$f it is significant that responses or +uestions fall into distinct groups%call attention to the fact in some &ay% for example grouping or la!eling.

Iist a fe& +uestions per screen: Bxcessive scrolling can !ecome a

!urden to respondents and lengthy &e! pages can give theimpression that the survey is too long to complete. There is someevidence that using a single screen or a fe& screens for short surveysminimiDes respondents a!andonment% &hereas using a single screenand forcing the respondent to scroll do&n in long surveys increasesa!andonment.

Bliminate unnecessary +uestions: 7void +uestions that have ans&ersthat the computer can determine% such as the date the +uestionnaire isfilled out.

'egin the &e! +uestionnaire &ith a +uestion that is fully visi!le on the

first screen of the +uestionnaire% and &ill !e easily comprehended andans&ered !y all respondents.

>resent each +uestion in a consistent format% including +uestionnum!ers% instructions% space !et&een lines% and space !et&een+uestion and ans&ers.

Iimit the length to decrease the lielihood of a long line of prose !eingallo&ed to extend across the screen of the respondent@s !ro&ser.

4o not re+uire respondents to provide an ans&er to each +uestion!efore !eing allo&ed to ans&er any su!se+uent ones. $t is suggested

to include a *prefer not to ans&er* and/or *donLt no&* category forevery item than to force respondents to ans&er every +uestion.

=rganiDe and partition an on-line survey to correspond &ith theorganiDation of semantic no&ledge in memory. Then ho& should thesurvey !e partitioned on-line into pagesM 1hould it !e $tem-!ased%orm-!ased% or some&here in !et&eenM

>aged surveys that are not congruent &ith sections are to !e avoided.

1ectioned surveys that re+uire scrolling should clearly indicate thatadditional items must !e accessed !y scrolling to them.

$ndexes to sections and pages are of marginal !enefit and maysometimes lead to confusion.

"ide inappropriate and irrelevant +uestions to shorten the apparentlength of the +uestionnaire and mae such +uestions availa!le only ifthe respondent specifically needs or &ishes to vie& them.

3.3.4 nstructions

>rovide instructions. 8nlie the traditional face-to-face or telephonesurveys% &e!-!ased surveys are not administered !y an intervie&er&ho motivates% clarifies and pro!es the respondents. EBxampleF

Bm!edded instructions distract respondents from their main tas. >rovide a help !utton to access the instructions.
http://lap.umd.edu/survey_design/ex_p_instructions.htmlhttp://lap.umd.edu/survey_design/ex_p_instructions.html


11/12

$nstructions on the pop-up &indo&s may distur! respondents

Bna!le respondents to report pro!lems: 7lmost certainly% somerespondents &ill experience some type of pro!lem &ith the survey.Thus% a *help des* should !e esta!lished that respondents cancontact easily !y e-mail and/or toll-free phone num!er.

>rovide specific instructions on ho& to tae each necessary computeraction for responding to the +uestionnaire. 7ns&ering techni+ues may!e o!vious for experienced users% !ut need to !e explained to lessexperienced respondents.

>rovide computer operation instructions as part of each +uestion&here the action is to !e taen% not in a separate section prior to the!eginning of the +uestionnaire.

rovide a >$ num!er for limiting access only to authoriDed users inorder to:

>revent duplicates and foreign elements (coverage error).

1ecure #onfidentiality.

Iet the researcher tae control over the access.

7l&ays pass&ord protect &e!-!ased surveys: 8ser pass&ords arenecessary to restrict access and uni+uely identify &e!-!ased surveyrespondents. =f course% restricting access for a convenience sample

does not mae sense% so pass&ords &ould not !e issued in thatparticular case.

Bnsure that respondents@ privacy and their perception of privacy areprotected: 'ecause there is the ris that transmissions sent over the$nternet may !e o!served !y unauthoriDed users% all survey datashould !e encrypted. 7 message explaining such procedures in aclear &ay% and displayed ,ust !efore respondents leave a secure areacan alleviate any privacy or security concerns they may have.

#arefully handle respondents &ho fail a screening test: 4epending onthe nature of a survey and the respondent population% access to a

&e!-!ased +uestionnaire can !e restricted until a respondent haspassed the screening +uestions. T&o possi!le approaches involveexcluding respondents from a survey as soon as they fail a screening
http://lap.umd.edu/survey_design/ex_p_firstpage.htmlhttp://lap.umd.edu/survey_design/ex_p_firstpage.html


12/12

+uestion% or allo&ing all respondents to complete the entire survey andeliminate ineligi!le respondents later on.

3.3. Automation

7utomating a survey instrument provides many important !enefits%including conditional !ranching% error checing and correction%

+uestions search% +uestion sort% calculations% and others. 'e a&are of the !urden transferred from the user to the computer in

the migration of paper-!ased surveys to &e!-!ased surveys to predictimpacts of data collection.

$t may !e an issue that users rely too much on the automation and areless liely to detect certain types of errors.

online survey design guide guidelines

Documents