Factors affecting response times:

A comparative analysis of smartphone and PC web surveys

Running Header: Smartphone versus PC web response times

Christopher Antoun, University of Maryland

Alexandru Cernat, University of Manchester

1

Abstract

This paper compares the factors affecting response times (RT) to survey questions when they

are answered using two different devices: personal computers (PCs) and smartphones. Several

studies have reported longer RTs when respondents use smartphones than PCs. However, few

have analyzed the specific reasons for the time difference. In this analysis we analyze timing data

from 822 respondents who completed the same web survey twice, once using a smartphone and

once using PC, as part of a randomized crossover experiment in the LISS panel (Longitudinal

Internet Studies for the Social Sciences). We include both page-level and respondent-level

factors that may contribute to the time difference in multilevel models. We find that respondent-

level characteristics account for more of the time difference than page-level characteristics.

Specifically, slower RTs by those with low levels of familiarity with smartphones and

multitasking on smartphones contribute to the time difference.

Keywords: completion times, mobile web surveys; smartphone surveys

Article type: Article

2

Factors affecting response times: A comparative analysis of smartphone and PC Web

surveys

Introduction

As respondents increasingly use smartphones rather than personal computers (PCs) to

complete web surveys, researchers are eager to study the design and implementation of mobile

web surveys (for reviews, see Couper, Antoun, & Mavletova, 2017; Link et al., 2014). But

because this shift to mobile technology is relatively recent, the survey field is missing a full

understanding of its implications – including why respondents tend to take longer to complete

surveys when using smartphones than PCs. This paper intends to address this question by

investigating the factors that affect how long it takes respondents to answer survey questions in

web surveys when using smartphones and when using PCs.

The article begins with a review of the literature. It follows with our data and analysis plan

and concludes with our results and their implications.

Previous Literature

A consistent finding is that web surveys take longer to complete on mobile devices,

particularly smartphones, than on PCs. Most evidence comes from secondary analyses of timing

data from web surveys where respondents choose to use different devices, including PCs, tablets,

and smartphones. For example, in a secondary data analysis of 21 web surveys taken on different

devices, Gummer and Roßmann (2015) report that respondents using smartphones took

significantly longer than those using PCs, even after controlling for demographic differences

between the groups. Couper and Peterson (2017) present ratios of web survey completion times

on mobile devices and PCs across 26 studies that included a mix of experiments with random

3

assignment to device and non-experiments. For 24 of them the ratios were positive (with a

median ratio of 1.4), indicating longer completion times among respondents using mobile

devices. In one of the few studies that looked at question-level RTs by device in a web survey,

Andreadis (2015) report that respondents using smartphones spent about 1.5 seconds longer per

question (on average) than those using PCs. By contract, recent investigations suggest that

respondents using tablets do not necessarily take more time to complete web surveys than those

using PCs (see e.g., Gummer & Roßmann, 2015). Thus, we focus our attention on the

smartphone-PC time difference.

Why do web surveys take longer on smartphones? The current state of knowledge about the

topic is still evolving. Gummer and Roßmann (2015) outline three potential explanations. One is

that mobile respondents take more time because of the extra scrolling required to view and

answer questions on a small device. Another explanation is that survey pages load more slowly

on smartphones because they rely on a cellular Internet connection (e.g., 3G, 4G LTE) that is

slower and sometimes less reliable than a typical Wi-Fi connection. A third explanation is that

respondents respond to questions more slowly because of increased multitasking or distractions

when using smartphones. This includes doing two activities simultaneously (e.g., completing the

survey while riding a bus) and taking interstitial breaks to switch from one task to another (see

Sendelbah, Vehovar, Slavec, & Petrovčič, 2016).

Unfortunately, Gummer and Roßmann were not able to test their proposed mechanisms.

Couper and Peterson (2017) expand upon these explanations and describe two others. The first is

that respondent may take longer to record their answers using touch input. In particular, typing

answers to open-ended questions may take longer on a smaller virtual keypad than a larger

physical one. The other explanation is that some of the respondents using smartphones have low

4

levels of comfort and familiarity with their devices. These novice users might be take more time

than experienced users to navigate through the questionnaire.

Couper and Peterson (2017) tested some of these mechanisms in a secondary data analysis

of page-level time data from three web surveys of college students. They found that survey pages

do indeed load more slowly on mobile devices. They estimated the time to load a new survey

page after clicking the “Next” button by taking the difference between the time spent on a page

according to server-side timestamps (for total time) and client-side timestamp (for time when a

page is loaded in a respondent browser). They report that average transmissions times across the

three surveys were between 0.31 and 0.57 seconds longer per page on PCs than smartphones.

This is consistent with Mavletova and Couper (2016) who also investigated transmission times in

a survey in Russia, finding that it took about two seconds longer to load each survey page (on

average) on mobile devices than PCs. Across the two studies, this extra loading time accounted

for about between 13% and 28% of the over difference in timings between devices. It seems that

other things being equal, the faster the cellular Internet connection, the smaller the time

difference will be.

Couper and Peterson (2017) also found support for the idea that screen size affects RTs

because it leads to increased scrolling. They report that smartphone users scrolled on a higher

percentage of screens (49%) than PC users (4%), and were especially likely to scroll on grid

questions. Mobile users took about 1.5 times longer than PC users for screens that required

scrolling, which in their analysis explained the majority of the difference in timings.

Another possible explanation for the time difference by device is that mobile respondents

take longer to read words because they are displayed on the screen in small fonts. However, no

studies have found evidence for this. In a secondary data analysis of page-level timings from a

5

web survey conducted in Greece, Andreadis (2015) tested for interactions between device and

question length and found no significant results (reported at the 0.01 significance level).

Likewise, Couper and Peterson (2017) report that the difference in timings was comparable for

questions with few words and questions with many words.

Another possible explanation for the difference in timings by device, that as far as we know

has not yet been tested, is that some mobile respondents take more because of physical

limitations that make it difficult to use smartphones. For example, those with poor eyesight may

have greater difficultly reading questions on a small screen. Those who have problems using

their hands (i.e. low dexterity) may find it harder to select a response and type on smartphones.

It should be noted that these prior investigations made an implicit assumption that group

differences between mobile users and PC users in completion times reflected device differences

rather than unaccounted for differences in the composition of the groups. By contrast, the

analysis presented here makes within-respondent comparisons using data from an experiment

where participants were invited to complete the same survey on both a smartphone and a PC.

Further, the study was conducted in a probability-based web panel and the sample members who

did not own smartphones were provided with devices. Thus, the resulting study sample is quite

diverse in terms of respondents’ demographics and level of smartphone experience.

Hypotheses and Research Question

Here, we test five of the hypotheses described above: the smartphone-PC time difference is

explained by (i) increased scrolling on smartphones, (ii) slower typing speed on smartphones,

(iii) increased completion of other tasks (multitasking, distractions) on smartphones, (iv) slower

completion on smartphones by those with low familiarity with their devices, and (v) slower

completion on smartphones by those with physical limitations.

6

Since both page-level and respondent-level factors are thought to affect survey RTs, some of

our hypotheses are related to page-level characteristics (whether scrolling and typing is required

to answer the questions), while others are related to respondent-level characteristics (the

respondent’s level of experience and physical capabilities). It is unclear which level contributes

more to the time overall difference. Here, we investigate this issue using multilevel models that

partition total variance into variance between pages and variance between respondents. Our

specific research question is: Will the time difference be more influenced by page-level or

respondents-level characteristics? If it is the former, then it suggests that the time gap can be

reduced substantially through better questionnaire design. On the other hand, if respondent

characteristics explain more variation, then this suggests there is little that researchers can do

(apart from recruiting different respondents) to reduce the time difference.

In sum, initial work shows that web surveys take longer on smartphones than PCs. Several

explanations for this have been offered. It is important to continue to test these explanations as

well as to test new ones using a carefully designed experiment and diverse study sample.

Methods

Data

The data are from an experiment conducted in the LISS panel (Longitudinal Internet Studies

for the Social sciences), a probability-based online panel consisting of about 7000 individuals

from the Netherlands who were originally recruited from the national population register. They

are invited to complete surveys for 15-30 minutes each month in exchange for payment (see

Scherpenzeel, 2011). The experiment used a crossover design: panelists were invited to complete

the same survey twice, once on a smartphone and once on a PC, with the order randomized and

7

with a month-long break in between. It was conducted in 2013 – from October 7 to October 29

for wave 1 and from December 2 to December 31 for wave 2. This design enables us to do two

types of analyses. The first is to investigate the determinants of RTs in each individual survey

and compare the results. The second is to investigate the determinants of the within-respondent

change in RTs from one survey to the other.

Panelists who did not have their own smartphone were provided one for the wave in which

they were invited to participate in the mobile web survey. While proving devices is not common

in practice, it is sometimes used as a way to include mobile users in smartphone surveys

conducted in online panels (e.g., Fernee & Sonck, 2013).

The mobile version of the questionnaire was adapted for small screens by eliminating the

sponsor’s logo from each page and by using larger fonts and wide, rectangular buttons for

response options.

The questionnaire contained 46 questions on topics ranging from health to politics. They

were displayed on 32 survey pages (consisting of one or more questions). Our analysis focuses

on the 22 pages that contained single-choice (radio button) and text entry questions. The two

pages near the end of the questionnaire that contained special formats (a slider question and

spinner question) were excluded because respondents using smartphones took an especially long

time to answer them (reported in Antoun, Couper, & Conrad, 2017). Eight pages at the end of the

questionnaire that contained non-traditional questions about respondents’ experience taking the

survey and current setting (e.g., location, whether they were distracted or not) were also

excluded. Most pages (19) contained only one question. The three pages that had more than one

question displayed only one type of question (i.e. all text entry questions or all single-choice

questions). In total, 10 pages contained one or more single-choice question and 12 pages

8

contained one or more text entry question. See Figures 1 and 2 for examples of both types of

questions in mobile and PC web. We viewed each page on a standard (4-inch) smartphone to

determine whether its question text was fully visible from the onset or not: 4 pages required

scrolling in mobile web and 18 did not.

[Figure 1 about here]

[Figure 2 about here]

Panelists were first asked in a screener questionnaire whether they were willing to

participate in the experiment. Of the 5,486 panelists who responded to the screener, 2,263

indicated that they were willing to participate in the experiment. A sample of willing panelists

was drawn. Of the 1,390 panelists who were selected and invited to participate, 895 completed

both surveys using the assigned device for an overall response rate of 64% (RR1, AAPOR 2015).

Cases with missing values for at least one of the variables used in our multivariate analysis were

removed, resulting in final sample size of 822. User agent strings were collected (see e.g.,

Callagaro 2010). When more than one device was used for a single survey, we used information

from the last one under the assumption that respondents switched devices early in the

questionnaire. Although the final sample would not be considered representative of the full LISS

panel, it was still quite diverse in terms of age: 22% 16-30 years, 27% 31-45 years, 30% 46-60

years, and 21% 61-75 years; education: 60% no college and 40% college; and gender: 51% male

and 49% female.

9

The dependent variable for our analysis is page-level timings. They were captured using

server-side timestamps and measured in seconds. In order to insure an approximately normal

distribution we have excluded the longest 1% RTs and have logged the dependent variable. The

complete case dataset used in the multilevel models has 18,011 rows.

Several characteristics were used as predictors (Table 1). Two page-level characteristics

were included: (1) question type and (2) whether scrolling was required on a smartphone.

Six respondent-level characteristics were included, all of which were self-reported. Two

were indicators of whether respondents’ attention was fragmented when they completed the

survey, specifically: (1) the number of other activities they completed during the survey and (2)

whether they were distracted. These indicators are time-varying because they could take on

different values for each survey, for example, if someone multitasked while completing the

mobile web survey but not the PC web survey. Two indicators about respondents’ familiarity

with smartphones were included: (1) a three-level indicator of experience using smartphones and

(2) their frequency of smartphone use. Finally, two indicators of physical limitations were

included: (1) visual acuity and (2) whether someone had problems working with their hands.

These last four indicators were fixed across the two surveys.

[Table 1 about here]

Analytic approach

In order to answer our research question we estimate multilevel models (see e.g., Snijders &

Bosker, 2011; Yan & Tourangeau, 2008). RTs, the dependent variable, is measured for each

combination of survey page and type of survey. In order to take into account the hierarchical

10

structure of the data, we estimate random effects at the page and respondent level. The advantage

of this approach, in addition to estimating correct regression coefficients, is that we can

investigate the proportion of variation that is due to page characteristics and respondent

characteristics for RTs. Models are estimated using R3.5 with the package lme4.

We estimated models for RTs when respondents used a smartphone and when they used a

PC separately. Since RTs come from a crossover experiment, the smartphone times include a

combination of values recorded in the first wave and second wave of the experiment. Similarly,

the PC times include a combination of values recorded in the first and second experimental

wave. The fact that the order of the surveys was randomized insures that this does not have a

large impact on our estimates. Still, we included a control variable for the wave in which the

survey was completed in our models.

To test if there are significant differences between PC and smartphone RTs we will

investigate if the confidence intervals of the corresponding estimates from two models overlap or

not. Due to our design we can also investigate the differences at the respondent level. For this,

we estimate a model in which the dependent variable is the within-respondent change in RTs

from one survey to the other for each survey page. This will help bring more insight to the

question regarding how RTs change when the same respondent completes the survey with a

different device, an approach that can’t be used without a within-person experimental design.

Results

Descriptive Results

Respondents took an average of 34.84 seconds per page when using smartphones and an

average of 24.83 seconds per page when using PCs. Thus, respondents took an average of 1.4

11

times (or 40%) longer to complete each survey page when using smartphones than PCs. The size

of the time difference is comparable to several other studies (e.g., Mavletova & Couper, 2015;

McGeeney & Marlar, 2013) and happens to be the same as the median difference from Couper

and Peterson’s (2017) summary of 26 studies.

Multivariate Models

Table 2 shows the results of the multilevel models predicting page-level RTs. For each of

the three dependent variables – RTs in the smartphone survey, RTs in the PC survey, and

differences in RTs between the two surveys – we run two models. An empty one, that allows us

to investigate the variance composition, and a full one, that allows us to see the contribution of

different page- and respondent-level factors.

[Table 2 about here]

We see that for the models we have 18,011 timings, 22 pages, and 822 respondents. The

variance decomposition for the empty models is presented in Figure 3. This indicates the

proportion of variation in our dependent variable, RT’s, that can be accounted for by pages and

by respondents. The figure highlights that for the first two dependent variables most of the

variance is residual (i.e., unexplained), followed by page characteristics and then respondent

characteristics. We observe that respondent characteristics are more important in the model for

the smartphone survey than the model for the PC survey. This is true even though we have the

same respondents in both groups.

12

Our research question focused on the variance decomposition for the within-respondent time

differences. The third bar of Figure 3 shows that most of the variance in the time difference is

unexplained, around 10% of the variation is explained by respondent characteristics, and less

than 2% is explained by page characteristics. Thus, we conclude that smartphone-PC time

difference was more influenced by respondent characteristics than page characteristics.

[Figure 3 around here]

Next we investigate the coefficients in the full models to test our hypotheses. In all of the

models, we included demographic variables (gender, age, education) as well and the

experimental wave in which the survey was completed. Not surprisingly we found significant

wave effect for RTs in both surveys, indicating that respondents were faster when answering

questions for the second time. The effects of education seem to be approximately linear, with

older respondents taking more time in both surveys. Surprisingly, the time difference between

surveys was smaller for middle-age and older adults (31 years and older) than for younger adults

(16-30 years). Education was not associated with RTs in the PC web survey. However, in the

mobile web survey those with a college degree were faster than those without a college degree.

We expected that the extra step of scrolling adds time and helps explain the time difference.

Scrolling did significantly increase mobile web RTs, consistent other research. However,

scrolling did not significantly predict the within-respondent differences in RTs. In other words,

the smartphone-PC time difference was not significantly larger for survey pages that required

respondents using smartphones to scroll.

13

We also expected that typing would be slower on a smartphone than PC and contribute to

the time difference. Having to type a response did take significantly longer than selecting one

from a list in both surveys, but text entry did not significantly predict the smartphone-PC time

difference. This is because the increase in RTs due to typing was approximately the same in both

surveys.

We also expected the completion of other tasks (multitasking, distractions) would contribute

to the time difference. As expected, respondent who divided their attention between other

activities did respond more slowly. Those reported multitasking took more time, regardless of

their device. Similarly, those who reported being distracted were slower, regardless of their

device. It is possible that multitasking or distractions had a larger impact when carried out using

one device or the other. But the activities had approximately the same effect on response timings

in both surveys. Nonetheless, because multitasking was more prevalent in mobile web than PC

web (53% versus 44%), it was still a significant predictor of the time difference. Thus, we

conclude that fragmented attention among mobile respondents does indeed contribute to the time

difference.

We predicted that the time difference would be larger for respondents who had low

familiarity with smartphones, and this was strongly supported. Focusing on descriptive statistics

(not shown), “low familiarity” users – that is, those who were using devices provided to them –

took 1.7 times longer when using smartphones, those classified as “medium” familiarity – that is,

those who use smartphones but never for surveys – took only 1.2 times longer, and those

classified as “high” familiarity – that is those who sometimes use their smartphones to complete

surveys – took approximately the same amount of time when using smartphones. Similarly, those

who report “never” using smartphones took 1.8 times longer when using such devices, whereas

14

those who report using smartphones “every day” took only 1.2 times longer. In the multivariate

models, as expected, having “low” or “medium” familiarity or being an infrequent smartphone

user significantly increased mobile web RTs and had no significant effect on PC web RTs. Most

importantly, both familiarity variables were significant predictors of the time difference. Taken

together this suggests that the more experience respondents have using smartphones, the smaller

the time difference will be.

Finally, we expected that the time difference would be larger for respondents who had

certain physical limitations. This was not supported. Physical characteristics were not significant

predictor of RTS in either of the individual surveys or of the time difference. This is perhaps

because the text and buttons in the optimized mobile survey were sufficiently large for those

with low levels of visual acuity or dexterity to read and select without slowing down.

Discussion

We compared the effects of several factors on RTs when respondents use smartphones and

when they use PCs. We have three main findings. First, those with fragmented attention are

slower on either type of device and respondents are more likely to divide their attention when

using smartphones than PCs. Thus, multitasking is an important factor in explaining the time

difference not because it was more impactful on one type of device than the other but because

they are more prevalent when using smartphones than PCs.

Second, familiarity and experience with smartphones had large effects on RTs in mobile

web. Novice users, especially those who were provided smartphones for the experiment (and

thus using an unfamiliar device), were slower when using smartphone than PCs. However,

experienced users took about the same amount of time to complete the survey on either device.

15

At first blush, this seems to suggest that the time difference should disappear in surveys where

respondents choose which device to use, as one might expect that it is only the expert users who

choose to use smartphones. However, such surveys still report time differences (e.g., Andreadis,

2015). This is likely due to the fact that respondents may choose their device not based solely on

comfort and familiarity but rather based on which device is available to them. For examples, they

might choose to use a smartphone because it happens to be nearby, they are on-the-go, or it is

their only device. Further, increased multitasking on smartphones contributes to the time

difference irrespective of respondent’s level of comfort with their devices.

Our third result was that respondent characteristics, not page characteristics, explain most of

the time difference. In other words, the time gap had more to do with the composition of the

sample than with the design of individual pages, at least in our particular survey. It is possible

that if there had been more variation in question types (e.g., if grids were included) and page

design (e.g., if more pages had multiple questions) then there would have been more variation at

the page level. Nonetheless, for our study this implies that if a researcher wanted to eliminate the

time gap then changes to the questionnaire design (e.g. eliminating scrolling or text entry

questions) would have had less impact than changes to the sample (e.g., recruiting more

experienced smartphone users). This finding presents a challenge for researchers given that

changes to the questionnaire design are often in their control whereas changes to the sample are

not if trying to recruit a representative sample.

We found no evidence that slower typing on touchscreens or respondents’ level of vision or

level of dexterity account for the time difference. While scrolling did add time in the mobile web

survey, we found that is did not have a significant effect of the smartphone-PC time difference.

The questions in the survey required little scrolling, and those that did required only vertical

16

scrolling which we assume to be quite natural on smartphones (unlike horizontal scrolling). It is

likely that if the survey had longer questions that required more scrolling (especially horizontal

scrolling), then scrolling would have contributed to the time difference.

There are limitations with this analysis. We could not construct a measure of transmission

time between pages because we did not have access to client-side timestamps. We did not have a

direct, passively measured indicator of whether respondents actually scrolled (see Couper &

Peterson, 2017). Instead, we viewed each page on a standard (4-inch) smartphone to determine

whether the full question text was initially visible or not. It is possible that some users of large

phones did not actually have scroll on every page that required scrolling on a 4-inch screen.

Another limitation is that several measures used in our analysis were self-reported and thus

subject to measurement error. Further, the multitasking and distraction measures were reported at

the end of the survey even though they were likely intermittent, with respondent attention

fluctuating throughout the survey. There would be value if future research that uses passively

measured, fine-grained indicators of respondents’ activities and environment (see Sendelbah et

al. 2017). Nonetheless, we think this analysis demonstrates that respondent characteristics – in

particular, multitasking and respondents’ level of experience with smartphones – contribute to

the smartphone-PC time difference.

Will the time gap eventually be erased? If mobile survey design improves and users gain

more experience using smartphones, the time gap will likely shrink. However, it seems unlikely

to be eliminated given the prevalence of multitasking on smartphones.

The shift towards mobile data collection shows no signs of slowing down, and research in

this area must keep pace. An important task will be to investigate the factors affecting response

17

behaviors when using smartphones, including response latencies. The resulting findings will

likely inform new strategies to improve the design and implementation of mobile surveys.

18

Author Information

Christopher Antoun is an assistant research professor in the Joint Program in Survey

Methodology and College of Information Studies at the University of Maryland. E-mail:

antoun@umd.edu

Alexandru Cernat is a lecturer in Social Statistics in the School of Social Sciences at the

University of Manchester. E-mail: alexandru.cernat@manchester.ac.uk

Acknowledgements

The data for this analysis come from an experiment carried out by the LISS (Longitudinal

Internet Studies for the Social sciences), which is administered by CentERdata (Tilburg

University, The Netherlands). We grateful to them for conducting this experiment and for

providing the timing data.

19

References

American Association for Public Opinion Research (AAPOR). 2015. “Standard definitions: Final

dispositions of case codes and outcome rates for surveys.” Available at

http://www.aapor.org/AAPORKentico/AAPOR_Main/media/publications/Standard-

Definitions2015_8theditionwithchanges_April2015_logo.pdf.

Andreadis, I. (2015). Comparison of response times between desktop and smartphone users. In

Toninelli, D. et al. (eds.) Mobile Research Methods, 63–79. London: Ubiquity Press.

Antoun, C., Couper, M. P., & Conrad, F. G. (2017). Effects of mobile versus PC web on survey

response quality: A crossover experiment in a probability web panel. Public Opinion

Quarterly, 81, 280–306.

Callegaro, M. (2010). Do you know which device your respondent has used to take your online

survey? Using paradata to collect information on device type. Survey Practice, 3 (6).

Couper, M. P., & Peterson, G. J. (2017). Why do web surveys take longer on smartphones?.

Social Science Computer Review, 35, 357–377.

Couper, M. P., Antoun, C. & Mavletova, A. (2017). Mobile web surveys: A total survey error

perspective. In Biemer, P. et al. (eds.) Total Survey Error in Practice, 133–154. New York:

Wiley.

Fernee, H., & Sonck, N. (2013). “Is Everyone Able to Use a Smartphone in Survey Research?”

Survey Practice, 6, 1–7.

Gummer, T., & Roßmann, J. (2015). Explaining interview duration in web surveys: A multilevel

approach. Social Science Computer Review, 33, 217–234.

Link, M. W., Murphy, J., Schober, M. F., Buskirk, T. D., Hunter Childs, J., & Langer Tesfaye,

C. (2014). Mobile technologies for conducting, augmenting and potentially replacing

20

surveys: Executive summary of the AAPOR task force on emerging technologies in public

opinion research. Public Opinion Quarterly, 78, 779–787.

Mavletova, A., & Couper, M. P. (2016). Device use in web surveys: The effect of differential

incentives. International Journal of Market Research, 58, 523–544.

McGeeney, K., & Marlar, J. (2013). “Mobile browser web surveys: Testing response rates, data

quality, and best practices.” Paper presented at the AAPOR annual conference, Boston,

May.

Scherpenzeel, A. C. (2011). Data collection in a probability-based internet panel: How the LISS

panel was built and how it can be used. Bulletin of Sociological Methodology 109, 56–61.

Sendelbah, A., Vehovar, V., Slavec, A., & Petrovčič, A. (2016). Investigating respondent

multitasking in web surveys using paradata. Computers in Human Behavior, 55, 777–787.

Snijders, T. A. B., & Bosker, R. (2011). Multilevel Analysis: An Introduction To Basic And

Advanced Multilevel Modeling (2nd Revised edition edition). Sage Publications Ltd.

Yan, T., & Tourangeau, R. (2008). Fast times and easy questions: the effects of age, experience

and question complexity on web survey response times. Applied Cognitive Psychology, 22,

51–68.

21

Table 1. Description and distribution of page-level and respondent-level predictors. Predictor Description Percentage/MeanPage-levelQuestion type Single-choice Radio button and check box questions 55% Text entry Mix of numeric entry and text entry 45%Scrolling required in mobile version Yes 18% No 82%Respondent-level (time varying)Multitasking (# of other tasks) Self-reported at end of each survey 0 Mobile web: 47%

PC web: 56% 1 Mobile web: 28%

PC web: 25% 2 Mobile web: 17%

PC web: 15% 3 Mobile web: 5%

PC web: 4% 4 Mobile web: 1%

PC web: <1% 5+ Mobile web: 1%

PC web: <1%Distracted Self-reported at end of each survey Yes Mobile web: 38%

PC web: 36%

No Mobile web: 62% PC web: 64%

Respondent-level (fixed)Experience with smartphones Survey taking experience self-reported in

earlier survey a. Low Smartphone provided for experiment 38% Medium Own smartphone but don’t use it to

complete web surveys56%

High Own smartphone and do use it to sometimes complete surveys

7%

Frequency of smartphone use Self-reported in earlier survey a. “Never” 26% “Rarely” 5% “Some days” 10% “Most days” 12% “Every day” 46%Corrected eye sight Eye site with “(reading) glasses or contact

lenses.” Self-reported in earlier surveyb. “excellent”/“very good”/“good” 89% “reasonable”/“poor” 11%Working with hands and Self-reported in earlier surveyb.

22

fingers “no complaints” 92%“complaints” 8%

aNonresponse and Measurement Error in Mobile Web Surveys – Baseline (Sept 2013)bLISS Core Study – Health Wave 7 (Nov-Dec 2013)

23

Table 2. Results from multilevel models explaining log response times for PC web, mobile web, and within-respondent differences between the two.

Coefficients PC-empty PC-full Mobile-empty Mobile-full Diff-empty Diff-full  Intercept 2.30*** 3.34*** 2.65*** 3.60*** 0.35*** 0.20*  

(-0.12) (-0.25) (-0.11) (-0.21) (-0.03) (-0.1)  Female 0.01 0.01 0  

(-0.02) (-0.02) (-0.02)  Age: 31-45 0.15*** 0.08* -0.07**  

(-0.03) (-0.03) (-0.03)  Age: 46-60 0.28*** 0.21*** -0.07*  

(-0.03) (-0.03) (-0.03)  Age: 61- 75 0.44*** 0.34*** -0.10**  

(-0.04) (-0.04) (-0.03)  College -0.03 -0.07*** -0.04  

(-0.02) (-0.02) (-0.02)  Text input -0.81*** -0.73*** 0.08

(-0.15) (-0.13) (-0.05)Scrolling 0.60** 0.60*** 0

(-0.2) (-0.16) (-0.06)Multitasking 0.07*** 0.06*** 0.06***

(-0.01) (-0.01) (-0.01)Distracted 0.06** 0.06** -0.02

(-0.02) (-0.02) (-0.02)Experience: Medium -0.03 0.06 0.09*  

(-0.04) (-0.04) (-0.04)  Experience: Low -0.06 0.24*** 0.30***  

(-0.05) (-0.05) (-0.05)  Freq. smartphone use -0.03** -0.07*** -0.04***  

(-0.01) (-0.01) (-0.01)  Poor eyesight 0.02 0.03 0  

(-0.03) (-0.03) (-0.03)  Dexterity complaints 0.01 0.01 0  

(-0.04) (-0.04) (-0.04)  Second wave -0.08*** -0.07***  

(-0.02) (-0.02)  Mobile first 0.16***              (-0.02)  AIC 34037.16 33821.29 33635.21 33144.53 42942.45 42706.76  BIC 34068.35 33977.26 33666.4 33300.5 42973.64 42862.74  Log Likelihood -17014.58 -16890.64 -16813.6 -16552.26 -21467.22 -21333.38  N: Respondents x Pages 18011 18011 18011 18011 18011 18011  N: Respondents 822 822 822 822 822 822  

24

N: Pages 22 22 22 22 22 22  Var: Respondents 0.1 0.06 0.15 0.07 0.09 0.05  Var: Pages 0.33 0.13 0.26 0.09 0.01 0.01  Var: Residual 0.35 0.35 0.34 0.34 0.59 0.59  

* p < 0.05, ** p < 0.01, *** p < 0.001

25

Figure 1. Single-choice question in mobile (left) and PC web (right).

26

Figure 2. Text-entry question in mobile (left) and PC web (right)

27

Figure 3. Variance decompositions for the three models explaining log times in PC web and mobile web as well as the difference between them.

28