Saturation effect on over-surveyed people versus willingness to participate in online
panels: experimental results
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
Prof. Silvia BiffignandiDr. Roberto ArtazUniversity of Bergamo (Italy)
Panel system PAADEL
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
PAADEL (Agro-food and Demographic Panel in Lombardy) Project & Task: building up two online panels • businesses panel (PAADEL producers agro-food sector firms) • households panel (PAADEL consumers)
Characteristics: representative panels
Pilot – experiment: panel with whole sub-population (PAADEL BgPA)
Theoretical approach: the research strand based on the Norm of Reciprocity (Gouldner, 1960), framework that enables to explain the effectiveness of inducements.
…in offline panels and in those online completing a given wave seems to be an indicator for responding to next wave (Presser,
1989, Göritz and Wolff, 2007; Göritz, Wolff and Goldstein, 2008).
…role of incentive in motivating survey participation (Göritz, 2006; 2008; Bandilla and Haas, 2009).
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
People becoming over-surveyed by the too many invitations received and so turn less interested in response (Jarret, 2012)
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
Hypothesis tested:
participation in previous wave sets a ceiling for the response in the next wave.
Findings: Hypothesis not supported in the last wave of a 4 waves experiment
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
3-wave experiment conducted online in September, October and November 2011
and a 4th Wave in November 2012.
E-mail invitations to a sample of 502 individuals at first (500 at last)
(members of local governments in province of Bergamo since in Italy publishing officials’ online e-mail address and C.V. is required by law).
Mean age of the individuals: 50 years, and 43,02% (n = 216) were female.
Members of the sample randomly assigned to one of two conditions:control condition (n = 250 at the start – 249 at the end) and experimental condition (n = 252 at the start – 251 at the end)
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
TREND OF THE RESPONSE RATE
TABLE 1 Units in
mailing
list
Units in
experiment
al group
Units
in
control
group
Total
amount of
respondents
Respondents
in
experimental
group
Respondents
in control
group
Response
rate
aggregate
Response
rate in
experimental
group
Response
rate in
control group
Wave 1
14.09.2011
Reminder
21.09.2011
502
=286M
+216F
252
=145M
+107F
250
=141M
+109F
154
=95M
+59F
70
=41M
+29F
84
=54M
+30F
30,67%
=33,21%M
+27,31%F
27,72%
=28,27%M
+27,10%F
33,60%
=38,29%M
+27,52%F
Wave 2
28.09.2011
Reminder
01.10.2011
501*
=286M
+215F
251
=145M
+106F
250
=141M
+109F
95
=48M
+47F
46
=21M
+25F
49
=27M
+22F
18,96%
=16,78%M
+21,86%F
18,32%
=14,48%M
+23,58%F
19,60%
=19,14%M
20,18%F
Wave 3
12.10.2011
Reminder
19.10.2011
500**
=285M
+215F
251
=145M
+106F
249
=140M
+109F
84
=42M
+42F
39
=20M
+19F
45
=22M
+23F
16,80%
=14,73%M
+19,53%F
15,53%
=13,79%M
+17,92%F
18,07%
=15,71%M
+21,10%F
Wave 4
03.11.2012
Reminder
10.11.2012
500
=285M
+215F
251
=145M
+106F
249
=140M
+109F
97
=54M
+43F
43
=24M
+19F
54
=30M
+24F
19,40%
=18,94%
+20%
17,13%
=16,55%
+17,92%
21,68%
=21,42%
+22,01%
* One subject (F) refused to take part in the wave 2 expressly and asked to be deleted from the mailing list.
** One subject (M) refused to take part in the wave 3 expressly and asked to be deleted from the mailing list after he received the reminder of the
wave 2. Another subject (M) asked to be deleted from the mailing list after he received the invitation to wave 3 but didn’t refuse to take part in it
expressly.
Label: M = male F = female
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
Response rates: distributions by age/gender classes
Wave1
Wave2
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
Wave3
Wave 4
ESRA 5th Conference of the European Survey Research Association
Ljubljana, Slovenia, July, 15–19, 2013
Results
saturation effect seems to be reduced by the long rest and more in control group, where an incentive wasn’t provided, than in the experimental group benefited by an incentive
(fourth wave shows in control group a response rate of 21,68% even better than W2 and in experimental group a response rate of 17,13% better than W3)
findings also point out that in a context of over-surveyed people left in peace for a long moment, inner motivations are more effective in boosting response rate than external inducements given in exchange for participation