measuring alcohol consumption and alcohol-related problems: comparison of responses from...

Addiction (2001) 96, 459–471

RESEARCH REPORT

Measuring alcohol consumption andalcohol-related problems: comparison ofresponses from self-administeredquestionnaires and telephone interviews

LUDWIG KRAUS & RITA AUGUSTIN

IFT Institute for Therapy Research, Munich, Germany

AbstractAims. Compared with surveys using self-administered questionnaires, telephone interviews generally yieldhigher coverage rates, have a lower proportion of missing values and result in fewer inconsistencies.Meta-analyses, however, show that responses to sensitive questions by telephone tend to be biased by socialexpectations. The aim of the study is to examine whether responses on alcohol consumption and alcohol-relatedproblems differ with respect to mode of administration (self-administered vs. telephone). Design andparticipants. Data were analysed from the 1995 self-administered survey among 6427 subjects and fromtelephone surveys conducted annually between 1994 and 1996 yielding a pooled sample of 6193 subjects.Measurements. Alcohol consumption within the last 30 days was measured using a beverage-speci� cquantity–frequency index. For a summary measure responses were converted into pure alcohol (ethanol) perday and categorized into no alcohol consumption (0 g), non-hazardous consumption ( # 20 g for female and# 40 g for males) and hazardous consumption ( . 20 g for females and . 40 g for males). Alcohol-relatedproblems were assessed using the CAGE questionnaire with a cut-off point of at least two positive responses.Findings. Using (cumulative) logistic regression, a signi� cant mode effect was found for both alcoholconsumption and alcohol-related problems. Lower beverage-speci� c prevalences in the telephone mode werefound to be responsible for the difference in the distribution of the summary consumption measure.Conclusions. Results indicate that patterns of drinking and alcohol-related problems are more easilyreported in self-administration questionnaires compared to telephone interviews.

IntroductionThe German National Survey on PsychoactiveSubstances (NSPS), commissioned by the Ger-man Federal Ministry of Health and conductedfor the � fth time since 1980, has always usedself-administered questionnaires for research onconsumption patterns and addiction (Kraus,

Bauernfeind & Buhringer, 1998; Kraus &Bauernfeind, 1998). In other � elds telephonesurvey methodology using the advantages ofmodern computer-assisted interviewing laborato-ries has already become an attractive alternativeto both mail and face-to-face inquiry (Frey,1989). Compared to other survey modes, tele-

Correspondence to: Dr Ludwig Kraus, IFT Institut fur Therapieforschung, Arbeitsgruppe Soziale Epidemiologie,Parzivalstr. 25, D - 80804 Munchen, Germany. Tel: 1 49 (0) 89 360804 30; fax: 1 49 (0) 89 360804 49; e-mail:[email protected]

Submitted 5th January 2000; initial review completed 12th May 2000; � nal version accepted 6th September 2000.

ISSN 0965–2140 print/ISSN 1360–0443 online/01/030459–13 Ó Society for the Study of Addiction to Alcohol and Other Drugs

Carfax Publishing, Taylor & Francis Limited

DOI: 10.1080/0965214002005428

460 Ludwig Kraus & Rita Augustin

phone interviews have shorter � eld times, areless expensive and allow for more ef� cient datacollection. Given comparable data quality, tele-phone interview methods are preferable for mea-suring drug and alcohol consumption in nationalor local samples of the general population.

Most previous research on mode differenceshave compared results from telephone question-naires with those conducted face to face.Investigations into differences between mail andeither telephone or face-to-face interview modeare far fewer (De Leeuw, 1992). The likelihoodof differences between either of the two interviewmodes seems smaller than between mail and thetwo interview methods, as the latter rely on auralcommunication as opposed to the visual infor-mation of mail questionnaires. Comparisons ofresponse outcome appear to corroborate this as-sumption. A meta-analysis of studies on differ-ences in data quality concerning health-relatedbehaviour (including alcohol consumption)from mail, telephone and personal interviewsfound that comparable conclusions can be drawnfrom well-conducted personal and telephoneinterviews (De Leeuw, 1992; De Leeuw &Collins, 1997). The most consistent � nding instudies on mode effects on alcohol- and drug-related behaviour comparing both interviewmethods is the lack of differences (Mangione,Hingson & Barrett, 1982; Sykes & Collins, 1998;Groves, 1989; Johnson, Hougland & Clayton,1989). On the other hand, comparisons of tele-phone interviews, personal interviews and self-administered answer sheets within personalinterviews showed substantial differences in fa-vour of self-administered questionnaires. Preva-lence rates of alcohol and illicit drug use werehighest in personal interviews with answersheets, lower in face-to-face interviews and low-est in telephone surveys (Aquilino & LoSciuto,1990; Aquilino, 1992, 1994; Gfroerer & Hughes,1992).

The � ndings of recent studies on self-reportedalcohol and drug use which compared mail toface-to-face surveys are not always consistentwith the above conclusion. For example, a Swissstudy, testing the effects of personal interview vs.postal questionnaire in a between-subjects de-sign, found higher reported alcohol consumptionin personal interviews (Rehm & Spuhler, 1993).Although the two methods in that study differedwith regard to sampling technique, another studybased on a within-subjects design yielded similar

results (Rehm, 1994). Since no sampling differ-ences were involved in the latter study, differ-ences of sampling techniques as a possibleexplanation for differences in the between-sub-jects design study could be ruled out, and thusdifferences could be attributed to assessmentmode (Rehm & Arminger, 1996). A recentDutch study also compared mail questionnairesand personal interviewing on self-reported al-cohol use and problem drinking. In the between-subjects design, the sampling frame was thesame for the two assessment modes. No notablemode differences between alcohol measures werereported (Bongers & Van Oers, 1998).

Relatively little research has been conductedon mode comparisons between mail and tele-phone surveys. A recent study using data fromthe Swiss Health Survey, in which respondentswere � rst interviewed by telephone and laterfollowed-up by a self-administered mail ques-tionnaire, reported signi� cantly more drinkers,more heavy drinkers and higher volume of drink-ing through self-administered questionnaires(Gmel, 2000). This result supports the hypoth-esis that an administration mode which guaran-tees more privacy in responding to sensitivequestions performs better.

Since mail surveys offer greater anonymity,more privacy and con� dentiality than both inter-view modes, it is hypothesized that mail surveyself-reports on sensitive topics show less socialdesirability bias (Hochstim, 1967; Sudman &Bradburn, 1974; Groves, 1990; Schwarz et al.,1991; De Leeuw, 1992). Often the presence ofinterviewers leads to a reluctance to reveal char-acteristics believed to be socially negative (DeLeeuw & Van der Zouwen, 1988; Aquilino,1994). Groves (1990) pointed out that sociallydesirable tendencies in interview situations arerather a function of con� dentiality than of socialdistance. In telephone interviews, however, thecredibility of the researchers to guaranteecon� dentiality is more dif� cult to establish. Thelack of theoretical concepts apart from the socialdesirability hypothesis as well as the focus onresponse differences alone has been criticized byDillman and colleagues (1996). They point outthree major differences between mail and tele-phone mode: � rst, presence or absence of inter-viewer; secondly, dependence on visual or auralcommunication; and thirdly, interviewer or re-spondent control of pace and information se-quence. In� uencing mechanisms may be the

Comparing responses from questionnaires and interviews 461

consciousness of social norms, the context ofresponding (sequential vs. simultaneous avail-ability of information), time pressure, memorylimitations and cognitive processing. With theexception of research on social desirability, evi-dence of the existence of consistent and predict-able differences in responses to mail andtelephone surveys is rather sparse (Dillman &Tarnai, 1991; Tarnai & Dillman, 1992; Dillmanet al., 1996). Much more evidence is providedconcerning mode differences such as responsevalidity, item non-response and similarity of re-sponses (De Leeuw, 1992).

Mode differences in non-response rates arewell documented with the lowest rates in per-sonal interviews, higher rates in telephone sur-veys and the highest in mail surveys (De Leeuw,1992; Hox & De Leeuw, 1994). Some studieshave found item non-response to be slightlyhigher in telephone than in face-to-face surveys(Groves & Kahn, 1979). Others have reportedno differences between personal and telephoneinterviews with respect to missing data, butfound both modes to be superior to mail orself-administered surveys (Hochstim, 1967; Dill-man, 1978). Another crucial issue examined wasaccess to households due to different samplingframes. The exclusion of households withouttelephones in telephone surveys may be a poten-tial bias. Aquilino (1992), for instance, foundhigher rates of alcohol, drug and tobacco usersamong respondents without telephones.

The literature on mode effects regardinghealth-related questions is consistent for per-sonal and telephone interviews. Con� icting re-sults exist when mail surveys are compared withface-to-face interviews. According to the socialdesirability hypothesis, one should expect thatthe greatest pressure occurs in face-to-face sur-veys and the least in mail surveys. With out-comes on alcohol use from telephone interviewsbeing quite similar to those from personal inter-views, one would predict higher rates of alcoholuse and alcohol-related problems in mail sur-veys. The aim of the present study is to examinedifferences in the effects of mail and telephoneadministration of a national survey of alcohol useand alcohol-related problems in the Germangeneral population.

MethodsSamplesThe data for the present analysis come from four

cross-sectional national surveys of the Germangeneral population conducted between 1994and 1996. The survey samples were designedto represent the German-speaking populationaged 18–59 years living in private households.The telephone samples differed from the mailsample by representing only households withphone service. The self-administered question-naire (SAQ) was delivered by an interviewer andcollected later or was sent back by the respon-dent by mail.

Sampling of the SAQ survey was based on amulti-stage probability sampling design (Kraus etal., 1998; Kraus & Bauernfeind, 1998). In the� rst stage constituencies were strati� ed accord-ing to region and selected at random. Withinselected constituencies a random sample wasdrawn based on a random route procedure: apoint on the city map was chosen at random.Starting from this point every third householdwas selected. In the last stage the respondentchosen in each household was the individualwith the most recent birthday. The overall re-sponse rate was 65%, resulting in 7833 respon-dents.

The telephone sample was also derivedthrough a multistage probability sampling de-sign. In the � rst stage communities werestrati� ed according to region and selected atrandom. Within selected communities a randomsample of telephone numbers proportional topopulation size was drawn from telephonebooks, where 85–90% of telephone numbers arelisted (Marhenke, 1997). In the last stage therespondent interviewed in each household wasthe individual with the most recent birthday.Each telephone sample comprises n 5 2500 re-spondents. Response rates decreased slightlyfrom 76% in 1994 to 73% and 69% in 1995 and1996, respectively. After pooling of data, sam-ples for both modes were almost the same size.

The basic assumption was that the 2-yeartime-span over which the surveys were conduc-ted was not long enough to allow major changesin consumption prevalence and alcohol-relatedbehaviour to occur. In this sense, the threetelephone surveys could be regarded as consti-tuting one larger survey occurring over the pe-riod of 1994–96. With the mail surveyconducted in 1995, data collection fell exactly inthe middle of the reference period of the threetelephone surveys. Nevertheless, the three data-sets were compared for similarity on frequency of


responses to alcohol questions such as life-timeuse, consumption in the past 12 months, bever-age-speci� c frequency and quantity and theCAGE questionnaire. With the exception of theCAGE indicator which remained nearly con-stant, all outcome measures decreased over time.After measures were controlled for age and gen-der, this tendency was still found. This system-atic effect may be due to increasing non-responserates in the observation period. The pooling ofthe dataset was therefore considered an averagewith 1995 in the middle of the reference periodand no major bias with regard to the relevantoutcome measures was expected.

A possible source of bias could be an increasefrom 75% to over 90% in the availability oftelephone service in Eastern Germany between1994 and 1996, 4–6 years after the reuni� cation.Although procuring telephone service in EasternGermany has not depended on income status(Statistisches Bundesamt, 1997), the presentdata analysis is based on the sample from West-ern Germany and Berlin only, where availabilityof telephone service has remained constant. Thisresults in a � nal telephone sample of n 5 6427respondents and a � nal SAQ sample of n 5 6193respondents.

MeasuresAlcohol consumption was measured by means ofbeverage-speci� c quantity–frequency questions.For each beverage (beer, wine/champagne, spir-its) respondents were asked for quantity andfrequency of consumption within the last 30days. Consumption was measured separately foreach beverage by multiplying the responses tothe following two questions: (i) “During the last30 days, on how many days did you drink beer(wine, spirits)?” (ii) “On average, on a day whenyou drank beer (wine, spirits), how many glassesof beer (wine, spirits) did you drink?” Responsesto the last question were coded according to themost widely used units of consumption: for beereither 0.3 l or 0.5 l glasses or bottles, for wine0.25 l glasses and for spirits either 0.02 l or 0.04 lglasses. While respondents in the mail surveycould report the consumption of both small andbig glasses of beer or spirits, respondents in thetelephone survey had to decide on the size of theglasses. Quantities of consumed beverages wereconverted into pure alcohol per day and summedup. For converting volume of consumed beer,

wine and spirits into grams of ethanol the follow-ing average ethanol contents for 1 litre wereused: 40 g, 92 g and 320 g ethanol, respectively.The alcohol measures in gram ethanol were re-coded into three drinking status groups: no al-cohol consumption (0 g), non-hazardousconsumption ( # 20 g for female and # 40 g formales) and hazardous consumption ( . 20 g forfemales and . 40 g for males). The limits of 20 gand 40 g for males and females, respectively, areused frequently in epidemiological research(Saunders et al., 1993; Edwards et al., 1994;Gmel, 2000). Item non-response was used as afourth category. This category also included in-consistent responses. Apart from the summarymeasure, past month prevalence of beer, wineand spirits as well as beverage-speci� c frequency,quantity per drinking day and natural logarithmsof beverage-speci� c mean quantity per day wereanalysed.

Alcohol-related problems were measured bythe following four items of the CAGE question-naire (May� eld, McLeod & Hall, 1974; Ewing,1984): (1) “Have you ever felt you ought to cutdown on your drinking?”, (2) “Have people an-noyed you by criticising your drinking?”, (3)“Have you ever felt bad or guilty about yourdrinking?” and (4) “Have you ever had a drink� rst thing in the morning to steady your nervesor get rid of a hangover?” The CAGE questionswere asked with respect to an occurrence ever inlife. Summary scores were calculated across re-sponses and two or more positive answers weretaken as cut-off point for the de� nition of alcoholabuse and dependence (Ewing, 1984). Since inthe mail survey only past year drinkers had toanswer the CAGE questions, the analysis of theCAGE questionnaire was restricted to this sub-group.

Except for the difference in the quantity mea-sure with regard to size of glasses wording andresponse categories in both questionnaires wereidentical. For the purpose of interviewing bytelephone, a modi� ed short version of the self-administered questionnaire was reformatted andprogrammed into a computer-assisted telephoneinterview (CATI).

Data analysisData from both surveys were weighted to � t thebivariate distribution of age group and gender,and the distribution of community size of each


Western German federal state as well as Berlin at31 December, 1995 (Statistisches Bundesamt,1997). Weights were calculated with the SPSS6.1.3 procedure GENLOG listing the iterativeproportional � tting algorithm (Agresti, 1990).This re-weighting was necessary to exclude ef-fects caused by different weighting algorithms ofthe � eld institutes.

According to Groves (1989), there are twoapproaches to the analysis of mode effects: the� rst aims at identifying inherent properties of theaural mode which might produce differences inthe survey results. The second is guided by thequestion if a telephone survey obtains the sameresults as a mail survey in spite of the differencesin the way the surveys are conducted. Weadopted the second approach, which has twoimplications in the data analysis: � rst, non-tele-phone households in the mail sample were in-cluded and secondly, the analysis was based onvalid cases.

To analyse whether mode of data collectionhas a substantial in� uence, separate regressionmodels were run using the dependent variablesof consumption of beer, wine and spirits, fre-quency and quantity per drinking day by bever-age type, alcohol consumption in grams ethanolper day and CAGE items. The independentvariables included in the models were gender,age group (18–29, 30–39, 40–49, 50–59 years)and mode of administration. Two-factor interac-tions were also included. The independent vari-ables were dummy-coded with the referencecategories female, age group 50–59 years andtelephone mode. The overall summary alcoholconsumption measure was divided in the cate-gories of no consumption, non-hazardous andhazardous drinking. We refrained from a linearregression model with the uncategorized sum-mary measure as the proportion of missing val-ues would be much higher: a respondent withhazardous quantities of one beverage and miss-ing values on another type can be included in thecategorical regression, but must be excludedfrom the linear regression. On the other hand,females reporting a mean consumption of lessthan 10 g ethanol per day and males reportingless than 20 g ethanol per day were categorizedas non-hazardous drinkers even if their data wereincomplete, thus assuming that beverages whichare not reported are not consumed, or onlyconsumed in small amounts.

The categories of overall alcohol consumption,

frequency and quantity and glasses per drinkingday were treated ordinally and a cumulative lo-gistic regression model was applied. Responsesto frequency were combined to create 12 cate-gories for beer and wine and nine frequencycategories for spirits; quantity per day was com-bined into seven categories for beer and sixcategories for both wine and spirits. Responsesto the CAGE questions, and to the consumptionof beer, wine and spirits were dichotomous andanalysed with logistic regression. The natural logof the mean consumption of beer, wine andspirits per day was analysed with ANOVA. Allregressions were calculated with SUDAAN 7.5(Shah, Baronwell & Bieler, 1997). The cumulat-ive logistic regression models the probabilities of,e.g. “no consumption” and “no consumption/non-hazardous consumption” simultaneously.Estimates for the other events of “non-hazardousconsumption” and “hazardous consumption”can then be derived easily. Apart from providinga parsimonious model, the cumulative modelavoids the problem of multiple tests, i.e. actualsigni� cance levels which may exceed the nominalsigni� cance level. This problem would occur if,for example, the probabilities of “no consump-tion” and “hazardous consumption” would bemodelled separately. Furthermore, separatemodels do not take into account that the depen-dent variable is ordinally scaled.

As reference category for the dependent vari-ables, the last category, e.g. hazardous drinking,was used in the cumulative logistic model. In thelogistic regression the � rst category, e.g. less thantwo positive answers in the CAGE questionnaire,was chosen as reference category. This affectsthe interpretation of the beta coef� cients of thetwo models. In the logistic model, a positive betacoef� cient, e.g. for the main effect “mode”, indi-cates a higher prevalence rate of, for instance,alcohol-related problems in the mail survey,whereas a positive beta coef� cient in the cumu-lative logistic model implies, for instance, a lowerlevel of alcohol consumption in the mail survey.

ResultsMode differences in alcohol consumptionIn the mail survey item non-response to thequestions on frequency and quantity of alcoholconsumption were always higher compared tothe telephone mode. In the summary measure ofethanol intake responses on frequency and quan-


Table 1. Past month alcohol consumption by administration mode, age and gender

Male Female

Self- Self-Age administered Telephone administered Telephone(years) Consumption (n 5 2.918) (n 5 2.795) (n 5 3.509) (n 5 3.398)

18–29 Missing 6.6 0.5 7.5 0.9No consumption 15.5 16.2 26.4 30.4Non-hazardous 64.6 77.5 57.8 63.5Hazardous 13.3 5.7 8.3 5.2




tity were combined. Non-response resulted fromnon-response to either one of the variables or toboth. Otherwise a missing value resulted frominconsistencies where respondents reported avalid frequency but a quantity of zero or viceversa. Most of the missing values for the sum-mary measure from the mail survey and almostall from the telephone survey were due to incon-sistencies. As shown in Table 1, non-responserates to overall alcohol consumption in the mailsurvey ranged from 4.9% (30–39-year-oldmales) to 10.1% (50–59-year-old females), whilethose in the telephone survey were much lowerranging from 0.5% (18–29-year-old males) to1.6% (40–49-year-old males).

As expected, the level of overall alcohol con-sumption was signi� cantly higher among menthan among women. Both in the telephone andthe mail survey the proportion of hazardous maledrinkers exceeded the proportions of hazardousfemale drinkers in all age groups. On the otherhand, in all age groups abstinence rates of maleswere lower than those of females. More respon-dents of the mail than of the telephone surveyfell in the low alcohol consumption and hazard-ous alcohol consumption category leading to asigni� cant main effect “mode” in the cumulativelogistic regression model. The interaction effect“mode by gender” is not signi� cant althougheach age group the observed mode effect for

prevalence rates of hazardous drinking is largerfor males than for females (Table 2).

The observed signi� cant differences betweenthe samples lead to the question whetherthe lower prevalence of hazardous drinkers in thetelephone interview is simply a result of thehigher abstinence rate or whether beverage-speci� c drinking patterns of drinkers differ be-tween the modes. Thus, beverage-speci� canalyses were based on drinkers only. With re-gard to the in� uence of the assessment mode theresults shown in Tables 2 and 3 resemble thosefor the overall summary measure. The consump-tion of each beverage type was reportedsigni� cantly more often in the mail than in thetelephone survey. Mode differences in the con-sumption of beer and spirits were signi� cantlylarger for males whereas for wine consumptionthe interaction effect “mode by gender” was notsigni� cant. Clearly, more males than femalesreported the consumption of beer, wine andspirits in the past month, but neither age effectsnor gender by age interactions were foundsigni� cant.

However, separate comparisons of frequencyand quantity per drinking day by drinkers ofeach beverage type showed a rather differentpattern (Table 4). While gender remained asigni� cant factor with males reporting higherbeverage-speci� c frequencies and quantities


Tab

le2.

Log

isti

cre

gres

sion

for

cons

umpt

ion

ofbe

er,w

ine

and

spir

its

amon

gpa

stm

onth

sdr

inke

rs,

and

the

CA

GE

scor

eam

ong

12-m

onth

drin

kers

a ;cum

ulat

ive

logi

stic

regr

essi

onfo

rpa

stm

onth

alco

hol

cons

umpt

iona

Alc

ohol

con

sum

ptio

nB

eer

con

sum

ptio

nW

ine

con

sum

ptio

nS

pir

its

con

sum

ptio

nC

AG

E(t

otal

sam

ple

(pas

tm

onth

dri

nke

r(p

ast

mon

thd

rin

ker

(pas

tm

onth

dri

nke

r(p

ast

12

mon

thd

rin

ker

n5

12.6

20)

n5

10.0

79)

n5

10.0

79)

n5

10.0

79)

n5

11.2

63)

Var

iabl

eb

p-va

lue

bp-

valu

eb

p-va

lue

bp-

valu

eb

p-va

lue

Mod

eM

ail

20.

360.

000.

380.

000.

600.

000.

620.

000

.96

0.00

Age

(yea

rs)

18–2

92

0.01

0.92

20.

050.

682

0.07

0.57

0.11

0.34

0.3

10.

1630

–39

20.

180.

042

0.01

0.91

20.

070.

512

0.08

0.51

0.4

00.

0540

–49

20.

390.

002

0.02

0.88

0.07

0.57

20.

200.

100

.46

0.04

Sex M

ale

20.

790.

001.

990.

002

1.13

0.00

0.27

0.01

1.3

50.

00M

ode

3ag

eM

ail,

18–2

90.

180.

142

0.18

0.25

0.10

0.46

20.

080.

572

0.4

50.

02M

ail,

30–3

92

0.02

0.86

0.05

0.71

0.03

0.80

20.

020.

902

0.0

90.

62M

ail,

40–4

90.

040.

740.

090.

552

0.02

0.90

0.30

0.03

20

.03

0.88

Mod

e3

Sex

Mai

l,M

ale

20.

140.

100.

280.

020.

080.

390.

340.

002

0.1

40.

28A

ge3

sex

18–2

9,M

ale

0.23

0.04

20.

060.

712

0.05

0.75

20.

090.

452

0.1

30.

5330

–39,

Mal

e0.

230.

050.

001.

002

0.04

0.74

20.

050.

692

0.2

90.

1340

–49,

Mal

e0.

320.

012

0.15

0.42

20.

080.

570.

150.

272

0.2

70.

17

a Du

mm

y-co

ded

vari

able

s.


Table 3. Proportion of beer, wine, and spirits consumption of past month drinkers by administration mode, ageand gender

Male Female

Age Self-administered Telephone Self-administered Telephone(years) Consumption (n 5 2.557) (n 5 2.357) (n 5 2.716) (n 5 2.433)

18–29 Beer 91.8 87.0 54.0 51.5Missing 0.9 0.4 2.5 0.5Wine 65.4 47.5 83.7 74.0Missing 0.9 0.1 2.5 0.1Spirits 53.8 33.5 41.3 28.9Missing 0.9 0.9 2.5 0.6




compared to females, mode was no longersigni� cant. Compared to the oldest age groupthe frequencies of each beverage type weresigni� cantly lower in the younger age group,whereas quantities were signi� cantly higher. Ex-cept for quantity of spirits consumption nosigni� cant interaction effects “mode by age”were found. Only for wine consumption asigni� cant interaction effect “gender by age” wasobserved.

Additional comparisons of the natural loga-rithms of mean overall quantity of beer, wineand spirits consumption per day produced simi-lar results. Males reported signi� cantly largerquantities than females, but no signi� cant“mode” and “mode by age” effects were found(data not shown).

Mode differences in alcohol-related problemsWhile in the telephone interviews nearly all re-spondents who reported consumption in the last12 months answered the CAGE questions ap-

propriately, item non-response in the mail surveyranged from 4% (30–39-year-old males) to10.8% (50–59-year-old females). In all groupsnotably more females than males failed to answerthese questions. There were, however, only mi-nor differences between the age groups (Table5).

Surprisingly, despite the fact that the CAGEitems re� ect life-time prevalence of alcohol-related problems, younger age groups sometimesexhibited higher prevalence rates than oldergroups. The highest prevalence rates were alwaysfound in the age group 40–49 years. Again,signi� cantly more males than females reportedalcohol problems and prevalence rates weresigni� cantly higher in the mail than in the tele-phone survey (Table 3). Neither the interactioneffect “mode by gender” nor “age by gender”were signi� cant. There was, however, onesigni� cant interaction effect between age groupand mode. In the youngest age group prevalencerates between self-administered questionnairesand telephone interviews differed less than inother age groups.


Tab

le4.

Cum

ulat

ive

logi

stic

regr

essi

onfo

rpa

stm

onth

freq

uenc

yof

beer

,w

ine

and

spir

itsco

nsum

ptio

na ,an

dqu

anit

ype

rdr

inki

ngda

yfo

rbe

er,

win

e,an

dsp

irit

sa

Fre

qu

ency

(pas

tm

onth

)Q

uan

tity

(pas

tm

onth

)

Bee

rd

rin

ker

Win

ed

rin

ker

Spi

rits

dri

nke

rB

eer

dri

nke

rW

ine

dri

nke

rS

pir

its

dri

nke

r(n

57.

324

)(n

56

.897

)(n

53.

905)

(n5

7.32

4)

(n5

6.8

97)

(n5

3905

)

Var

iabl

eb

p-va

lue

bp-

valu

eb

p-va

lue

bp-

valu

eb

p-va

lue

bp-

valu

e

Mod

eM

ail

20

.20

.12

20.

10.

610.

10.

470

.00

.54

0.1

0.65

20.

00.

80A

ge(y

ears

)18

–29

0.5

0.0

00.

40.

000.

70.

002

0.8

0.0

02

0.3

0.03

21.

00.

0030

–39

0.3

0.0

20.

10.

440.

70.

002

0.5

0.0

02

0.1

0.24

20.

70.

0040

–49

20

.00

.72

20.

10.

360.

30.

012

0.5

0.0

02

0.3

0.03

20.

20.

27S

ex Mal

e2

1.0

0.0

02

0.3

0.00

20.

50.

002

1.6

0.0

02

0.4

0.00

20.

50.

00M

ode

3ag

eM

ail,

18–2

90

.00

.76

20.

10.

412

0.2

0.23

0.1

0.4

22

0.1

0.69

0.4

0.06

Mai

l,30

–39

20

.00

.76

0.1

0.65

20.

20.

250

.10

.25

0.1

0.62

0.4

0.04

Mai

l,40

–49

0.1

0.6

62

0.1

0.41

20.

10.

540

.00

.91

20.

00.

960.

20.

31M

ode

3se

xM

ail,

mal

e2

0.2

0.0

20.

00.

672

0.2

0.11

20

.30

.00

0.1

0.25

20.

30.

02A

ge3

sex

18–2

9,M

ale

20

.00

.95

0.5

0.00

0.2

0.28

0.3

0.0

20.

40.

010.

20.

1830

–39,

Mal

e2

0.1

0.4

60.

40.

010.

01.

000

.10

.40

0.3

0.04

0.2

0.22

40–4

9,M

ale

20

.00

.73

0.4

0.01

0.1

0.73

0.2

0.0

90.

10.

502

0.1

0.69

a Du

mm

y-co

ded

vari

able

s.


Table 5. Alcohol-related problems (CAGE) of past 12-month drinkers by administration mode, age and gender

Male Female

Age Self-administered Telephone Self-administered Telephone(years) CAGE (n 5 2.568) (n 5 2.524) (n 5 2.849) (n 5 2.877)

18–29 Missing 6.1 0.2 8.5 0.0One or none 76.2 86.6 85.4 95.1Two or more 17.8 13.2 6.1 4.9




DiscussionIn this paper, mode differences of data collectionon drinking behaviour and alcohol-related prob-lems were assessed. Differences in responses be-tween self-administered questionnaires andtelephone interviews were found to be in theexpected direction, but were larger than reportedin related survey research (Aquilino, 1994;Gmel, 2000). Mode differences in abstinencerates indicate that fewer respondents admit al-cohol consumption if questioned by telephonecompared to self-reports from questionnaires.Differences in the proportion of non-hazardousand hazardous drinkers lead to the question ofwhether the lower prevalence of hazardousdrinkers in the telephone interview is simply aresult of the higher abstinence rate or whetherbeverage-speci� c drinking patterns of drinkersdiffer between the modes.

While the prevalence rates of beer, wine andspirits consumption among drinkers weresigni� cantly lower in the telephone survey com-pared to the mail survey, the main mode effectwas no longer signi� cant if beverage-speci� c fre-quencies and quantities per drinking day werecompared. Thus, the � ndings indicate differentresponse patterns in that drinkers appear to ad-mit the consumption of more than one beverageless frequently in telephone interviews. Only13.7% of past month drinkers reported the con-sumption of all three beverages in the presenttelephone survey compared to 43.2% of those inthe mail survey. Since overall quantity of alcoholconsumption is an additive function of beverage-

speci� c quantities, under-reporting of each bev-erage type results in lower quantities of overalldrinking and thus in lower prevalence rates ofhazardous drinking.

The signi� cant “mode by gender” interactionwith regard to all measures on beer and spiritsconsumption reveals a larger in� uence of theassessment mode on males compared to females.The larger gender differences for quantities perdrinking day in the mail survey compared to thetelephone survey, however, may result partlyfrom differences between the survey questionson beer and spirits: in the mail survey 3.6% ofmale beer drinkers and 2.8% of male spiritsdrinkers reported the consumption of both smalland big glasses compared to 1.9% of female beerdrinkers and 2.1% of female spirits drinkers. Inthe telephone survey respondents had to decideon the size of the glasses.

However, these � ndings may be biased bydifferences in non-response between the twosurvey modes. According to Groves (1989),non-response bias is a function of response rateand the difference between respondents andnon-respondents with respect to relevant out-come measures. Follow-up of non-respondentscannot support the common expectation thatnon-response will seriously affect survey esti-mates (Lemmens, Tan & Knibbe, 1988; Cas-par, 1992). In most of these studies differencesfound in self-reported alcohol consumption mea-sures were non-signi� cant. In a recent study onthe effect of non-response in Switzerland Gmel(2000) found slightly but not signi� cant higher


mean alcohol consumption in non-respondentsand concluded that non-response rates affectsurvey estimates less than mode of administra-tion.

Item non-response also appears to be corre-lated with mode of administration. A high pro-portion of missing values and inconsistencies wasencountered in the self-administered question-naire while telephone interviewing produced al-most no missing data and only fewinconsistencies. This is not surprising, given thattelephone interviews are highly structured andguided by a professional interviewer. However,missing values cannot account for the lower esti-mates in the telephone survey, since its missingvalue rate is lower than in the self-administeredversion (see Tables 1 and 5).

Investigation into patterns of missing values inthe self-administered questionnaire revealed thatamong current drinkers (past 12 months) a sub-stantial proportion of missing data found in theCAGE items was produced by those respondentswho reported past-month abstinence (between17% and 40%, depending on age and gender).Only a small proportion of missing values wasrelated to heavy drinking. This � nding indicatesthat item non-response and inconsistencies inself-administered questionnaires are caused bycarelessness or overlooking of certain itemsrather than by deliberate refusal.

Since there is a growing tendency toward useof mixed-mode surveys, the question of thein� uence of non-telephone households emerges.Although over 95% of Western German house-holds had telephones, the proportion of house-holds without telephone varied with householdsize. In the mail survey 6% of households re-ported to have no telephone and among drinkersthis proportion was 5%. Comparisons of heavydrinkers and CAGE scores (two or more posi-tive) showed higher rates among respondentswithout telephones both for males and females.Given the low proportion of households withouttelephones, however, impact on the full sample israther negligible.

The present results agree strongly with recent� ndings showing differences between self-admin-istered and telephone modes for various alcoholmeasures (Gmel, 2000). They are also in linewith research reporting differences on alcohol-and drug-related behaviour in favour of self-completed questionnaires within face-to-face in-terviews, indicating that admission of

alcohol-related behaviour is facilitated by self-administration rather than by the interviewerrespondent interaction (Aquilino & LoSciuto,1990; Aquilino, 1992; 1994; Gfroerer & Hughes,1992). Rehm & Arminger (1996) foundsigni� cantly higher responses to drinking in per-sonal interviews compared to self-administeredquestionnaires and offered the hypothesis thatsince drinking is a social event, the “social event”of an interview should also produce more report-ing of alcohol-related behaviour. Bongers & vanOers (1998), however, could not � nd support forthis alternative hypothesis in their recent study,where results pointed towards higher rates onalcohol-related behaviour in the self-administra-tion mode.

In most of the literature on mode differences,researchers emphasize the social desirability hy-pothesis with the notion that absence of inter-viewers facilitates response willingness.However, as pointed out by Dillman and col-leagues (1996), other mechanisms (e.g. timepressure, memory limitations) are expected tohave an effect on answers in the telephone for-mat that are not expected to occur in the self-administration format. The research literature isalso consistent about the inter-relationship be-tween socio-demographic variables such as sex,age, and income status and survey response(Goyder, 1987; Groves, 1989). Unravellingthese mechanisms, their interactions and the im-pact of socio-demographic traits requires furtherresearch, especially since the question of interesthas shifted from that of comparability to that ofwhether and how different modes of administra-tion can be combined (e.g. Tarnai & Dillman,1992; Rehm & Arminger, 1996).

It has to be kept in mind that since surveyestimates underestimate sales data by between40 and 60% the mode yielding higher estimatesis generally considered the more valid one. Fol-lowing the common “the more, the better” argu-ment, evidence has been collected in support ofthe self-administration mode in survey research.It appears that beverage-speci� c drinking andalcohol-related problems are more likely re-ported in situations where the respondent is notinteracting with an interviewer, where he or shehas control over the context as well as over paceof responding, and where information is taken invisually without time pressure. Although per-sonal and telephone interview modes may havethe advantage of higher overall response rates


and less item non-responses, self-administeredquestionnaires seem to be less affected by socialdesirability and interviewer effects.

AcknowledgementsThis research was supported by the GermanFederal Ministry of Health which also funded allwaves of the German National Survey on Psy-choactive Substances (NSPS) a repeated cross-sectional survey, used in the present analysis.

ReferencesAGRESTI, A. (1990) Categorical Data Analysis (New

York, John Wiley and Sons, Inc.).AQUILINO, W. S. & LOSCIUTO, L. A. (1990) Effects of

interview mode on self-reported drug use, PublicOpinion Quarterly, 54, 362–395.

AQUILINO, W. S. (1994) Interview mode effects insurveys of drug and alcohol use: a � eld experiment,Public Opinion Quarterly, 58, 210–240.

AQUILINO, W. S. (1992) Telephone versus face-to-faceinterviewing for household drug use surveys, Inter-national Journal of the Addictions, 27, 71–91.

BONGERS, I. M. B. & VAN OERS, J. A. M. (1998) Modeeffects on self-reported alcohol use and problemdrinking: mail questionnaires and personal inter-viewing compared, Journal of Studies on Alcohol, 59,280–285.

CASPAR, R. (1992) Follow-up of nonrespondents in1990, in: TURNER, C.F., LESSLER, J.T. & GFROERER,J.C. (Eds) Survey Measurement of Drug Use, Methodo-logical studies, DHHS Pub. no. (ADM) 92–1929, pp.155–173 (Washington DC, US Government Print-ing Of� ce).

DE LEEUW, E. D. & COLLINS, M. (1997) Data collec-tion methods and survey quality: an overview, in:LYBERG, L., BIEMER, P., COLLINS, M., DE LEEUW, E.,DIPPO, C., SCHWARZ, N. & TREWIN, D. (Eds) SurveyMeasurement and Process Quality, pp. 199–220 (NewYork, John Wiley and Sons, Inc.).

DE LEEUW, E. D. & VAN DER ZOUWEN, J. (1988) Dataquality in telephone and face-to-face surveys: a com-parative analysis, in: GROVES, R.M., BIEMER, P.P.,LYBERG, L.E., MASSEY, J.T., NICHOLLS II, W.L. &WAKSBERG, J. (Eds) Telephone Survey Methodology,pp. 283–299 (New York, John Wiley and Sons,Inc.).

DE LEEUW, E. D. (1992) Data Quality in Mail, Tele-phone, and Face to Face Surveys (Amsterdam, TT-Publikaties).

DILLMAN, D. A. & TARNAI, J. (1991) Mode effects ofcognitively designed recall questions: a comparisonof answers to telephone and mail surveys, in:BIEMER, P.P., GROVES, R.M., LYBERG, L.E.,MATHIOWETZ, N.H. & SUDMAN, S. (Eds) Measure-ment Errors in Surveys, pp. 73–93 (New York, JohnWiley and sons, Inc.).

DILLMAN, D. A. (1978) Mail and Telephone Surveys(New York, John Wiley and Sons, Inc.).

DILLMAN, D. A., SANGSTER, R.L., TARNAI, J. & ROCK-

WOOD, T. H. (1996) Understanding differences inpeople’s answers to telephone and mail surveys, in:BRAVERMANN, M. & SLATER, J. (Eds) Advances inSurvey Research, pp. 45–61 (San Francisco, Jossey-Bass Publishers).

EDWARDS, G., ANDERSON, P., BABOR, T. F., CASSWELL,S., FERRENCE, R., GIESBRECHT, N., GODFREY, C.,HOLDER, H. D., LEMMENS, P., MAKELA, K., MI-

DANIK, L. T., NORSTROM, T., OSTERBERG, E.,ROMELSJO, A., ROOM, R., SIMPURA, J. & SKOG, O. J.(1994) Alcohol Policy and the Public Good (Oxford,Oxford University Press).

EWING, J. (1984) Detecting alcoholism: the CAGEquestionnaire, Journal of the American Medical Associ-ation, 252, 1905–1907.

FREY, J. H. (1989) Survey Research by Telephone (Lon-don, Sage Publication).

GFROERER, J. C. & HUGHES, A. L. (1992) Collectingdata on illicit drug use by phone, in: TURNER,C.F., LESSLER, J.T. & GFROERER, J.C. (Eds)Survey Measurement of Drug Use. Methodologicalstudies , DHHS Pub. No. (ADM) 92–1929, pp. 277–295 (Washington DC, US Government PrintingOf� ce).

GMEL, G. (2000) The effect of mode of data collectionand of non-response on reported alcohol consump-tion: a split-sample study in Switzerland, Addiction,95, 123–134.

GOYDER, J. (1987) The Silent Minority (Cambridge,Blackwell).

GROVES, R. M. & KAHN, R. L. (1979) Surveys byTelephone: a national comparison with personal inter-views (New York, Academic Press).

GROVES, R. M. (1989) Survey Errors and Survey Costs(New York, John Wiley and Sons, Inc.).

GROVES, R. M. (1990) Theories and methods of tele-phone surveys, Annual Review of Sociology, 16, 221–240.

HOCHSTIM, J. R. (1967) A critical comparison of threestrategies of collecting data from households, Journalof the American Statistical Association, 62, 976–989.

HOX, J. J. & DE LEEUW, E. D. (1994) A comparison ofnon-response in mail, telephone, and face-to-facesurveys, Quality and Quantity, 28, 329–344.

JOHNSON, P. T., HOUGLAND, J. G. & CLAYTON, R. R.(1989) Obtaining reports of sensitive behavior: acomparison of substance use reports from telephoneand face-to-face interviews, Social Science Quarterly,70, 174–183.

KRAUS, L. & BAUERNFEIND, R. (1998) Reprasentativer-hebung zum Konsum psychotroper Substanzen beiErwachsenen in Deutschland 1997 [Population sur-vey on the consumption of psychoactive substancesin the German adult population], Sucht, 44 (Sonder-heft 1), S3–S82.

KRAUS, L., BAUERNFEIND, R. & BUHRINGER, G. (1998)Epidemiologie des Drogenkonsums in Deutschland.Ergebnisse aus Bevolkerungssurveys 1990 bis 1996 [Epi-demiology of Drug Consumption in Germany: resultsfrom population surveys 1990–1996)] (Baden–Baden,Nomos).

LEMMENS, P. H. H. M., TAN, E. S. & KNIBBE, R. H.(1988) Bias due to non-response in a Dutch survey


on alcohol consumption, British Journal of Addiction,83, 1069–1077.

MANGIONE, T. W., HINGSON, R. & BARRETT, J. (1982)Collecting sensitive data: a comparison of three sur-vey strategies, Sociological Methods and Research, 10,337–346.

MARHENKE, W. (1997) Telefonanschlussdaten alsAuswahlgrundlage [Sampling based on telephone di-rectories], in: GABLER, S., HOFFMEYER-ZLOTNIK,J.H.P. (Eds) Stichproben in der Umfragepraxis [ThePractice of Survey Sampling], pp. 207–220 (Opladen,Westdeutscher Verlag).

MAYFIELD, D., MCLEOD, G. & HALL, P. (1974) TheCAGE questionnaire: validation of a new alcoholismscreening instrument, American Journal of Psychiatry,131, 1121–1123.

REHM, J. & ARMINGER, G. (1996) Alcohol consump-tion in Switzerland 1987–93: adjusting for differen-tial effects of assessment techniques in the analysis oftrends, Addiction, 91, 1335–1344.

REHM, J. & SPUHLER, T. (1993) Measurement error inalcohol consumption: the Swiss Health Survey, Eu-ropean Journal of Clinical Nutrition, 47 (suppl. 2),25–30.

REHM, J. (1994) Reliabilitat und Stabilitat des Indika-tors fur Alkoholkonsum in der SchweizerischenGesundheitsbefragung [Reliability and Stability ofthe indicator for alcohol consumption in the SwissHealth Survey], Drogalkohol, 18, 3–8.

SAUNDERS, J. B., AASLAND, O. G., BABOR, T. F., DELA

FUENTE, J. R. & GRANT, M. (1993) Development ofthe Alcohol Use Disorders Identi� cation Test (AU-DIT): WHO collaborative project on early detectionof persons with harmful alcohol consumption, Addic-tion, 88, 791–804.

SCHWARZ, N., STRACK, F., HIPPLER, H.-J. & BISHOP, G.(1991) The impact of administration mode on re-sponse effects in survey measurement, Applied Cogni-tive Psychology, 5, 193–212.

SHAH, B., BARNWELL, B. & BIELER, G. (1997)SUDAAN User’s Manual, release 7.5 (Research Tri-angle Park NC, Research Triangle Institute).

STATISTISCHES BUNDESAMT (Ed.) (1997) StatistischesJahrbuch 1997 [Annual Statistics 1997] (Stuttgart,Metzler-Poeschel).

SUDMAN, S. & BRADBURN, N. (1974) Response Effects inSurveys: a review and synthesis (Chicago, Aldine).

SYKES, W. & COLLINS, M. (1988) Effects of mode ofinterview: experiments in the UK, in: GROVES, R.M.,BIEMER, P.P., LYBERG, L.E., MASSEY, J.T.,NICHOLLS II, W.L. & WAKSBERG, J. (Eds) TelephoneSurvey Methodology, pp. 301–320 (New York, JohnWiley and Sons, Inc.).

TARNAI, J. & DILLMAN, D. A. (1992) Questionnairecontext as a source of response differences in mailand telephone surveys, in: SCHWARZ, N. & SUDMAN,S. (Eds) Context Effects in Social and PsychologicalResearch, pp. 115–129 (New York, Springer).

measuring alcohol consumption and alcohol-related problems: comparison of responses from...

Documents