idea group publishing · 2020. 9. 29. · 46 international journal of e-collaboration, 2(2), 46-64,...

46 International Journal of e-Collaboration, 2(2), 46-64, April-June 2006

Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc.is prohibited.

Antecedents and Consequencesof User Satisfaction with

E-Mail SystemsKevin Dow, Kent State University, USA*

Alexander Serenko, Lakehead University, Canada

Ofir Turel, McMaster University, Canada

Jeffrey Wong, University of Nevada at Reno, USA

ABSTRACT

E-mail is an important component of the e-collaborative environment. This studycontributes to the literature by using the American Customer Satisfaction Index(ACSI) framework to model the antecedents and consequences of customersatisfaction with e-mail systems. We employ a survey to gather empirical evidenceto test the modified ACSI model for e-mail systems. Additionally, we test whetherspam has an influence on a user’s satisfaction with his or her e-mail system. Ourresults generally support the notion that the ACSI framework can be used to modele-mail user satisfaction, but we do not find statistically significant evidence thatspam affects overall user satisfaction with their e-mail system.

Keywords: American Customer Satisfaction Index (ACSI); electroniccollaboration; electronic mail; unsolicited (spam) e-mail; usersatisfaction

INTRODUCTIONAND BACKGROUND

E-mail is a form of computer-me-diated communication that facilitates im-portant aspects of e-collaboration. Itmay be conceptualized as a socio-tech-

nical system comprised of two compo-nents. The first component of e-mail in-cludes technology, software, user train-ing, industry standards, and practices.The second component consists of thesocial and psychological factors shap-

IDEA GROUP PUBLISHING

This paper appears in the publications, International Journal of e-Collaboration, Volume 2, Issue 2edited by Ned Kock © 2006, Idea Group Inc.

701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USATel: 717/533-8845; Fax 717/533-8661; URL-http://www.idea-group.com

ITJ3116


International Journal of e-Collaboration, 2(2), 46-64, April-June 2006 47

ing individual acceptance and usage(Lucas, 1998).

Information communication tech-nologies such as e-mail are the foundationfor collaborative efforts that allow geo-graphically dispersed individuals to workas a team (Kock, 2005). The pervasiveuse of e-mail provides evidence of its im-portance as a communication medium. Ithas been reported that in 2003, 31 billione-mail messages were sent daily worldwide with an average of 56 e-mails per e-mail address and 174 e-mails per person(Industry Canada, 2004). Over 600 Mil-lion individuals around world wide wereusing e-mail systems by the end of 2004(Radicati Group Inc., 2004). The phe-nomenal number of users was broughtabout partially by the declining costs ofcomputing, fees for long-distance commu-nication (Sproull & Kiesler, 1991), andadvances in computer and telecommuni-cations technologies.

The widespread use of e-mail alsohas introduced some problematic situa-tions. For example, extreme user over-load has resulted from the proliferation ofe-mail as a communications medium (Ire-land, 1997; Sherwood, 2002). At thesame time, the increasing number of un-solicited commercial e-mail messages(typically regarded as junk mail or spam)(Boykin & Roychowdhury, 2005; Buderi,2005; Erbland, 2005; Goodman,Heckerman, & Rounthwaite, 2005;Holmes, 2005; Pfleeger & Bloom, 2005;Van Duine, 2005; Whitworth &Whitworth, 2004) has exacerbated theinformation overload problem. Unwantedcommunication is often credited as being

the most critical component of e-mailoverload (Hinde, 2002).

Currently, the flood of unsolicitedmessages is growing significantly and maysoon account for half of all U.S. e-mailtraffic (Krim, 2003). An average e-mailuser receives 6 spam messages per day.Among the salient drivers for this growthof spam is that senders find it more ex-pensive to target their e-mail messages topotential customers rather than simply sendthe same message to large distribution lists(Gopal, Walter, & Tripathi, 2001). In ad-dition, there are insufficient and effectiveanti-spam regulations in place. At the mo-ment, only 26 countries have mandatedanti-spam laws, leaving the opportunity forlegal spamming activities in more than 100other countries.

Spam messages range from irritat-ing e-marketing offers to potentially dan-gerous messages with infected files(Anonymous, 2004). Consequences of thegrowing flood of unsolicited commercialmessages are that the usefulness of e-mailas a communication means may be im-pinged upon (Hall, 1998), productivity islost (Whitworth & Whitworth, 2004), andusers are harassed (Khoo & Senn, 2004).From the perspective of e-mail serviceproviders, the negative effects of spam mayhinder prospective future business. Themagnitude of e-mail communication rep-resents a potentially lucrative businessopportunity for e-mail service providers.For them, the volume of e-mails and sub-scribers can translate into advertising dol-lars and in some cases, user fees. Accord-ingly, it is believed that spam can have cru-cial effects on both users’ use of e-mail



and on e-mail service providers’ profit-ability. However, there is little empiricalacademic research on the impact of spamon the interaction of users with e-mail sys-tems. Our study seeks to contribute to theliterature by studying the antecedents andconsequences of user satisfaction with e-mail, and to explore the potential impactof spam on user satisfaction.

In the personal communicationsspace, Web-based e-mail services suchas Hotmail, Yahoo! and America Onlinelead the market. However, the emergenceof various service providers and the en-trance of established Internet firms to thee-mail messaging domain increase com-petition. Comcast and Google are ex-amples of companies with establishedInternet and search engine services whohave entered the market as e-mail pro-viders. This increased competition drivesthe importance of satisfaction and loyaltyin the e-mail space. While user satisfac-tion and the events surrounding it havebeen researched in the IS (e.g.,Abdinnour-Helm, Chaparro, & Farmer,2005; Armstrong, Fogarty, Dingsdag, &Dimbleby, 2005; Likourezos et al., 2004;Nelson, Todd, & Wixom, 2005; Wixom& Todd, 2005) and the marketing fields(e.g., Christianson, Parente, & Feldman,2004; Kohli, Devaraj, & Mahmood,2004; Leo & Philippe, 2002; Luna-Arocas, Tang, & Du, 2004; Oliver, 1997;Walsh, Wiedmann, & Groth, 2004), thereis a lack of empirical research that exam-ines the effect of e-mail specific phenom-ena, such as spam, on user satisfaction withe-mail as a communication system.

To survive in the competitive envi-ronment of technology services, it is es-sential to both attract and retain custom-ers. User satisfaction is a critical factor inthe determination of loyalty to a serviceprovider (Dawkins, 1990; Reichheld,2003). Customer turnover can be costlybecause of resources expended to replacecustomers lost, and the possibility of dam-age to an e-mail provider’s reputation. Onthe other hand, satisfied customers mayreduce the cost of attracting new subscrib-ers through word-of-mouth advertising.Findings from empirical studies suggest thatwhen customers perceive a firm is pro-viding superior products or services, thosefirms enjoy higher financial returns thanfirms with less satisfied customers (Ander-son & Fornell, 2000).

As such, our study contributes to theliterature by testing the impact of individu-als receiving spam (junk e-mail) on theirperception of customer satisfaction. Weemploy the American Customer Satisfac-tion Index (ACSI) model developed in the1990s (Fornell, Johnson, Anderson, Cha,& Bryant, 1996) in our study. The ACSIwas created by leading researchers in cus-tomer satisfaction, Claes Fornell and Eu-gene Anderson, to measure overall cus-tomer satisfaction in a way that can becompared between companies or betweenindustry segments. Customer satisfactionhas increasingly become important as com-panies have become service-oriented, andcustomers are faced with many alterna-tive providers from which to choose ser-vices and products.

The value of customer satisfactionmeasured by the ACSI is built on the



premise that without satisfied customers,economic prosperity is not possible. ACSIwas designed to measure customer satis-faction in a standardized way that wouldprovide insights into the consumereconomy for companies, industry tradeassociations, and government agencies.The developers of the ACSI felt that theirmethods would help to meet the need foran overall measure of the consumptionexperience. The index measures the qual-ity of goods and services as perceived bythose who consume them, and is designedprimarily to explain customer loyalty.

There are two principal conceptsunderlying the ACSI model. First, the con-structs of the model represent differenttypes of customer evaluations that can onlybe measured indirectly. This is why theACSI uses a multiple indicator approachthat measures customer satisfaction as alatent variable. Second, the ACSI is builton a series of cause and effect relation-ships that allows the antecedents and con-sequences of overall user satisfaction tobe examined. In the case of this paper,

we examine user satisfaction with e-mailsystems. Figure 1 illustrates the modifiedACSI model and relationships betweenvariables used in our study, including spam.The goal of this model is to present a setof causal relationships among several qual-ity-related constructs. The variables willbe further discussed in the next sectionwhere we develop the hypotheses testedin this paper.

The remainder of the paper is struc-tured as follows. The next section devel-ops the hypotheses we test. The subse-quent section outlines the research meth-odology and statistical results. The discus-sion, conclusions and directions for futureresearch are provided in the last section.

HYPOTHESES DEVELOPMENTThis study seeks to employ the ACSI

framework to generate a satisfaction scorefor e-mail services that may be comparedto those with other services. This score,coupled with a causal model, may depictthe perceptions, beliefs and behaviors ofe-mail users. In addition, the model is ex-

Figure 1. The E-Mail Services Customer Satisfaction Model (Adapted from theAmerican Customer Satisfaction Model by Anderson and Fornell, 2000)

PerceivedQuality

UnsolicitedEmails

UserSatisfaction

UserComplaints

UserLoyalty



pected to test whether there is an effect ofunsolicited e-mail on user satisfaction.

As seen in Figure 1, overall customersatisfaction has two antecedents and twoconsequences in the model proposed byFornell et al (1996). We have modifiedthe model for this study by omitting thevariables related to the perceived valueconstruct because the e-mail services aregenerally free for any given Internet ser-vice provider.

There are two primary researchquestions addressed in this paper:

1. What is the level of user satisfaction withelectronic mail measured by the Ameri-can Customer Satisfaction Index?

2. What is the relationship between spammessages and the degree of user satis-faction with electronic mail?

Satisfaction with e-mail is a users’reaction to their judgment of the state offulfillment (Oliver, 1997). Users have aset of perceptions regarding their e-mailbased on either previous experience orexternal influences. While using their e-mail system, these perceptions are com-pared with their actual experience and aset of beliefs regarding the extent to whichthe e-mail service has met their expecta-tions is developed. As a result, individuale-mail users adjust their perceptions ac-cordingly.

The American Customer SatisfactionIndex (ACSI) (Fornell, Johnson, Ander-son, Cha, & Bryant, 1996) has becomeone of the most frequently utilized satis-faction measures in the marketing litera-ture. The ACSI represents a customer-

centered quality measurement system toevaluate the performance of individualcompanies, industries, economic sectors,and national economies. The index is cal-culated quarterly for many industries, in-cluding information technology. The ACSIis available for online news and informa-tion services, portals, search engines,online retailers, and telecommunicationscompanies.

The contemporary literature presentsa variety of models that measure the de-gree of customer satisfaction with prod-ucts or services (c.f. Bansal, Irving, &Taylor, 2004; Parasuraman, Zeithaml, &Berry, 1988). In our study, the ACSI wasadapted because it demonstrates goodpsychometric capabilities and enables thegeneration of a standardized satisfactionscore, which is comparable across indus-tries and sectors.

According to this model, the degreeof perceived quality of an e-mail systempositively influences the level of user sat-isfaction with it. It is derived from the de-grees of personalization and reliability ofthe e-mail service. Further, the number ofunsolicited e-mail messages, which is fre-quently cited as the major e-mail dissatis-faction reason, is posited to have a directnegative effect on satisfaction. Accordingly,the related hypotheses we test are:

H1: The degree of perceived quality ofan e-mail application has a positiveassociation with the overall level ofuser satisfaction.

H2: The number of unsolicited spam mes-sages received by a user is negatively



associated with the overall level of usersatisfaction.

Prior research has found that satis-faction has a positive effect on both loy-alty and retention (Bolton, 1998; Gerpott,Rams, & Schindler, 2001) and a negativecausal effect on the number of customercomplaints (c.f. Anderson & Fornell,2000; Bolton, 1998; Kim, Park, & Jeong,2004; Price, Arnould, & Tierney, 1995).Marketing scholars have argued that inindustries where some switching barriersexist, it is more valuable to examine loy-alty than to measure retention (Reichheld,2003). In high-switching barrier industries(such as e-mail), customers that are dis-satisfied might still remain with their ser-vice provider. In these industries, disloyalcustomers that remain with a company dueto these high switching costs may ultimatelyhave a negative effect on new customeracquisition as they share their opinions andbeliefs with others (Reichheld, 2003).

In the case of e-mail systems, thereare at least two potential switching barri-ers. First, for Web-based systems, onemain barrier is the e-mail address (whichis not portable to other service provid-ers). In general, users find it inconvenientto regularly change their e-mail addressesbecause they have to locate all people andnotify them about the change. Second, inthe case of PC-based applications, themain barriers are the cost associated withthe purchasing of new software, for ex-ample, Microsoft Outlook and the setupefforts. It follows that the extent of usersatisfaction is negatively associated withan individual’s propensity to either formally

or informally complain about services. Atthe same time, user satisfaction with e-mailpositively influences the degree of loyaltyto a specific e-mail system. User loyalty isa favorable attitude towards a specific e-mail system that leads to choosing the samesystem given a need for a new e-mail ap-plication. Accordingly, the related hypoth-eses we test are:

H3: The overall level of user satisfactionwith an e-mail application is negativelyassociated with the frequency of usercomplaints.

H4: The overall level of user satisfactionwith an e-mail application has a posi-tive association with the degree of loy-alty to an e-mail system.

Following the ACSI framework, wetest the relationship between the endog-enous outcomes of overall satisfaction.There are no direct ways to test the effi-cacy of how an e-mail company deals withhandling customer complaints. If a cus-tomer becomes satisfied with the way theircomplaints were handled, this treatmentcould engender loyalty. However, if thecustomer is dissatisfied with the manner inwhich complaints are addressed, the ef-fect on loyalty would be expected to benegative. Therefore, we can only hypoth-esize that an association exists. Accord-ingly, the related hypothesis we test is:

H5: The frequency of user complaintsabout an e-mail application is associ-ated with the degree of loyalty to ane-mail system.



These five hypotheses can be rep-resented on the ACS model as depictedin Figure 2.

METHOD AND DATA ANALYSIS

Survey Instrument andConstruct Measures

A survey was used to sample sub-jects from a population of e-mail userslocated in Canada. All responses werekept anonymous; however some demo-graphic data was collected to determinethe nature of the sample. The survey wasadministered to 200 undergraduate andgraduate students of Canadian universi-ties. The completion of the questionnairewas voluntarily, and no rewards or incen-tives were offered to the subjects forcompletion of the survey. Overall, 200questionnaires were administered and 186questionnaires were returned (93% of thesubjects agreed to participate in the study).Eight instruments were either incompleteor partially complete and were excludedfrom data analysis. The final sample con-sisted of 178 usable responses (89% us-able response rate).

The measures used in this study wereadopted from previously validated instru-ments (Fornell, Johnson, Anderson, Cha,& Bryant, 1996) and also from existingliterature to create measures for the fiveconstructs used in this study to minimizethe potential for measurement error. Whileit is acknowledged that all latent measuresare imperfect, using existing and previouslyvalidated measures provides more of aconsensus for appropriate representationof the constructs detailed in this paper. TheAppendix contains a listing of the con-structs used in this study and their corre-sponding measures.

Perceived quality (of an e-mail sys-tem) is simply an individual’s judgmentabout the e-mail systems excellence orsuperiority and it relates to the consumersperception of quality, based upon theirprior expectations about how well theproduct fits personal requirements andwhether expectations about reliability aremet. Satisfaction is a measure of anindividual’s experience based on theirevaluation of the e-mail system and canbe thought of as the overall evaluation ofthe total consumption experience. Cus-

Figure 2. The study model with hypotheses

PerceivedQuality

UnsolicitedEmails

UserSatisfaction

UserComplaints

UserLoyalty

H1

H2

H3

H4

H5



tomer Loyalty expresses an individual’sintended behavior related to the e-mailsystem and it is operationalized with asingle item used to reflect an individual’slikelihood to remain loyal to the e-mailsystem. Customer Complaints can bethought of voicing dissatisfaction and ismeasured with a single dichotomous itemto reflect whether or not individual usershave complained about their e-mail sys-tem. If subjects have complained, then theyindicated to what extent have they havedone so. Unsolicited e-mail is a categori-cal representation of the number of unso-licited e-mails received by the user on adaily basis. This is operationalized with a7-point categorical scale reflecting therange of unsolicited e-mails received daily.

In addition to questions pertaining tothe model, subjects were asked a set oflimited demographic questions and on theire-mail usage patterns. As such, respon-dents indicated their age, sex, name andtype of the most frequently utilized e-mailsystem. In addition, actual e-mail usagedetails, such as the number of messagessent or received daily (excluding spam)and time spent working with an e-mailapplication were reported. The develop-ment of the research instrument followedan iterative process to ensure face validityof the questionnaire. Specifically, a smallgroup of information technology practitio-ners and academics was consulted andasked whether the items proposed in theinstrument adequately measured the de-sired constructs. As a result of their feed-back, several minor modifications to thequestionnaire were made.

Descriptive StatisticsThe subjects surveyed in this study

represented a diverse sample. Their agesranged from 18 to 50 with 51% of thesubjects are male while 49% are female.A majority of respondents indicated thatHotmail was the most frequently utilizede-mail system, followed by MS Outlookand Yahoo! Only a few people preferredcountry or university-specific e-mail envi-ronments, for example, Univmail (a pro-prietary e-mail system that is administeredby a large University) or Sina.com (a pre-dominantly Chinese portal). On average,respondents utilized this system for fiveyears ranging from three months to 14years. Overall, 83% of these e-mail envi-ronments are accessed through a Web-based interface. Table 1 outlines descrip-tive statistics for each of the items used inthe survey. Table 2 outlines the Pearsoncorrelations among the variables includedin this study.

It should be noted that the correla-tion between Perceived Quality and Sat-isfaction is relatively high. We believe,however, that this does not threaten thevalidity of the model for several reasons.First, the loadings of these items on theconstructs to which they belong are higherthan their cross-loadings. Second, thesetwo constructs represent an independentand a dependent variable that are expectedto be correlated. As argued by Straub,Boudreau, and Gefen (2004, p. 25),“loadings across what are traditionallyknown as independent and dependentvariables are not relevant to the issue ofconstruct validity and such tests may/should be avoided in PCA [principle com-



ponent analysis].” Third, other studies thatutilized or adapted the American CustomerSatisfaction Index framework also reporthigh correlations between quality and sat-isfaction. All of these studies argue thatthere is a reasonable confidence in the dis-criminant validity, and treat quality andsatisfaction as distinct. For example, re-searchers have consistently uncovered ahigh correlation between quality and sat-isfaction (Babakus, Bienstock, & Scotter,2004; Choi, Lee, Lee, & Kim, 2004;O’Loughlin & Coenders, 2004). Overall,it should be noted that even though someof the correlations in our model are fairlyhigh, they are still within the norm, and as

such are not inconsistent with the discrimi-nant validity of our constructs.

The American CustomerSatisfaction Index

To help better understand the results,knowing how the ACSI is calculated ishelpful. The ACSI index number was cal-culated for e-mail systems (both in theaggregate and for each individual e-mailprovider) based on the formula suggestedby Anderson and Fornell (2000):

3 3

1 13

1

1009

i i ii i

ii

w x wACSI

w= =

=

× -= ·

×å å

å(1)

Table 1. Descriptive statistics

Item N Minimum Maximum MeanStd.Deviation Skewness Kurtosis

QUALITY 1 178 2 10 7.06 1.31 -0.83 1.47QUALITY 2 178 3 10 7.38 1.41 -0.52 0.05QUALITY 3 178 2 10 7.25 1.60 -0.56 0.30UNSOLICITEDE-MAILS 177 0 7 2.29 1.61 1.06 0.28ACSI 1 178 2 10 7.28 1.43 -0.52 0.85ACSI 2 177 3 10 6.59 1.39 -0.43 -0.13ACSI 3 178 3 10 6.72 1.51 -0.43 -0.13LOYALTY 176 1 10 6.94 1.96 -0.60 0.13COMPLAINTS 175 0 3 0.22 0.46 2.24 6.91

Table 2. Correlation matrix of variables in model

PerceivedQuality

UnsolicitedE-Mails ACSI

CustomerComplaints Loyalty

Perceived Quality 1.000UnsolicitedE-Mails -0.114 1.000

ACSI 0.828 -0.079 1.000

Customer Complaints -0.105 0.203 -0.187 1.000Loyalty 0.531 0.021 0.573 -0.085 1.000



Where, wi represents the weight ofthe ith item obtained from the outer modelgenerated by PLS, and ix represents theaverage of the ith item that loads on theACSI construct.

Partial Least Squares AnalysisPLS is a regression-based technique,

with roots in principal-components analysisthat can estimate and test the relationshipsamong latent constructs. PLS producesfactor loadings between the individualitems and latent constructs as well as esti-mates standardized coefficients (i.e., betacoefficients) for the paths between con-structs. One benefit of PLS is that thereare fewer demands on the data. Specifi-cally, data do not have to be normally dis-tributed, the scales may be ordinal or nomi-nal, and the sample size can be small (Chin,1998).

Item Reliability. Item reliability in-dicates whether the indicators for a par-ticular latent variable measure that latentvariable only. Guidelines provided by Hairet al. (1995) were used in determining the

item reliability for each latent variable.Following their suggestion, only items withloading greater than or equal to 0.50 wereretained. As can be seen in Table 3, eachof the factor loadings are above the mini-mum threshold of 0.50. As such, all indi-vidual items were retained for the finalstructural model.

Convergent Validity. Constructvalidity indicates the degree to which alatent construct is representative of the trueconstruct. It is often measured usingCronbach’s alpha. A popular rule ofthumb is that 0.70 (Fornell & Larcker,1981). All values for the Cronbach’s al-phas exceeded this rule of thumb. Giventhis, the convergent validity criteria hasbeen satisfied.

Discriminant Validity. Discrimi-nant validity represents the extent to whichmeasures of a given latent construct differfrom measures of other latent constructs inthe same model. Essentially, a latent con-struct should share more variance with itsindicators than it shares with other latent

Table 3. Loadings and cross-loadings for each item and construct

PerceivedQuality ACSI

UnsolicitedE-Mail Complaints Loyalty

PQ1 0.863 0.702 -0.073 -0.132 0.402PQ2 0.888 0.735 -0.069 -0.099 0.491PQ3 0.841 0.708 -0.129 -0.042 0.484ACSI1 0.818 0.864 -0.094 -0.160 0.543ACSI2 0.547 0.778 -0.101 -0.156 0.345ACSI3 0.698 0.893 -0.037 -0.163 0.541JUNK -0.104 -0.089 1.000 0.185 0.062UC -0.105 -0.187 0.185 1.000 -0.085UL 0.531 0.537 0.062 -0.085 1.000



constructs. To assess discriminant validity,Fornell and Larker (1981) suggest the useof Average Variance Extracted — simplythe average variance shared between aconstruct and its measures. The averagevariance extracted is obtained by the sumof the loading squared, divided by the num-ber of items in the construct, whereas thevariance shared between two constructscorresponds to the square of the coeffi-cient of correlation between the latter. Thismeasure should be greater than the vari-ance (squared correlation) shared betweenthe latent construct and other latent con-structs in the model. Each construct passedthe test for discriminant validity. The load-ings and cross-loadings for each constructin the study can be found in Table 3.

RESULTS

Partial Least Squares ModelWe employed the PLS Graph soft-

ware package (Chin, 2001) to evaluate

the structural paths in our model. Statisti-cally significant levels of the estimated pathcoefficients were determined using thebootstrap procedure. The t-values weobtain are estimates of the bootstrap pathcoefficient divided by the standard error.

Figure 3 and Table 4 show the re-sults of PLS analysis. The significant pathsindicate that three of the five hypothesesare supported. Specifically, as hypoth-esized, the path coefficient from perceivedquality to user satisfaction (0.829, p <0.01) support H1, that the degree of per-ceived quality of an e-mail application ispositively associated with user satisfaction.The results however, do no support H2.This suggests that spam does not have astrong effect on user satisfaction. The pathcoefficient from user satisfaction to usercomplaints (-0.187, p < 0.05) support H3,that user satisfaction is negatively associ-ated with complaints. H4, user satisfac-tion is positively associated with the de-gree of loyalty, is supported by the results

PerceivedQuality

UnsolicitedEmails

UserSatisfaction

UserComplaints

UserLoyalty

0.829**

.015

-0.187*

.578**

.024

R2 = 0.685

R2 = 0.329

R2 = 0.035

Figure 3. Results of hypotheses testing

* Significant at p < 0.05** Significant at p < 0.01



(0.578, p < 0.01). Finally, the path coef-ficient from user complaints to user loy-alty was not statistically significant anddoes not support H5. In summary, mostof the hypothesized linkages in the ACSImodel were supported by our findings;however, spam did not appear to affectuser satisfaction as predicted by H2.

American CustomerSatisfaction Index Results

The mean ACSI scores calculatedfor each e-mail service indicated is listedin Table 5. The satisfaction rating acrossall of the e-mail providers was fairly con-sistent, and t-tests show that the meanscores are not significantly different fromeach other. An average ACSI score forall of the e-mail services was calculatedto be 65.5, as seen in Table 6. This tableis presented to get a sense of ACSI scoresfor other industries North America for Q4,2003. Table 6 outlines the specific com-parison of e-mail systems. As can be seen,the satisfaction score for e-mail serviceranks just below that of the airline indus-try, and just above that of the publishing/news papers industry.

DISCUSSIONOur findings largely support the re-

lationships between the elements of the

ACSI model, suggesting that the ACSIindex is a feasible measurement of cus-tomer satisfaction related to e-mail usage.The degree of perceived quality of an e-mail application appears to strongly influ-ence an individual’s level of satisfactionwith this e-mail system. A more satisfiede-mail user complains less about their e-mail experience and demonstrates a highdegree of loyalty to a particular e-mailsystem.

An interesting finding of our study isthat unsolicited e-mail making it throughthe many spam filters of e-mail providerswas not a significant determinant of cus-tomer satisfaction. This finding is surpris-ing because the literature (Anonymous,2004) seems to indicate that spam is areal problem to e-mail users. We suspectthat, in general, issues that are beyond thecontrol of the technology do not seem tonegatively impact the level of satisfactionwith this e-collaborative technology. Assuch, users appear to be coping with thesedownside issues better than the expertswould have predicted.

Alternatively, this finding may be par-tially explained by the motivational pre-mises of attribution theory. This theoryexplains how people make causal expla-nations about events and describes theprocesses and behavioral outcomes of

Table 4. Table of hypothesis testing

Hypothesis Estimate T-Value P-Value Supported?H1: PQà ACSI 0.829 30.682



those rationalizations (Heider, 1958; Jones& Davis, 1965; Kelly, 1972; Weiner etal., 1972). According to attribution theory,people tend to give credit to themselvesfor events with positive outcomes andblame the external environment for eventswith a negative outcome.

When a spam e-mail message ar-rives, a person spends some time to de-termine the relevance of this message andto delete it. This is a negative outcome situ-ation that e-mail users may blame on ex-ternal factors, such as the quality of an e-mail system, a spammer, the lawmakersdelaying anti-spam legislation, and so forth.Research in the area of human-computerinteraction (HCI) suggests that people tendnot to hold software responsible for mis-takes, problems, or bugs because users

expect computer applications to be gen-erally unreliable (Lieberman, Rosenzweig,& Singh, 2001). Therefore, it is plausiblethat a recipient of spam attributes it to fac-tors not associated with their e-mail ap-plication. For example, a spammer is con-sidered directly responsible for the spam,not the e-mail service provider whosespam filters did not detect and remove thespam.

On the surface, the ACSI index cal-culated for e-mail services appears to berelatively low compared with those of re-lated products and services in other in-dustries in North America. However, theproper way to use the ACSI score maybe to compare scores from one period tothe next instead of in cross-section amongother goods and services. Alternatively,

Table 5. The ACSI for individual e-mail systems1

E-Mail System n = ACSIMicrosoft Hotmail 113 65.1Microsoft Outlook 25 65.7Yahoo 21 67.3Others 18 66.8

Table 6. The ACSI for select industries in North America

Sector ACSITelephone-Long Distance 82E-Commerce 80.8Telephone-Local 79Retail 75Finance/ Insurance 74.7Fixed-Wire Telephone Services 72Scheduled Airlines 67E-Mail Systems 65.5Publishing/ News Papers 64Cable & Satellite TV 61United States Postal Service 60



comparisons to similar service are moremeaningful than industry or sector cross-sectional comparisons. For example, the65.5 e-mail ACSI score could be com-pared with the score of 61 for Cable &Satellite TV.

Finally, we found no evidence of arelationship between a user’s tendency tocomplain about an e-mail system and theirlevel of loyalty to the particular e-mail ser-vice. This may be due to high switchingbarriers that prevent people from movingfrom one e-mail system to another. Eventhough an e-mail service might be free ofcharge, the change of an e-mail address,time to install a new application, and soft-ware acquisition costs create high switch-ing barriers for e-mail users. Therefore,even though people complain about theire-mail experience, they tend to stay withtheir current e-mail system. In addition,there are few substitutes for e-mail.

CONCLUSIONE-mail is increasingly important as a

modality in the e-collaborative environ-ment, and customer satisfaction has sig-nificant economic implications for e-mailproviders. Satisfaction with e-mail systemsis an important topic because of the sig-nificant role that e-mail plays in communi-cation and e-collaboration. Our studyemploys the ACSI customer satisfactionmodel to offer insights into the anteced-ents and consequences of user satisfac-tion with e-mail systems.

We find evidence that perceivedquality is a determinant of user satisfac-tion. In turn, user satisfaction is positivelyrelated to user loyalty, and negatively as-

sociated with user complaints. Our find-ings generally support the ACSI model ofcustomer satisfaction. Surprisingly, unso-licited e-mail was not a statistically signifi-cant determinant of user satisfaction, whichwas an unanticipated finding based uponthe literature discussing the potential ef-fect of spam on e-mail users. Addition-ally, user complaints did not appear to havean effect on user loyalty, as some of theliterature on customer loyalty might sug-gest. The findings of this paper may be ofvalue not only to e-mail users, e-mail pro-viders, but also to researchers interestedin other e-collaborative technologies.

Our study has its limitations. Thesample consisted mostly of young Cana-dian students which may not be generaliz-able to the entire e-mail user population.However, we feel that this population hasstrong familiarity with and dependence one-mail, making them typical of regular e-mail users (Fallows, 2002). Ideally, itwould have been preferable to survey alarger sample that draws from a cross-section of individuals that includes peoplefrom business and governmental sectors,of heterogeneous geographical dispersionusing a wider variety of e-mail systems.Future research may employ a broadercross section of e-mail users to modify,validate, or expand on our findings.

REFERENCESAbdinnour-Helm, S. F., Chaparro, B. S.,

& Farmer, S. M. (2005). Using the end-user computing satisfaction (EUCS)instrument to measure satisfaction witha Web site. Decision Sciences, 36(2),341-364.



Anderson, E. W., & Fornell, C. (2000).Foundations of the American CustomerSatisfaction Index. Total Quality Man-agement & Business Excellence,11(7), 869-882.

Anonymous. (2004). Bugs in the spam.Communications News, 41(6), 8-8.

Armstrong, B., Fogarty, G., Dingsdag, D.,& Dimbleby, J. (2005). Validation of acomputer user satisfaction questionnaireto measure IS success in small busi-ness. Journal of Research and Prac-tice in Information Technology,37(1), 27-42.

Babakus, E., Bienstock, C. C., & Scotter,J. R. V. (2004). Linking perceived qual-ity and customer satisfaction to storetraffic and revenue growth. DecisionSciences, 35(4), 713-737.

Bansal, H. S., Irving, P. G., & Taylor, S.F. (2004). A three-component modelof customer commitment to service pro-viders. Journal of the Academy ofMarketing Science, 32(3), 234-250.

Bolton, R. N. (1998). A dynamic modelof the duration of the customer’s rela-tionship with a continuous service pro-vider: The role of satisfaction. Market-ing Science, 17(1), 45-65.

Boykin, P. O., & Roychowdhury, V. P.(2005). Leveraging social networks tofight spam. Computer, 38(4), 61-68.

Buderi, R. (2005). News analysis -Microsoft declares war on spam. Tech-nology Review, 108(2), 22-23.

Chin, W. W. (1998). The Partial LeastSquares approach for Structural Equa-tion Modeling. In G. A. Marcoulides(Ed.), Modern methods for businessresearch (pp. 295-336). Mahwah, NJ:

Lawrence Erlbaum Associates.Chin, W. W. (2001). PLS-Graph User’s

Guide, Version 3.0. Houston, TX: SoftModeling Inc.

Choi, K. S., Lee, W. H., Lee, H., & Kim,C. (2004). The relationships amongquality, value, satisfaction and behav-ioral intention in health care providedchoice: A South Korean study. Jour-nal of Business Research, 57(8), 913-921.

Christianson, J. B., Parente, S. T., &Feldman, R. (2004). Consumer expe-riences in a consumer-driven healthplan. Health Services Research, 39(4),1123-1139.

Dawkins, P. M., Reichheld, F. F. (1990,Summer). Customer retention as acompetitive weapon. Directors &Board, 14, 42-47.

Erbland, J. (2005). Spam on the increase.IEEE Software, 22(2), 9-10.

Fallows, D. (2002). Email at work: Fewfeel overwhelmed and most arepleased with the way email helpsthem do their jobs. Unpublished manu-script, Washington, DC.

Fornell, C., Johnson, M. D., Anderson,E. W., Cha, J., & Bryant, B. E. (1996).The American Customer SatisfactionIndex: Nature, purpose, and findings.Journal of Marketing, 60(7), 7-18.

Fornell, C., & Larcker, D. F. (1981).Evaluating structural equation modelswith unobservable variables and mea-surement error. Journal of MarketingResearch, 18(1), 39-50.

Gerpott, T. J., Rams, W., & Schindler, A.(2001). Customer retention, loyalty,and satisfaction in the German mobile



cellular telecommunications market.Telecommunications Policy, 25(4),249-269.

Goodman, J. , Heckerman, D., &Rounthwaite, R. (2005). Stoppingspam. Scientific American, 292(4),42-49.

Gopal, R. D., Walter, Z., & Tripathi, A.K. (2001). Admediation: New horizonsin effective email advertising. Commu-nications of the ACM, 44(12), 91-96.

Hair, J. F., Jr., Anderson, R. E., Tatham,R. L., & Black, W. C. (1995). Multi-variate data analysis with readings(4th ed.). Englewood Cliffs, NJ:Prentice Hall.

Hall, R. J. (1998). How to avoid unwantedemail. Communications of the ACM,41(3), 88-95.

Heider, F. (1958). The psychology ofinterpersonal relations. New York:John Wiley & Sons.

Hinde, S. (2002). Spam, scams, chains,hoaxes and other junk mail. Comput-ers & Security, 21(7), 592-606.

Holmes, N. (2005). In defense of Spam.Computer, 38(4), 86-88.

Industry Canada. (2004). The digitaleconomy in Canada: Unsolicitedcommercial electronic mail (SPAM).Retrieved July 30, 2004, from http://e-com.ic.gc.ca/epic/internet/inecic-ceac.nsf/en/h_gv00170e.html#stats

Ireland, P. S. (1997). A management para-digm to reduce information overload-the introduction of the intranet into BT.In IEE Colloquium on IT Strategiesfor Information Overload, (DigestNo: 1997/340), (pp. 1-6).

Jones, E. E., & Davis, K. (1965). From

acts to dispositions: The attribution pro-cess in person perception. In L.Berkowitz (Ed.), Advances in experi-mental social psychology (Vol. 2, pp.219-266). New York: Academic Press.

Kelly, H. H. (1972). Attribution in socialinteraction. In E. E. Jones, D. E.Kanouse, H. H. Kelly, R. E. Nisbett,S. Valins & B. Weiner (Eds.), Attribu-tion: Perceiving the causes of behav-ior (pp. 1-27). Morristown, NJ: Gen-eral Learning Press.

Khoo, P. N., & Senn, C. Y. (2004). Notwanted in the inbox! Evaluations of un-solicited and harassing e-mail. Psychol-ogy of Women Quarterly, 28(3), 204-214.

Kim, M. K., Park, M. C., & Jeong, D.H. (2004). The effects of customer sat-isfaction and switching barrier on cus-tomer loyalty in Korean mobile tele-communication services. Telecommu-nication Policy, 28(2), 145-159.

Kock, N. (2005). What is e-collabora-tion. International Journal of e-Col-laboration, 1(1), i-vii.

Kohli, R., Devaraj, S., & Mahmood, M.A. (2004). Understanding determinantsof online consumer satisfaction: A de-cision process perspective. Journal ofManagement Information Systems,21(1), 115-135.

Krim, J. (2003, March 13). Spam’s costto business escalates: Bulk e-mailthreatens communication arteries. TheWashington Post, p. A01.

Leo, P. Y., & Philippe, J. (2002). Retailcentres: Location and consumer’s sat-isfaction. Service Industries Journal,22(1), 122-146.



Lieberman, H., Rosenzweig, E., & Singh,P. (2001). Aria: An agent for annotat-ing and retrieving images. IEEE Com-puter, 34(7), 57-62.

Likourezos, A., Chalfin, D. B., Murphy,D. G., Sommer, B., Darcy, K., &Davidson, S. J. (2004). Physician andnurse satisfaction with an electronicmedical system. Journal of Emer-gency Medicine, 27(4), 419-424.

Lucas, W. (1998). Effects of e-mail onthe organization. European Manage-ment Journal, 16(1), 18-30.

Luna-Arocas, R., Tang, T. L. P., & Du,L. Z. (2004). Consumers’ motivationprofiles at a sports center in Spain: Themotivation in sports scale and consumersatisfaction. International Journal ofPsychology, 39(5-6), 592-592.

Nelson, R. R., Todd, P. A., & Wixom, B.H. (2005). Antecedents of informationand system quality: An empirical exami-nation within the context of data ware-housing. Journal of Management In-formation Systems, 21(4), 199-235.

O’Loughlin, C., & Coenders, G. (2004).Estimation of the European customersatisfaction index: Maximum likelihoodversus partial least squares: Applicationto postal services. Total Quality Man-agement & Business Excellence,15(9-10), 1231-1255.

Oliver, R. L. (1997). Satisfaction: Abehavioural perspective on con-sumer. Boston: Irwin McGraw-Hill.

Parasuraman, A., Zeithaml, V. A., &Berry, L. L. (1988). Servqual — A mul-tiple-item scale for measuring consumerperceptions of service quality. Journal

of Retailing, 64(1), 12-40.Pfleeger, S. L., & Bloom, G. (2005). Can-

ning spam: Proposed solutions to un-wanted email. IEEE Security & Pri-vacy, 3(2), 40-47.

Price, L. L., Arnould, E. J., & Tierney, P.(1995). Going to extremes: Managingservice encounters and assessing pro-vider performance. Journal of Mar-keting, 59(2), 83-97.

Radicati Group Inc. (2004). Market num-bers summary update. Palo Alto, CA.

Reichheld, F. F. (2003). The one numberyou need to grow. Harvard BusinessReview, 81(12), 46-54.

Sherwood, K. D. (2002). Overcomeemail overload with Microsoft Out-look 2000 and Outlook 2002: Getthrough your electronic mail faster.Palo Alto, CA: World Wide WebfootPress.

Sproull, L., & Kiesler, S. B. (1991). Con-nections: New ways of working in thenetworked organization. Cambridge,Mass: The MIT Press.

Straub, D., Boudreau, M.-C., & Gefen,D. (2004). Validation guidelines for ISpositivist research. Communicationsof the Association for InformationSystems, 13, 1-80.

Van Duine, D. S. (2005). Spam on theincrease. IEEE Software, 22(2), 9-9.

Walsh, G., Wiedmann, K. P., & Groth,M. (2004). Examining consumer be-havior in the liberalized German energymarket-the influence of customer sat-isfaction on customer willingness toswitch public utility companies. In Ad-vances in Consumer Research, Vol-



ume XXXI (pp. 373-374).Weiner, B., Frieze, I., Kukla, A., Reed,

L., Rest, S., & Rosenbaum, R. M.(1972). Perceiving the causes of suc-cess and failure. In E. E. Jones, D. E.Kanouse, H. H. Kelly, R. E. Nisbett,S. Valins & B. Weiner (Eds.), Attribu-tion: Perceiving the causes of behav-ior (pp. 95-120). Morristown, NJ:General Learning Press.

Whitworth, B., & Whitworth, E. (2004).Spam and the social-technical gap.Computer, 37(10), 38-45.

Wixom, B. H., & Todd, P. A. (2005). A

theoretical integration of user satisfac-tion and technology acceptance. Infor-mation Systems Research, 16(1), 85-102.

ENDNOTES* The order of authorship is presented

alphabetically. All authors contributedequally to this research.

1 A ttest was performed to ascertain thestatistical difference in the ACSI be-tween the e-mail systems. The resultsof this test indicated that there is no dif-ference in ACSI across e-mail systems.

APPENDIX

Constructs and Their Measures

Construct ItemWhat is your overall evaluation of the quality of this e-mail system?Anchored by Very Low………Very HighWhat is your evaluation of the extent to which this e-mail system meets your personalrequirements?Anchored by Very Low………Very High

Quality

What is your evaluation of the extent to which this e-mail system is reliable?Anchored by Very Low………Very HighOverall, how satisfied are you with this e-mail system (all things considered)?Anchored by Very Dissatisfied………Very SatisfiedTo what extent has this e-mail system fallen short or exceeded your expectations of agood communications medium?Anchored by Falls Very Short………Exceeded by Far

Satisfaction

How close are the services offered by this e-mail system to your ideal e-mail system?Anchored by Very Far From Ideal………Very Close to Ideal

LoyaltyIf you required a new e-mail system, how likely is it that you would choose yourcurrent e-mail system?Anchored by Very Unlikely………Very LikelyHave you ever complained (either formally or informally) about this e-mail system tothe service provider or other people?Anchored by Yes or NoCustomer ComplaintsIf yes, indicate the number of times that you have complained.A numeric response

Unsolicited E-Mails The average number of junk e-mails you receive per day by using this system is



Kevin E. Dow is an assistant professor of accounting in the College of Business,Kent State University. He earned a BS in electrical engineering, an MBA, and aPhD in business administration. Dr. Dow’s research interests focus on businessvalue of information technology and the interaction of information technologyand managerial accounting. His work has appeared in Information SystemsResearch, Data Base for Advances in Information Systems, the International Journalof Accounting Information Systems, and others.

Alexander Serenko is an assistant professor of management information systemsin the Faculty of Business Administration, Lakehead University, Canada. Heholds an MSc in computer science, an MBA in electronic business, and a PhD inmanagement science/systems. Dr. Serenko’s research interests pertain to usertechnology adoption, knowledge management, and innovation. Dr. Serenko’sarticles appeared in various refereed journals, and his papers have receivedawards at Canadian and international conferences.

Ofir Turel is a PhD candidate at the DeGroote School of Business, McMasterUniversity. He holds a BSc in industrial engineering and an MBA in technologymanagement. Before joining the DeGroote School of Business, he has held seniorpositions in the information technology and telecommunications industries. Hisresearch interests include human computer interaction, online collaboration, andmobile services. His work has been published in several peer-reviewed journalsand presented at various international conferences.

Professor Wong received his PhD from the University of Oregon in 1999 and is acertified public accountant. His research interests focus on understanding howthe actions of a firm ultimately map to its financial results, and the interactionbetween managerial accounting and information systems technology. ProfessorWong’s research has been presented at national and international conferencesand has been published or is forthcoming in journals such as Database, BehavioralResearch in Accounting, and Internal Auditing.

idea group publishing · 2020. 9. 29. · 46 international journal of e-collaboration, 2(2), 46-64,...

Documents