1520-9202/03/$17.00 © 2003 IEEE September ❘ October 2003 IT Pro 53

Web Personalization:Is it Effective?Kar Yan Tam and Shuk Ying Ho

T he Web has become amajor communicationchannel between anorganization and its

stakeholders. Recently, morefirms have migrated their adver-tising, ordering, customer sup-port, and information dissem-ination to the Web. Web-basedservices expand the reach offirms and collect a vast amountof information during customerinteractions. Firms can now col-lect transaction histories andclick streams (the trail of mouseclicks that users make when per-forming computer operations);they can also download statisticsfrom online customers. Usefulcues to a customer’s traits andprofiles are likely embedded inthis data, providing informationcrucial to designing customer-centric Web sites (T.Koivumäki,“Customer Satisfaction andPurchasing Behavior in a Web-based Shopping Environment,”Electronic Markets,Fall 2001,pp.186-192).

By knowing their customers’needs better, firms can offerproducts and services at theright price, in the right context,and at the right time. Web per-sonalization enables this sort ofinteraction (L. Ardissono andcolleagues, “Personalization in

Business-To-Customer Inter-action,” Comm. ACM, May2002, pp. 52-53). Firms canimplement personalization byintegrating independent pack-ages such as data mining andcollaborative filtering tools withthe Web server; or personaliza-tion could take the form of apackage solution.

In general, Web personaliza-tion serves three main objec-tives: It

• draws attention to a companyand its products and services;

• implants messages; and • attempts to persuade.

Companies engage in fiercecompetition for attention on theInternet. They use banners,dynamic contents (which arecontents dynamically generatedby the Web server to meet indi-vidual users’ needs), and ani-mation (which we will call mes-sages) on the Web to serve thesethree objectives (T.H. Daven-port and J.C. Beck, The Atten-tion Economy: Understandingthe New Currency of Business,Harvard Business School Press,2001).

These messages have long-and short-term effects. Adver-tising messages irrelevant to thespecific Web page’s contextbecome implanted in the user’slong-term memory to facilitaterecall in future encounters withproducts and services related tothe original message. Althoughthese messages might not imme-

This study showsthat subjectswho receive personalizedinformation downloadedmusic files morefrequently.

diately impact a user’s decisions,they subconsciously becomeencoded in the user’s memoryand serve as memory cues to thefirm’s offerings.

On the other hand,persuasivemessages are context sensitiveand companies using them in-tend to change the user’s behav-ior at the point of encounter(P.E. Pedersen, “Behavioral Ef-fects of Using Software Agentsfor Product and Merchant Bro-kering: An Experimental Studyof Consumer Decision-Making,”

Study Methodology

Inside

54 IT Pro September ❘ October 2003

P E R S P E C T I V E S

Int’l J.Electronic Commerce,Fall 2000,pp. 125-141).

Let’s look at how companies try tochange user behavior using thesemessages. A Web personalizationagent, for example, can offer dis-counts on complementary productsby checking the user’s shopping cartduring an online shopping session.Alternatively, a persuasive messagecould make personalized offerings tousers. The personalized book recom-mendation on Amazon’s site (http://www.amazon.com) is an example.Theobjective is to persuade a user to con-sider complementary products thatmaximize cross- and up-sell opportu-nities. (Cross-selling is to persuade acustomer to consider buying a com-plementary product, such as sellingcar insurance after a car sale; up-sell-ing is to persuade a customer to buya more expensive or upgraded line ofproduct.)

An intelligent Web personalizationagent should be able to decide whichobjective to pursue in real time. Inparticular, at any point in time, theagent must determine

• message content,• Web layout, and• the timing and frequency of mes-

sages appropriate to achieving the

desired objective (D. Billsus andcolleagues,“Adaptive Interfaces forUbiquitous Web Access,” Comm.ACM, May 2002, pp. 34-38).

The schematic diagram in Figure 1shows the interaction among usersand Web personalization.

Companies looking for Web per-sonalization tools and software havehigh expectations, so the stakes arehigh for vendors offering such solu-tions. However, little empirical evi-dence exists to support theeffectiveness of today’s Web person-alization.We attempt to close this gapby conducting an empirical study toassess the effectiveness of persuasivemessages on user behavior and satis-faction. In particular, we address thefollowing question: Will users act onrecommendations made by a person-alization agent? If this is the case, weexpect three outcomes.First,users willexpend less effort in considering alter-native products and services. Second,the firm will gain more transactions.Third,users will express a higher levelof satisfaction with the recommendeditems than those they select withoutpersonalization support.

To address the question and out-comes, we studied a Web site devotedto personalized music downloads.

PERSONALIZED MUSICDOWNLOADING IN A CLOSED COMMUNITY

In 2002, a company set up an onlinemusic site running on an Apache Webserver. It was a music-downloadingsite for a closed community; that is,one not available to the public. Thecompany connected the Web serverwith an Oracle database server. Themusic site used Obert 1.2 by Radica(http://www.radicasys.com) as a Webpersonalization agent. This productprovided functions common in manycommercially available personaliza-tion solutions, such as creating andmanaging a user’s personal profile;analyzing a user’s dynamic navigationrecords, such as for browsing, filedownload, and product rating; andidentifying users’ unique preferences.Also, Obert 1.2 was capable of adjust-ing to any real-time changes in auser’s downloading behavior.

In our study, we used Obert 1.2 toelicit users’ preferences for differentartists or categories of music (such asclassical, pop, or jazz). We coded per-sonalized information in a set of rulesthat drove the delivery of personalizedmessages to the Web site’s users. Themusic site had an easy-to-use interfaceso that users could sample all songsand albums at any time. The serverhosted 1,184 songs in various lan-guages and rhythms.Visitors had to logon before downloading, and werecorded all click streams and down-loads.We then analyzed the data in realtime and offered personalized recom-mendations to the users. The “StudyMethodology” sidebar explains howwe organized the study to achieveaccurate results.

FINDINGSWe collected transaction and click

stream data. Because a few subjectsnever logged on after registration, wediscarded these records in the analysis,resulting in 139 remaining subjects outof the original 182. All together, thesubjects browsed 26,397 pages andnavigated through 6,952 item descrip-tions in the six-week study period.

Registrationlogon or cookie

click stream

Userprofile

click stream

Surfers withdifferent

characteristics Web server

Personalizationrules

Message contents—

Web layout—

Presentationtiming

Areas forcontrol by

personalization

Attention—

Purchasedecisions

PersonalizedWeb pagesto influence

Figure 1. Overview of personalization.

September ❘ October 2003 IT Pro 55

download button more than once. Inour analysis, we discarded duplicateddownloads that took place within avery short time. This resulted in 4,386

downloads. Figure 2 depicts four out-comes of the two groups: file down-load, inept browsing (which is a largenumber of pages browsed before

There were 5,843 music file downloads.Users downloaded some files repeat-edly within five minutes. In such cases,it was likely that subjects pressed the

We began the study by recruitingsubjects through e-mail advertise-ments. Study participants were allvolunteers who had previous expe-rience in downloading music. Weprovided them with two incentives.First, they could download musicfiles for free. Second, each receiveda $5 coupon book.

All subjects signed a contract andagreed that all downloaded musicfiles were for research purposes, andthat they would not further distrib-ute the files to others. During regis-tration at the beginning of theexperiment, subjects provideddemographic information, such asage, gender, and education level, aswell as their music preferences. Thepersonalized agent analyzed sub-jects’ preferences, together withtheir profiles. We sent regular mar-keting announcements to the per-sonalized and control groups. Wegave personalized recommenda-tions, however, to the personalizedgroup only.All subjects could accessthe music site from home or fromtheir offices.

Of the 182 subjects participatingin the study, 106 were males and 76were females.Their average age was23.3. We randomly assigned 40 ofthe subjects to the control group,while we assigned 142 into the per-sonalized group. In the controlgroup, we did not offer personalizedmessages to subjects and did notpersonalize the Web interface. Weintentionally put more subjects inthe personalized group because weconducted a survey at the end of theexperiment to collect commentsfrom those receiving personalizedmessages.

Each time a subject (personalizedand control) logged on to the musicWeb site, song offerings appeared onthe right-hand side of the Web inter-face, as Figure A shows, five underMarket Offers and five under New

Offers. Market Offers were songsthat publishers were promoting.New Offers were newly releasedsongs. Subjects could click on asong’s hyperlink and enter anotherpage that described the song by giv-ing album details and singer infor-mation. Users could also select theirfavorite music and singer categoryby using the hyperlinks fromPersonal Offers located on the leftside of the Web interface.

Subjects in the personalized groupreceived personalized recommen-dations throughout the study. Basedon their profiles and download his-tory, Obert 1.2 recommended songs

for each individual after each logon.The system recommended person-alized song items under the headingOur Offers on the interface’s rightside.This list remained the same fortwo consecutive logons and occu-

pied about 15 percent of the naviga-tion area. The control group did notreceive these offers.The central partof the interface displayed additionalpersonalized recommendations forthe personalized group while pre-senting subjects in the control groupwith random offers.

After the subjects had used themusic site for six weeks, we distrib-uted an online questionnaire tothose receiving personalized mes-sages. This survey aimed at enhanc-ing our understanding of thesubjects’ perception of the person-alization agent’s performance and itspotential application domains.

Study Methodology

Figure A. User interface in the music download site.

56 IT Pro September ❘ October 2003

downloading),browsing-to-downloadratio, and song rating.

Our findings indicate that the sub-jects receiving personalized offersdownloaded music files more fre-quently than those in the controlgroup.

On average, personalized usersdownloaded 51.82 songs during theperiod, compared with 20.3 songs forcontrol users.Also, because personal-ization aims to reduce the cognitiveeffort in selection, a user interacting

with a personalization agent shouldexpend less effort in browsingthrough the available choices. Ourempirical results support this argu-ment. Regarding inept browsing, thepersonalized group browsed fewerpages (4.69 on average) than the con-trol group, implying personalizedusers exert less effort in making theirdownload decisions. The browse/download ratio further supports this;the personalized group browsedfewer items per download compared

with the control group. We also col-lected users’ ratings (based on a 5-point scale) on the recommendedsongs on display in the center of theWeb interface. The personalizedgroup gave the songs an average scoreof 3.73. The control group gave anaverage score of 3.49. Thus, the per-sonalized group was more satisfied,presumably because they chose fromagent-recommended songs.

Our findings indicate that the per-sonalized group performed less ran-dom browsing.That is, they expendedless effort in finding their favoritesongs. Personalized users in generalfound the recommended offeringsmatched their preferences and theywere willing to accept the offers with-out further choice consideration. Wedraw these conclusions based on thepersonalization group’s higher num-ber of downloads and less extensivebrowsing among subjects. Further-more, they were more satisfied withtheir downloaded songs.

To supplement our analysis on userbehavior, we sent a questionnaire tosubjects in the personalized group.There were 110 complete responsesto our survey. More than 90 percentof the surveyed subjects reported thatthey would like to receive and con-sider personalized recommendations.Table 1 reports subjects’ perceivedusefulness of personalized services forthe online purchase of different cate-gories of products.As the table shows,they were more receptive to low-price, low-involvement products, suchas books and DVDs. On the otherhand, subjects showed lower accep-tance for personalized offers forgoods leading to potential risks. Forinstance, misuse of cosmetics andbody care products could lead to skinallergies. Our findings show that per-sonalization on cosmestics might notbe too persuasive.

T he study provides empirical sup-port that Web personalizationcan motivate users to consider

agent-recommended items. In this

P E R S P E C T I V E S

Table 1. Perceived usefulness of personalized services.

Statement Participants agreeing with statement (percentage)

Personalized information is useful for the following products:

Books 62.73

Event or movie tickets 54.55

Music CD 44.55

DVD or video CD 40.00

Computers 18.18

Cosmetics or skin care products 11.82

Financial products 4.55

Personalization is not useful at all 14.55

Filedownload

0

10

20

30

40

50

60

Use

rs p

erfo

rmin

g a

ctiv

ity

(per

cen

tag

e)

Control groupPersonalized group

20.3

51.82

Ineptbrowsing

11.22

4.69

Browse/downloadratio

2.64 1.13

Songrating

3.49 3.73

Figure 2. Internet activity of control and personalized group.

September ❘ October 2003 IT Pro 57

case, the recommended items werefree music files. Although somebelieve there is little differencebetween the personalized and controlgroups because the downloaded itemswere free for both groups, we foundsignificant differences in user behav-ior in the two groups. The personal-ized group downloaded more songsthan the control group. This findingprovides insights important to ourunderstanding of free download itemssuch as electronic coupons and trialversions of software products. Firmsshould pay equal attention to freedownload items because they incurcosts on the merchant and user sides.User costs include cognitive effortand time spent in the selectionprocess. By using personalizationagents to target relevant segmentsand offering e-coupons or trial ver-sions of a digitalized product that

meet users’ needs, firms can achieveshort- and long-term objectives.Usingeffective persuasion messages, theagent can increase the yield of cross-and up-sell opportunities in the shortterm. In the long term, personalizedofferings are more likely to implant amessage about the product or servicein the user’s memory.

The Web has become an importantchannel for firms to promote and dis-tribute products and services (L.P.A.Simons, C. Steinfield, and H.Bouwman, “Strategic Positioning OfThe Web In A Multi-Channel MarketApproach,” Internet Research, vol. 12,no. 4, 2002, pp. 339-347).Web person-alization offers attractive valuepropositions to customers and canhave a major impact on a firm’s bot-tom line as yield on cross- and up-sellopportunities increases. This currentwork focuses on download activity

rather than on a consumer’s purchaseintention. Future work should inves-tigate the link between personalizedofferings and transactions thatinvolve a monetary exchange. �

Kar Yan Tam is professor of informa-tion and systems management, anddirector of the Research Center forElectronic Commerce at the HongKong University of Science and Tech-nology. Contact him at kytam@ust.hk.

Shuk Ying Ho is a PhD candidate atthe Hong Kong University of Scienceand Technology. Contact her atsusanna@ust.hk.

For further information on this or anyother computing topic, visit our Digi-tal Library at http://computer.org/publications/dlib.

Get accessto individual IEEE Computer Society

documents online.

More than 67,000 articles and

conference papers available!

$9US per article for members

$19US for nonmembers

http://computer.org/publications/dlib