an integrated approach to measure web site effectiveness in the

Information Technology & Tourism, Vol. 6 pp. 257–271 1098-3058/04 $20.00 + .00Printed in the USA. All rights reserved. Copyright © 2004 Cognizant Comm. Corp.

www.cognizantcommunication.com

257

Address correspondence to Karl W. Wöber, Vienna University of Economics and Business Administration, Institute for Tourism andLeisure Studies, Augasse 2-6, A-1090 Vienna, Austria. E-mail: [email protected]

AN INTEGRATED APPROACH TO MEASURE WEB SITE

EFFECTIVENESS IN THE EUROPEAN HOTEL INDUSTRY

ARNO SCHARL,* KARL W. WÖBER,† and CHRISTIAN BAUER‡

*The University of Western Australia, Business School, 35 Stirling Highway, Crawley, WA 6009, Australia†Vienna University of Economics and Business Administration,

Institute for Tourism and Leisure Studies, Augasse 2-6, A-1090 Vienna, Austria‡University of Notre Dame Australia, School of Information Technology

19 Mouat St, Fremantle, WA 6959, Australia

This study employs a novel method of Web content extraction and analysis to investigate the evolvingcompetitive landscape in an important business-to-consumer (B2C) area: travel and tourism. Findingsfrom a comprehensive Web mining endeavor and a supplier survey shed light on the effectiveness oftourism Web sites. Important dimensions of the automated measurement are ease of navigation, inter-active elements such as reservation and booking features, volume of textual and graphical informa-tion, number of available languages, and the textual diversity of documents. Precise textual informa-tion and interactive features are crucial to the success of a hotel Web site, measured in terms oftourists’ awareness, electronic inquiries, and online bookings. The article discusses differences be-tween four European destinations and the implications of benchmarks for Web site management.

Key words: Content mining; Web site evaluation; Technology acceptance model

Klein, 1999). Tourists have to travel to the place ofconsumption and cannot test the product in advance.They must agree upon the contract before being ableto directly evaluate and consume the product. Third-party information is the only means to assist themin the decision-making process. Thus, travel, trans-portation, and holiday services are among the mostpopular items to sell online in Europe (next to salesof computer hardware). European online travel salesgrew to 6.5 billion in 2002, which represents a mar-

Introduction

Organizations entering electronic markets findthemselves in an unusual role. They become con-tent providers and have to electronically deliver ac-curate and timely information about their businesspolicies, products, and services. This is particularlytrue for the travel and tourism industry, whose spe-cial characteristics increase the attractiveness of dis-seminating information electronically (Werthner &

258 SCHARL, WÖBER, AND BAUER

ket share of 3.6%—up from 2.3% in 2001(Marcussen, 2003). Marcussen expects this trend tocontinue with an increase of about 38% in 2003 toabout 10.3 billion, equivalent to 4.7% of the mar-ket. According to Hotel & Lodging Commerce re-port of PhoCusWright, a travel industry intelligencefirm specializing in research, hotel bookings madeonline will increase from 9% to 20% of total book-ings between 2002 and 2005 (PhoCusWright, 2000,2002). The European market also enjoys a very highacceptance of mobile services (Buhalis, 2003). Thenext generation of mobile phones will enable trav-elers to interact with tourism organizations duringall stages of the consumer decision-making process.These technological trends in the distribution of tour-ism products will cause fundamental changes in eco-nomic models and market leadership.

Web sites of suppliers dominate the Europeanonline travel market. Airlines, tour operators, hotels,railways, and car rental companies accounted for74% of Europe’s online travel in 2000 (Cohen,2000). Online intermediaries captured only the re-maining 26% of the market. This microstructuredappearance of the travel and tourism online marketsuggests that Web-based strategies will have a dra-matic impact on the competitive dynamics withinthe industry. Because of the significant costs of de-veloping, advertising, and maintaining a Web site,the accurate measurement of its characteristics andcompetitiveness is crucial to long-term success.

Many studies on Web site effectiveness in the tour-ism sector draw upon the respective authors’ assump-tions and experiences and thus fail to systematicallyaddress questions of intra- and intercoder reliability(e.g., Hamill & Gregory, 1997; Morrison &Morrison, 2000; Olsina, Godoy, Lafuente, & Rossi,1999). Traditionally, the evaluation of Web sites isbased on expert opinions (Buhalis & Spada, 2000;Chung & Law, 2003; Jung & Butler, 2000; Whyte& Bytheway, 1996), or on user feedback (Jeong, Oh,& Gregoire, 2003; Kucuk & Arslan, 2000; Loban,1998).

The dynamic nature of electronic content requiresrepeated evaluations in regular intervals (McMillan,2000; Mitra, 1999). Manual approaches, however,preclude frequent benchmark assessments and largesample sizes because it is cumbersome to evaluatetechnical structure and layout elements of Web sitesthrough “standard” browsing. Manual approaches

often suffer from the subjective nature of humanjudgments and the results’ rapid obsolesce. Conse-quently, this study applies an automated Web assess-ment methodology to analyze content from a largenumber of sites, and integrates the results with em-pirical success criteria. From a random sample of328 European hotel Web sites in the German-speak-ing Alpine region (Austria, Germany, Switzerland,and South Tyrol, the northern part of Italy), the au-tomated process collected more than 80 site attributesdescribing ease of navigation, number of availablelanguages, the volume and diversity of textual andgraphical information, and interactive elements suchas reservation and booking features.

The following section reviews effectiveness stud-ies of Web sites, with emphasis on the travel andtourism industry. A conceptual framework then leadsto hypotheses about factors that influence acceptanceand success of tourism Web sites. After outliningthe automated Web assessment methodology, the keyindicators of the Web site evaluation model are ex-plained. Then the identified success factors are pre-sented and discussed, with a special emphasis onlanguage characteristics and implications for tech-nology diffusion, Web development, and contentproduction. The article concludes with a section onlimitations of the study and directions for furtherresearch.

Research Background

Generally, manual and automated approaches toanalyzing and evaluating online resources have tobe distinguished. They cover a broad range of is-sues, from the basic questions of syntactical correct-ness and browser compatibility to problems of highercomplexity such as benchmarking server perfor-mance, rating content, evaluating representationalquality, or assessing the application’s overall busi-ness value. In the following paragraphs, an overviewof tourism-related studies of Web site effectivenessis given, with a special emphasis upon user decisionbehavior and information quality.

Effectiveness Studies of Tourism Web Sites

Tierney (2000) noted that published research onthe effectiveness of tourism Web sites is limited.Following a study by Schonland and Williams(1996), he investigated the effectiveness of the offi-

HOTEL INDUSTRY WEB SITE EFFECTIVENESS 259

cial Web site of the California Division of Tourism(http://www.gocalif.ca.gov), which represented atypical tourism destination Web site without reser-vation or booking facilities. Tierney applied bothonline and email surveys among 833 persons tomeasure the effectiveness of the site. Due to the fo-cus on only one site, his findings demonstrated therole of electronic media in survey projects but wereonly of limited value with regard to evaluating thesite itself. Tierney (2000) acknowledges this whenhe states: “A topic that may be useful for future Website assessment is an evaluation of the attributes ofthe site itself for ease of navigation, finding, andrequesting information” (p. 219).

Only recently, first attempts have been under-taken to evaluate such attributes for tourism Websites. Jung and Butler (2000) examined the percep-tions of marketing managers from the tourism andhospitality industry. Although their findings aremethodologically limited (email survey with un-known bias in response), useful insights can begained in the way managers measure the effective-ness and success of their Web sites. With regard tosuccess factors, most managers perceived value-adding content, ease of use, and security as the mostimportant issues. They also found that Web siteeffectiveness varies significantly by sector (e.g.,hospitality, destination management, or airline).Consequently, the industry should not adopt gen-eral recommendations that do not reflect evalua-tion criteria specific to the sector under consider-ation (Jung & Butler 2000).

In another study that captures tourism experts’opinions on destination management systems,Buhalis and Spada (2000) evaluate a list of successfactors from different stakeholder perspectives (con-sumer, tourism suppliers, public sector, investors,tour operators, travel agents). Important issues raisedby this exploratory study were user friendliness,comprehensive destination information (content),online booking and payment facilities, reliability andaccuracy, the ability to interface with multiple dis-tribution channels, speed of responses, adoption ofpersonalized services, and multimedia presentationof products.

Combining the concept of information qualityand consumer decision behavior theory, a study byJeong and Lambert (2001) tested a framework toevaluate the information quality of experimental

hotel sites specifically developed for that purpose.Considering the general Technology AdoptionModel (TAM) originally introduced by Davis(1989) and further discussed by Adams, Nelson,and Todd (1992), Jeong and Lambert develop theirown model comprising four measures of informa-tion quality: perceived usefulness, perceived easeof use, perceived accessibility, and attitudes. Con-ference attendees (n = 240) tested eight hypotheti-cal hotel Web sites, suggesting that informationattributes such as timeliness, relevancy, and accu-racy represent key determinants of customers’online behavior. As their findings are limited bythe convenience sample of conference attendeesand the limited choice of alternative Web designs,the authors propose to extend their model by real-world data and to consider additional motivationalfactors such as prior experience and desire.

To analyze indicators of Web site quality, Jeonget al. collected data from a random sample of 1743US Internet shoppers through an electronic survey(Jeong et al., 2003). They found that informationsatisfaction impacts the perceived quality of a siteand plays a critical role in determining the customer’spurchase intentions. The data set did not reflect spe-cific site characteristics, which limited the implica-tions that could be drawn from the study.

Benchmarking

Benchmarking is a management tool for Web siteevaluation and improvement (Johnson & Misic,1999). The success of benchmarking initiatives forthe evaluation of tourism Web sites in recent years(Oertl, Thio, & Feil, 2001; Schegg, Steiner, Frey, &Murphy, 2002) relates to their knowledge-sharingand motivational characteristics. Although the ini-tiatives present useful guidelines for individual com-panies, they fail to provide help for best-practicecompanies and cannot identify the most relevantcriteria for long-term success. Because the objec-tive of benchmarking is to improve a certain pro-cess, a benchmark must obviously define the term“better.” While simple in concept, managers havefound it difficult to identify operational quality char-acteristics of Web sites. Essentially, this requiresdefining the term “better” in the context of systemdevelopment and within the broader perspective ofthe supplier’s operating environment.


Theoretical Framework

This research studies the effectiveness of struc-tural and textual components of tourism Web sitesand focuses on hotel sites in the German-speakingAlpine region. Effectiveness is investigated throughthe relationship between Web site characteristics anddesired commercial outcome. Confirming the posi-tive and negative impacts of these characteristicshelps optimize content production and content rep-resentation in the hotel industry. In relating struc-tural and textual site characteristics to success mea-sures, this research builds on previous studies thathave defined theoretical models of Web site success.

Conceptual Models of Web Site Effectiveness

Literature on management information systemsand decision support systems helps build a generalmodel for user adoption of information technol-ogy. Elaborating on the success of marketing man-agement support systems, for example, Wierenga,Van Bruggen, and Staelin (1999) find the matchbetween the decision processes to be supported(demand side) and the functionality of the man-agement support systems employed (supply side)to be the primary driver for the potential successof a system. In the management sciences, numer-ous studies have been performed to investigate thedeterminants of system usage and success.Eierman, Friedman, and Adams (1995) conducted

the most comprehensive literature review, using 15out of 200 articles from acknowledged researchjournals to identify the most important variablesdeveloped and tested for evaluating decision sup-port systems. Their literature review indicated thatonly half of the possible relationships among con-structs had actually been tested, and that divergentresults may result from interactions with other con-structs, necessitating richer research models to un-derstand these complex relationships.

Davis (1989) undertook a widely recognized re-search effort on success factors of information anddecision support systems by investigating the influ-ence of perceived usefulness and perceived ease ofuse on actual system usage. He developed a mea-surement instrument for assessing user acceptanceof information technology, the TAM (see inner partof the model displayed in Fig. 1). Using the estab-lished TAM scales, Davis and other authors success-fully predicted future system utilization. Davis(1989) and Davis, Bagozzi, and Warshaw (1989)used regression analysis to determine the relation-ships in TAM. Several alternative structural modelsof the TAM instrument were tested by Adams et al.(1992), Szajna (1996), Hendrickson and Collins(1996), and Igbaria, Zinatelli, Cragg, and Cavaye(1997). The findings of the latter two studies sup-ported a full causal model, which was recently ap-plied in a tourism management related study byWöber and Gretzel (2000).

Figure 1. Conceptual Web site adoption model.

Expectations

Experience

Perceived

Usefulness

PerceivedEase of Use

Usage

Interactivity

Navigation

Languages

Services

Layout

Product

Speed

Intelligence

Speed

Operability

Personal Factors

uncontrollable to hotel/tourism management

System Factorscontrollable tohotel/tourismmanagement

Media Factors

uncontrollable to hotel/tourism management

Visits

Awareness

Hits

SuccessMeasures

Uncontrolled

Controlled

Variables inthis study

TechnologyAcceptance Model

Davis 1989Reliability

Page views

Availability

Inquiries

Revenue

Bookings


The validity of TAM was assessed and confirmedfor Web-based communication (Teo, Lim, & Lai,1999) and work-related tasks (Lederer, Maupin,Sena, & Zhuang, 2000). Teo et al. investigated theinfluence of perceived enjoyment as a motivationalfactor during Internet usage. Self-reported measuresamong 1370 respondents indicate that utility is gen-erally more important than perceived ease of use andperceived enjoyment. From a tourism managementperspective this implies that attractive systems anduser-friendly systems may be abandoned in the longrun if they fail to provide critical functionality suchas reservation and booking services.

The study by Lederer et al. (2000) surveyed 163managers and their use of regularly accessed Websites to perform job-related functions. Antecedentitems for ease of use suggested by the authors in-cluded consistency of text and graphics, number oflinks to other information, navigation and searchtools, the responsiveness of the system, and the (vi-sual) attractiveness of the layout. From a method-ological point of view, their investigation is limitedby evaluating only frequently visited sites, neglect-ing other sites that might have been conceptuallyand technically superior.

Dependent Variables

For the present study the authors extended andadapted TAM to online tourism as shown in Figure1. Perceived ease of use and perceived usefulnessinfluence system usage and thus relate to two gen-eral aspects of consumer decision making: intensityof information search and number of sales gener-ated by the system. Search for product informationis conceptualized as the number of alternatives forwhich detailed information is requested (Moorthy,Ratchford, & Talukdar, 1997). In Web site effective-ness studies, this coincides with the number ofviewed pages containing attribute information abouta particular product.

Alternative metrics commonly used to measureWeb site success are the number of hits and visitstracked by the server’s protocol features. Hits havebeen widely criticized, as their weakness as a validmeasure of Web traffic is quite evident. They mea-sure how close a server is to reaching its capacity,but are less useful in determining how many peopleaccess the server. Because hits include all types of

content sent by the server when a particular docu-ment is accessed (images, text, scripts, applets, etc.),they are noncomparable across Web sites (Novak &Hoffman, 1997). Visits are identified by summariz-ing a stream of raw hits, for example a sequence ofdocuments including embedded objects. Unfortu-nately, multiuser systems, irregular access patterns,and the routing of Web traffic via proxy servers re-duce the validity of visits as a measure of Web traf-fic. Entries in the log file do not guarantee that usershave comprehended or even read the transferred in-formation. Only a more detailed analysis of theirclickstream can increase the observation’s validity.

A well-designed application can lead to increasedcustomer awareness, which may result in attemptsto establishing contact with the vendor through oneof the available channels. Many customers still pre-fer traditional communication via telephone or fax,although they discovered the offer by accessing aWeb site. At the same time, not every email receivedby a company necessarily stems from the Web site.Many tourism and hospitality businesses dissemi-nate their email address via a number of media, in-cluding printed brochures and magazines. Similarlimitations of measuring the success of corporateWeb sites refer to the number of online bookingsand the generation of revenue by online sales. Sta-tistics solely based on server log files are mislead-ing if the system’s impact on traditional communi-cation channels is disregarded. To accuratelymeasure the overall impact of a tourism Web site, itis necessary to evaluate and monitor the origins ofcontacts and to record them systematically withinthe company. Unfortunately, this type of informa-tion is not accessible externally. Therefore, this studysurveys tourism managers asking for key indicatorsof online success. The questionnaire, which was de-veloped, pretested, and subsequently sent to all ho-tels contained in the sample, requested informationconcerning the number of bookings, inquiries, andthe managers’ estimate of how many guests are ac-tually aware of their site.

Independent Variables

Whereas substantial theoretical and empiricalwork supports TAM, little research modeled the de-terminants of perceived ease of use and perceivedusefulness (e.g., Venkatesh & Davis, 2000). Incor-


porating travel counseling research and managementscience literature, a set of indicators pertaining tothese determinants has been developed. These indi-cators fall into three categories: personal factors,system factors, and media factors. Personal factorsinclude the user’s experience in travel planning, in-formation technology usage, and expectations to-wards the system. Previous studies suggest includ-ing experience in technology adoption models(Benbasat, Dexter, & Masulis, 1981; Lucas &Nielson, 1980; Novak, Hoffman, & Yung, 2000;Taylor & Todd, 1995) and reveal significant rela-tionships with performance indicators. For traveldecision making in particular, which is a highly com-plex and strongly motivational-driven process, ex-perience is an important determinant (Wöber &Gretzel, 2000). In contrast to personal factors, sys-tem factors and media factors refer to technologicalaspects of the system and its environment. Variablesreferring to the characteristics of the distributionmedia such as Internet, WAP, or interactive TV aretheir responsiveness and their suitability for the re-spective task (Dellaert & Kahn, 1999; Pröll &Retschitzegger, 2000). The proposed nine discretion-ary variables focus on the system’s core capabili-ties, as hotel management cannot directly controlpersonal and media factors:

1. Product: The perceived utility of services andtheir Web-based representation (e.g., hotelrooms, sports facilities, transfers, etc.), mea-sured in relation to the total costs of obtainingthem.

2. Speed: The responsiveness of the system, deter-mined by server characteristics and the Internetservice provider. Dellaert and Kahn (1999) foundthat waiting time negatively affects consumerevaluation of Web content only when speed limi-tations are managed inadequately.

3. Intelligence: The recommendation strategiesimplemented into the application and how wellthey adapt to varying customer needs. From aconceptual perspective, adaptive Web presen-tation techniques can be grouped into content-level, link-level, and meta-level adaptation(Scharl, 2000). The effectiveness of recommen-dation agents in online shopping environmentshas been demonstrated recently by Häubl andTrifts (2000).

4. Layout: The degree of attention raised by thetextual content and by the visual representationof the system (e.g., Benbasat et al., 1981). Thetrajectory established on a specific Web site (theuser’s clickstream and reading path) can be in-fluenced by a wide variety of means such asarrangement and perspective, relative size, con-trast, color, crispness, or transparency.

5. Services: The variety and appropriateness ofservices offered by the system (e.g., handlingof vacancy inquiries, reservation and bookingservices, or online payment services). Contin-ued site usage without specific goals may de-cline over time when the novelty effect of Website wears off (Teo et al., 1999).

6. Languages: The number of available lan-guages, which determines the percentage ofpotential customers that can understand anduse the system. An analysis of the global dis-tribution of native speakers as of March 2003(http://www.glreach.com/globstats/) revealsthat only 35.2% of the online population wasof English-speaking origin, followed by Chi-nese (11.9%), Japanese (10.3%), Spanish(8.1%), and German (6.5%).

7. Navigation: Features assisting the user to con-trol the system and to maneuver through its vir-tual space (Lederer et al., 2000). The primarynavigational system of a Web site includes con-textual links embedded in the textual informa-tion of a document and local noncontextuallinks. In advanced electronic environments, us-ers may verify the relation between differentvirtual spaces by supplemental navigationalsystems such as indexes or site maps (Scharl,2000).

8. Interactivity: This factor is a general attributeof procedural, participatory environments(Murray, 1997). While the term proceduralstands for the ability to execute a series of rules,only participatory systems allow the user toinduce this rule-generated behavior.Interactivity thus delineates the “media’s poten-tial ability to let the user exert an influence onthe content and/or form of the mediated com-munication” (Jensen, 1998, p. 201). For evalu-ating B2C applications, interactivity comprisesthe system’s capability to support users in con-tacting the supplier (e.g., via forms or email),


or in communicating with each other by meansof virtual communities. Advanced systems regu-larly support features such as guest books, mail-ing lists, or online chat facilities.

9. Reliability: The concepts of reliability (consis-tency, stability, reproducibility, and accuracy)and validity (quality of the approximation oftrue economic value) are inextricably inter-twined. They describe the quality of the infor-mation provided as well as the integrity andworkability of the system itself.

Methodological considerations led to an adjustedWeb site adoption model based on six independentfactors. With regards to the second variable, speed,it is often impossible to determine the actual bottle-neck in the case of delayed transmissions. For thevariables product and intelligence, manual investi-gations are inevitable and thus were not covered bythe automatic content analysis methodology, whichwill be described in the following section.

Data Collection and Preprocessing

This research relates profiles from an automatedtool to capture and benchmark Web sites (indepen-dent variables) to measures of Web site success (de-pendent variables). The automated tool mirrored Websites from the tourism industry in monthly intervals.This research used 328 Web sites of hotels in theGerman-speaking Alpine region to empirically testthe conceptual Web site adoption model introducedin Figure 1 (randomly selected from hotel cataloguespublished by regional and federal tourism associa-tions in Austria, Germany, South Tyrol, and Swit-zerland).

A questionnaire to gather success measures wasdeveloped, pretested, and subsequently sent to eachhotel of the sample in May 2001. It asked for theyear of initial site implementation, type of develop-ment (in-house, external partner, tourism portal,other), site awareness, percentage of guests thatbooked electronically during the last 6 months, andpercentage of inquiries via email, Web-based forms,etc.

Managers of 144 hotels completed the question-naire. This equals a response rate of 43.9%, whichis comparably high for this type of survey. The Aus-trian Federal Economic Chamber (http://

www.wko.at/) contributed additional hotel attributessuch as star category, size, or product range tocomplement the questionnaire data.

Figure 2 outlines the method used to capture theindependent variables (Bauer & Scharl, 2000;webLyzard, 2004), which automatically accesses andanalyzes publicly available Web sites in regular in-tervals. From a methodological perspective, it canbe used for both snapshot and longitudinal studiesand generates two main types of parameters: struc-tural and textual. The structural parameters reflectgeneral system features such as the total number ofdocuments, distinct file types, or the number ofembedded images. Due to the multidimensional na-ture of hypertext (embedding content, visual, struc-tural and navigational information), the markup tagscan be used to extract information about all of thesedimensions simultaneously. The number of distinctimages, for example, has a visual impact on the site,the use of tables influences the structure of docu-ments, internal and external links define the naviga-tional system, and raw text represents a significantpart of site content.

The study needed to overcome a number of diffi-culties associated with automated data gathering,stemming from both general Web issues and par-ticulars of the hotel industry. The generic limitationof automated Web assessment (and indeed every at-tempt of browsing the Web) is the interpretation ofhyperlink-embedding technologies. Instead ofimplementing parsing techniques for these technolo-gies independently, this research utilized the open-source mirroring tool HTTrack (Roche et al., 2004)to capture the source data from publicly availablehotel sites. Hence, all technologies supported by the

Figure 2. Automated Web assessment.


tool such as HTML, image maps, and JavaScript arecovered, neglecting rarely used techniques not imple-mented in HTTrack (e.g., advanced MacromediaFlash or Java applets).

The limitation specific to the hotel industry is thehigh incidence of Web portals, in contrast to indi-vidual efforts of implementing and maintaining Websites. Portals complicate the identification of Website “boundaries,” which are typically defined by thesite’s domain name. As Web portals use one domainname for a large number of hotels, the data gather-ing method had to be restricted to specific documentpaths within the portal.

The data extracted from the hypertext documentscan be used to investigate a site’s textual content(Scharl, 2004). The first step of lexical analysis is con-verting the stream of characters contained in the origi-nal text into a stream of coding units (words). Identi-cal words are then grouped together by counting theiroccurrences in order to create a word list that is usu-ally sorted in order of decreasing frequency. Thesedata are then used to study attributes such as textualrichness, available languages, and interactivity. It alsoreflects industry trends and competitive strategies asdisplayed through the hotels’ Web site.

Table 1 displays a classification scheme for morethan 80 site metrics: (1) interactive features such asthe use of forms, scripts, applets, and so forth; (2)navigational mechanisms: structure and accessibil-ity of internal links, external links, and anchor linkswithin and between documents; (3) layout and mul-timedia characteristics: information on frames, em-bedded pictures, use of fonts and styles, etc.; (4)content-related descriptives such as the standardizedtype token ratio or the average lengths of linguisticunits (words, sentences, documents). The type to-ken ratio (TTR) is a syntactical index that dividesthe number of distinct words by the total number of

words. High values indicate texts with a heteroge-neous vocabulary. A running average based on con-secutive 1000-word chunks of text (sTTR) accom-modates the varying size of Web sites. Table 1outlines the groups of variables that are generatedfor each site of the sample in monthly intervals. Formost of the presented groups, total and distinctivevalues are distinguished and processed both in ab-solute terms and relative to the number of mirroreddocuments.

Findings and Discussion

Success Factors

Hotels of the observed destinations emphasizedifferent aspects of their Web content. Feedbackfrom the questionnaires helps explain these differ-ences and determine those aspects that have a sig-nificant impact on site success in terms of aware-ness, percentage of bookings, and number ofelectronic inquiries.

Compared across countries, the success measuresdo not differ significantly at the 0.05 level (Fig. 3).With regards to the percentage of electronic book-ings, arguably the most important measure, SouthTyrol leads with a mean value of 21.0% (SD = 0.14),ahead of Austria and Switzerland with a mean of15.1% each and SDs of 0.18 and 0.14, respectively.

Hierarchical cluster analysis and collinearityanalysis reduced the number of independent vari-ables to 31. The dependent variables were derivedfrom items 3–5 of the questionnaire: Q3 (awareness),Q4 (bookings), Q5 (inquiries), and Q345 (arithmeticmean of the three responses).

Correspondingly, four best-subsets regressionanalyses were performed, using a combination ofadjusted R2 and Mallow’s Cp statistic to determinethe optimal number of variables (Fig. 4).

Table 1Classification Scheme and Selected Site Attributes

Global Interactivity Navigation Layout Textual

Name # forms # internal links # images # paragraphsURL # fields # broken links # frames # sentencesMirror size (total, text) field/form ratio # external links # tables # typesNumber of documents # applets # SSL links # fonts # tokensNumber of tags # scripts # anchors # styles sTTRNumber of file extensions # email links # anchor links average length of sentences


For the model m based on pm out of a total of p

independent parameters, Mallow’s Cp utilizes thecriterion Cp = SS

e(p)/σ

e2 – n + 2p

m, where σ

e2 is the

residual variance estimate based on the model withall predictors. In evaluating alternative regressionmodels with p independent parameters, the goal isto find models that provide a maximum R2 whilekeeping Mallow’s Cp close to or below p

m+ 1

(Gagné & Dayton, 2002).A comparably low R2 adjusted of 19.8 demon-

strates the important role of other factors not cap-tured by this study such as multichannel marketing

initiatives to promote the site (e.g., publication ofthe site’s URL in brochures, advertisements in tra-ditional media, or direct marketing efforts). Suchinitiatives are not covered by the data set used inthis study, but obviously even the technologicallymost advanced Web sites will not be accessed andused frequently if not advertised adequately.

The left diagram of Figure 5 plots the R2 adjustedand Mallow’s Cp statistic for the best-subset regres-sion models based on 1–15 variables, and the arith-metic mean of the questionnaire responses (Q345)representing the dependent variable. Based on thisdistribution, the model comprising 12 independentvariables was chosen, whose coefficients and col-linearity statistics are listed in the table next to theR2/Cp plot.

In addition to the overall success factors as rep-resented by the coefficients in Figure 5 and the firstrow of variables in Figure 6, three additional bestsubset regression analyses (Q3/Q4/Q5) provided amore fine-grained picture of the different aspectsthat influence the success of a particular hotel site.The additional factors presented in the lower partof Figure 6 are not part of the combined Q345model, but have a significant impact on one or moreof the individual success measures. Figure 6 listsonly technical success factors and does not includethe number of beds a hotel provides, a proxy forthe size of the organization with a highly signifi-

Figure 3. Percentages of customers knowing the site, inquir-ing, or booking electronically between June and August 2001.

AustriaSwitzerlandSouth TyrolGermany

1.0

.8

.6

.4

.2

0.0

Awareness

Inquiries

Bookings

Figure 4. Best-subsets regression models for Q345.


cant (p≤ 0.01) negative influence on both the gen-eral success (Q345) and its subcategories (aware-ness, bookings, inquiries). In other words, smallerhotels leverage their Web sites more effectively.This may be explained partially by larger organi-zations’ economies of scale and the resulting focuson more expensive communication channels andpromotion strategies.

The number of tokens, the relative length of scriptsper document, the number of fields per form(p < 0.01), and, to a lesser degree, the number ofdistinct forms per document (p < 0.05) have a posi-tive influence on customers’ awareness of hotel sites.This confirms that customers tend to take more no-tice of a Web site with rich information and interac-tive features, as provided by forms and JavaScript.

Figure 5. R2 adjusted and Mallow’s Cp statistic for best-subset regression models based on 1–15 variables and Q345 as dependentvariable (left); coefficients and collinearity statistics of the chosen model comprising 12 variables (right).

Figure 6. Standardized beta coefficients and significance of the technical success factors’ influence on customer awareness, electronicbookings, and number of electronic inquiries.


The negative standardized beta coefficients for theaverage sentence length and the standardized typetoken ratio indicate that customers prefer conciseinformation over complicated linguistic structures,particularly during the booking process.

It is interesting to note that the number of internalbroken links, one of the most important negativefactors with regards to the perceived quality of Websites, has a positive impact on the number of inquir-ies. Broken links presumably cause some custom-ers to contact the hotel in order to report the prob-lem and obtain the missing information. Futureresearch should address this phenomenon and gathermore data about the exact nature of customer in-quiries.

The opposite effect can be seen with the numberof distinct images, which tend to reduce the numberof inquiries. Photos of the hotel, the different roomcategories, and other facilities often eliminate thecustomer’s need for further investigations.

The most significant factor for the number of in-quiries in positive terms is the number of distinctforms per document. Providing forms on a site in-creases the customer’s willingness to contact thehotel electronically—instead of using the telephone,

for example, or not contacting the hotel at all. Butonly when providing rich forms (i.e., forms that notonly contain a simple text field, but allow the struc-tured gathering of information) can this willingnessbe translated into actual bookings.

Language Characteristics

Providing multilingual content is often a neces-sity for European vendors, particularly in customer-oriented industries such as hospitality. Not surpris-ingly, the multilingual destinations Switzerland andSouth Tyrol take the lead in this category (see Fig.7). Documents are assigned to a language if theycontain at least 3 out of 10 characteristic short words;support of a particular language is assumed if it ac-counts for at least 5% of all published documents.Taking a closer look at the Austrian market, mostregions only provide English content in addition tothe standard German front-end. Only hotels inSalzburg, Carinthia, and Tyrol publish a limited num-ber of Italian and French documents.

In terms of textual richness, measured by thestandardized type token ratio as delineated above,we observe a rather different distribution. German

Figure 7. Average number of languages and relative language percentages per destination.


sites (0.48; SD = 0.05) employ the most complexvocabulary, closely followed by South Tyrol (0.47;SD = 0.05), Switzerland (0.44; SD = 0.06), and Aus-tria (0.41; SD = 0.11).

Technology Diffusion

The left diagram of Figure 8 summarizes the an-swers to the first questionnaire item (“Since whendoes your organization operate a Web site?”) andgroups them by destination. Austrian hotels were firstmovers with regard to offering Web-based informa-tion systems, although the initial wave of implemen-tation has subsided by now. Within Austria, four-star hotels initiated the trend, followed by hotels inthe three-star category.

In light of these findings, many organizations ofthe other destinations can be considered late adopt-ers (Rogers, 1995) that have been catching up rap-idly in recent years. Late adopters, however, do notseem to have a significant disadvantage relative tothe pioneers and are often able to achieve compa-rable awareness and online booking ratios.

Web Development and Content Production

The answers to the second questionnaire item (“Inwhich way do you implement and maintain your Website?”) reveal significant differences in Web site man-agement. Austrian sites rely heavily on portals, due

to the early promotion and public funding of portalssuch as TIScover (http://www.tiscover.com/). Thefindings of this study suggest that inclusion in tour-ism portals and booking systems, as an alternative todeveloping an independent Web site, may affect imple-mentation costs but does not automatically translateto a competitive advantage.

Conclusions and Research Outlook

This article presents a novel approach for gather-ing and interpreting a multidimensional set of Website characteristics in the tourism industry. A quan-titative method for large-scale Web assessment iscombined with supplementary success measurescollected via a questionnaire in order to investigatetrends within the industry and determine the primarysuccess factors. A Web site adoption model basedon the Technology Acceptance Model by Davis(1989) is introduced, which yields valuable insightsinto factors relevant for the long-term success ofWeb-based information systems. For the investigatedhotel sites in the German-speaking Alpine region,several success factors are identified and supportedby empirical evidence.

Small and medium-sized businesses in particularbenefit from technological change. In terms of onlinebooking ratio, Web offerings of smaller hotels aremore successful—an important finding consideringthe highly fragmented and microstructured charac-

Figure 8. Year of initial Web implementation by destination and hotel category.


ter of this sector. The rapid development of infor-mation technology generates both opportunities andthreats for small accommodation providers. Web-based information systems are additional tools forbusinesses to manage and market themselves in amore efficient and effective way. The strategic weak-nesses of small and medium-sized enterprises, how-ever, are the lack of expertise to take advantage ofthese opportunities (Buhalis, 2003).

With regard to market entry and implementationstrategy, late adopters are not at a significant disad-vantage relative to Web pioneers. They are able toachieve similar online booking and awareness ra-tios, while often considerably reducing implemen-tation costs.

Online users’ information satisfaction dependsupon technical and content-related attributes of aWeb site. This study found that online awarenesscan be improved through rich but precise textualinformation and interactivity (different types of struc-tured forms or JavaScript). The mere quantity ofonline information does not significantly influenceonline awareness. The ease of use for a hotel Website can be enhanced by presenting only relevantinformation, applying consistent structuring guide-lines, and providing simple and clear directions ofuse. Finally, it appears less difficult to detect andanalyze factors that increase awareness for a spe-cific Web site than for its ratio of online bookings.This limitation certainly relates to the methodologi-cal characteristics of the applied data mining proce-dure, which predominantly tackles design elementsimportant in the early phase of the decision makingprocess.

Directions for Further Research

In the German-speaking Alpine region, Austrianhotels in general and organizations belonging to thefour-star category in particular were the first accom-modation providers to offer Web-based informationsystems, although the initial wave of implementa-tion has subsided by now. As late adopters catch uprapidly, iterative improvements along the lines sug-gested by the Web site adoption model presentedwill be vital even for those hotels that still hold acompetitive advantage.

With this research, the foundations have been laidfor the observation, description, and content analy-

sis of large samples of Web-based artifacts. As ofJuly 2004, more than 6000 Web sites are monitoredon a monthly basis. In addition to the hotel sites cov-ered in this article, the tourism sample comprisesapproximately 650 sites of regions, cities, countries,airlines, and reservation agencies. Incorporatingother categories in the Web site adoption model pre-sented in this article, a comprehensive tourismbenchmarking and monitoring system is planned.Secondary research will help leverage these data toderive theoretical knowledge and advance existingtheories about the nature and dynamics of Web con-tent. Both snapshot and longitudinal studies regard-ing Web content on the macro-level (comparativeanalyses within industries and specific groups ofmedia) and the micro-level (trends in the coverageof certain topics or benchmarking organizationalefforts to implement and maintain Web-based infor-mation systems) will facilitate and complement thistheory-building process.

Acknowledgments

The Austrian Federal Economic Chamber (http://www.wko.at) provided funding and additional sup-port during the data-gathering phase of this project,which has been conducted in cooperation with Prof.Martin Natter and Prof. Alfred Taudes from the De-partment of Production Mangement, Vienna Univer-sity of Economics and Business Administration. Theauthors also thank Prof. Jamie Murphy from theUniverstiy of Western Australia for his valuable feed-back on the original manuscript.

Biographical Notes

Arno Scharl is Professor of Information Systems at the Busi-ness School of the University of Western Australia. He com-pleted his doctoral research on reference modeling of massinformation systems at the Vienna University of Economicsand Business Administration. Additionally, he holds a Ph.D.and M.Sc. from the University of Vienna, Department ofSports Physiology. Drawing on 10 years of experience indeveloping and evaluating Web-based information systems,he founded the ECOresearch Network (www.ECOresearch.net) and published more than 70 refereed articles on envi-ronmental online communication, information systems de-velopment, and automated Web assessment.

Karl W. Wöber acquired his Ph.D. from the Vienna Univer-sity of Economics and Business Administration where he


became Associate Professor at the Department for Tourismand Leisure Studies in 2000. He is also free consultant forthe Austrian Society of Applied Research in Tourism, andTechnical Advisor of European Cities Tourism and the Eu-ropean Travel Commission. His main research activities arein the fields of decision support systems, strategic market-ing planning, and evolutionary computation particularly withapplications in city tourism and hospitality management.

Christian Bauer has been researching and managing the de-velopment of Web information systems since 1994. Co-founder of the webLyzard project and long-time member ofthe EcoMonitor research network, Dr. Bauer has publishedmore than 30 articles on electronic commerce and onlinecommunication. Previous academic appointments includeassistant professor at the Vienna University of Economicsand Business Administration and a postdoctoral fellowshipat Curtin Business School.

References

Adams, D. A., Nelson, R. R., & Todd, P. A. (1992). Perceivedusefulness, ease of use, and usage of information tech-nology: A replication. MIS Quarterly, 16, 227–247.

Bauer, C., & Scharl, A. (2000). Quantitative evaluation ofWeb site content and structure. Internet Research: Elec-tronic Networking Applications and Policy, 10(1), 31–43.

Benbasat, I., Dexter, A. S., & Masulis, P. S. (1981). An ex-perimental evaluation of graphical and colour-enhancedinformation presentation. Management Science, 31(11),1348–1364.

Buhalis, D. (2003). eTourism. Information technology forstrategic tourism management. Harlow: Prentice Hall.

Buhalis, D., & Spada, A. (2000). Destination managementsystems: Criteria for success—an exploratory research.Information Technology & Tourism, 3, 41–58.

Chung, T., & Law, R. (2003). Developing a performance in-dicator for hotel websites. Hospitality Management, 22,119–125.

Cohen, N. (2000). eGlobal report. International Data Cor-poration [Online]. http://www.emarketer.com/

Dellaert, B. G. C., & Kahn, B. E. (1999). How tolerate isdelay? Consumers’ evaluations of Internet Web sites af-ter waiting. Journal of Interactive Marketing, 13(1), 41–54.

Davis, F. D. (1989). Perceived usefulness, perceived ease ofuse, and user acceptance of information technology. MISQuarterly, 13, 319–340.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). Useracceptance of computer technology: A comparison of twotheoretical models. Management Science, 35, 982–1002.

Eierman, M. A., Friedman, F., & Adams, C. (1995). DSStheory: A model of constructs and relationships. Deci-sion Support Systems, 14, 1–26.

Gagné, P., & Dayton, C. M. (2002). Best regression using

information criteria. Journal of Modern Statistical Meth-ods, 1(2), 479–488.

Global Reach. (2003). Global Internet statistics by language[Online]. http://www.glreach.com/globstats/

Hamill, J., & Gregory, K. (1997). Internet marketing in theinternationalization of UK SMEs. Journal of MarketingManagement, 13, 9–28.

Hendrickson, A. R., & Collins, M. R. (1996, Spring). Anassessment of structure and causation of IS usage. TheDatabase for Advances in Information Systems, 1–67.

Häubl, G., & Trifts, V. (2000). Consumer decision making inonline shopping environments: The effects of interactivedecision aids. Marketing Science, 19(1), 4–21.

Igbaria, M., Zinatelli, N., Cragg, P., & Cavaye, A. L. M. (1997,September). Personal computing acceptance factors insmall firms: A structural equation model. MIS Quarterly,279–301.

Jensen, J. F. (1998). Interactivity: Tracing a new concept inmedia and communication studies. Nordicom Review, 19,185–204.

Jeong, M., & Lambert, C. U. (2001). Adaptation of an infor-mation quality framework to measure customers’ behav-ioral intentions to use lodging Web sites. Hospitality Man-agement, 20(2), 129–146.

Jeong, M., Oh, H., & Gregoire, M. (2003). ConceptualizingWeb site quality and its consequences in the lodging in-dustry. International Journal of Hospitality Management,22(2), 161–175.

Johnson, K. L., & Misic, M. M. (1999). Benchmarking: Atool for Web site evaluation and improvement. InternetResearch: Electronic Network Applications and Policy,9(5), 383–392.

Jung, T. H., & Butler, R. (2000). Perceptions of marketingmanagers of the effectiveness of the Internet in tourismand hospitality. Information Technology and Tourism, 3(3/4), 167–176.

Kucuk, S. U., & Arslan, M. (2000). A cross cultural com-parison of consumers’ acceptance of the Web marketingfacilities. Journal of Euromarketing, 9(3), 27–43.

Lederer, A. L., Maupin, D. J., Sena, M. P., & Zhuang, Y.(2000). The technology acceptance model and the WorldWide Web. Decision Support Systems, 29(2), 269–282.

Loban, S. (1998). Effective documents for tourist informationsystems. Information Technology & Tourism, 1, 59–71.

Lucas, H. C., & Nielson, N. R. (1980). The impact of themode of information presentation on learning and per-formance. Management Science, 26(10), 982–993.

Marcussen, C. H. (2003). Trends in European Internet dis-tribution of travel and tourism services [Online]. ResearchCentre of Bornholm, Nexo, Denmark. http://www.crt.dk/uk/staff/chm/trends.htm

McMillan, S. J. (2000). The microscope and the moving tar-get: The challenge of applying content analysis to theWorld Wide Web. Journalism and Mass CommunicationQuarterly, 77(1), 80–98.

Mitra, A. (1999). Characteristics of the WWW text: Tracingdiscursive strategies. Journal of Computer-MediatedCommunication, 5(1). http://www.ascusc.org/jcmc/


Moorthy, S., Ratchford, B. T., & Talukdar, D. (1997). Con-sumer information search revisited: Theory and empiri-cal analysis. Journal of Consumer Research, 23, 263–277.

Morrison, A. M., & Morrison, A. (2000). Suggestions forWorld Wide Web design and development for destinationmarketing organizations [Online]. http://omni.cc.purdue.edu/~alltson/

Murray, J. H. (1997). Hamlet on the holodeck: The future ofnarrative in cyberspace. New York: Free Press.

Novak, T. P., & Hoffman, D. L. (1997). New metrics for newmedia: Toward the development of Web measurementstandards. World Wide Web Journal, 2(1). http://www.w3j.com/5/s3.novak.html

Novak, T. P., Hoffman, D. L., & Yung, Y.-F. (2000). Measur-ing the customer experience in online environments: Astructural modeling approach. Marketing Science, 19(1),22–42.

Oertel, B., Thio, S. L., & Feil, T. (2001). Benchmarking tour-ism destinations in the European Union. In P. Sheldon,K. Wöber, & D. Fesenmaier (Eds.), Information and Com-munication Technologies in Tourism. ENTER 2001. Pro-ceedings of the International Conference (pp. 234–243),Montreal, Canada.

Olsina, L., Godoy, D., Lafuente, G. J., & Rossi, G. (1999).Specifying quality characteristics and attributes forWebsites. First ICSE Workshop on Web Engineering(WebE-99), Los Angeles.

PhoCusWright. (2000). The European online travel market-place, 2000–2002. Sherman: Author.

PhoCusWright. (2002). Hotel & lodging commerce 2002–2005: Distribution strategies and market forecasts.Sherman: Author.

Pröll, B., & Retschitzegger, W. (2000). Discovering nextgeneration tourism information systems: A tour onTIScover. Journal of Travel Research, 39(2), 182–191.

Roche, X.; et al. (2004). HTTrack Website copier—opensource offline browser, Documentation. http://www.httrack.com/html/

Rogers, E. M. (1995). The diffusion of innovations (4th ed.).New York: Free Press.

Scharl, A. (2000). Evolutionary Web development. London:Springer. http://webdev.wu-wien. ac.at/

Scharl, A. (2004). Web coverage of renewable energy.Envionmental online communication (pp. 25–34). Lon-don: Springer. [Online]. http://www.ecoresearch.net/springer.

Schegg, R., Steiner, T., Frey, S., & Murphy, J. (2002). Bench-marks of Web site design and marketing by Swiss hotels.Information Technology & Tourism, 5(2), 73–89.

Schonland, A., & Williams, P. W. (1996). Using the Internetfor travel and tourism survey research: Experiences fromthe Net Traveler Survey. Journal of Travel Research,35(2), 81–87.

Szajna, B. (1996). Empirical evaluation of the revised tech-nology acceptance model. Management Science, 42(1),85–92.

Teo, T. S. H., Lim, V. K. G., & Lai, R. Y. C. (1999). Intrinsicand extrinsic motivation in Internet usage. Omega, 27,25–37.

Taylor, S., & Todd, P. (1995). Assessing IT usage: The roleof prior experience. MIS Quarterly, 19(4), 561–570.

Tierney, P. (2000). Internet-based evaluation of tourism Website effectiveness: Methodological issues and survey re-sults. Journal of Travel Research, 39, 212–219.

Venkatesh, V., & Davis, F. D. (2000). A theoretical extensionof the technology acceptance model: Four longitudinalfield studies. Management Science, 46, 186–204.

webLyzard. (2004). [Online]. http://www.weblyzard.com/technology.html

Werthner, H., & Klein, S. (1999). Information technologyand tourism—a challenging relationship. Wien/New York:Springer.

Whyte, G., & Bytheway, A. (1996). Factors affecting infor-mation systems’ success. International Journal of Ser-vice Industry Management, 7(1), 74–93.

Wierenga, B., Van Bruggen, G. H., & Staelin, R. (1999). Thesuccess of marketing management support systems. Mar-keting Science, 18(3), 196–207.

Wöber, K. W., & Gretzel, U. (2000). Tourism managers’ adop-tion of marketing decision support systems. Journal ofTravel Research, 39, 172–181.