BISE – STATE OF THE ART

Preserving Knowledge on IS Business Value

What Literature Reviews Have Done

Based on a comprehensive literature search, this meta review analyzes to what extent pastliterature reviews on IS business value have covered key research areas and preserved theirkey findings. The results show that while some areas have been explored extensively, someother crucial areas, such as accounting performance, the growth of intangible assets, andthe differentiation between economic output and derived or perceived value, have beenneglected. They need to be considered in future reviews. The results also reveal thoseresearch areas where even primary research is weak and needs to get intensified beforeliterature reviews can be applied to synthesize findings.

DOI 10.1007/s12599-010-0111-y

The Author

PD Dr. Guido Schryen M.O.R. (�)Information Systems ResearchInstitut für AllgemeineWirtschaftsforschungAlbert-Ludwigs-Universität FreiburgKollegiengebäude IIPlatz der alten Synagoge79085 FreiburgGermanyschryen@is.uni-freiburg.deurl: http://www-users.rwth-aachen.de/guido.schryen/

Received: 2009-05-29Accepted: 2010-01-25Accepted after two revisions by Prof.Dr. Buxmann.Published online: 2010-07-06

This article is also available in Ger-man in print and via http://www.wirtschaftsinformatik.de: Schryen G(2010) Ökonomischer Wert von In-formationssystemen. Beitrag von Li-teratur-Reviews zum Wissenserhalt.WIRTSCHAFTSINFORMATIK. doi: 10.1007/s11576-010-0232-4.

Electronic Supplementary MaterialThe online version of this article(doi: 10.1007/s12599-010-0111-y)contains supplementary material,which is available to authorizedusers.

© Gabler Verlag 2010

1 Introduction

Information systems (IS) started to beembedded in economic environmentsmany decades ago and are even consid-ered commodity inputs nowadays (Carr2003). The reliance on IS has mean-while occurred to an extent that forsome firms the failure of IS impedes oreven renders business activities impossi-ble. IS have also gained macroeconomicimportance: according to the World In-formation Technology Services Alliance(WITSA 2008, p. 1), the global market-place for information and communica-tion technology is likely to have topped$3.7 trillion in 2008. The economic rel-evance of IS has made research on “ISbusiness value” highly attractive to re-searchers, who have shaped the academicdiscussion by publishing an abundance ofresearch papers, according to the litera-ture reviews analyzed in this paper.

Some researchers provide sobering ar-guments on the economic relevance of IS.For example, West and Courtney (1993)and Hitt and Brynjolfsson (1996) doubtthe strategic power of IS, and argue thatIS are commodities and that any IS-based advantages will be soon eroded.Carr (2003) sums up doubts by even en-titling his paper “IT doesn’t matter”. An-other discourse is rooted in empiricalstudies that do not find evidence that ISpositively affect performance (Dos San-tos et al. 1993; Rai et al. 1997; Im et al.2001; Dehning and Stratopoulos 2002;Ko and Osei-Bryson 2004; Stiroh andBotsch 2007). Apparently, IS researchershave (at least not fully) managed to iden-

tify and to explain the economic rele-vance of IS. Business executives and re-searchers continue to question the valueof IS investments, as Kohli and Grover(2008, p. 23) note in their recent review.However, answering this question is re-garded fundamental to the contributionof the IS discipline (Agarwal and Lucas2005).

A straightforward approach to revealIS business value is to synthesize empir-ical findings of the literature. However,the large number of studies is accompa-nied by a variety of methods, researchobjects, research models, and findings.The discussion of IS business value hasreached a high level of complexity, whichmakes it extremely difficult to overlookkey research findings. This complexityhas been addressed by researchers whopublished literature reviews in as many as15 different outlets, including such perti-nent journals as MISQ, ISR, JMIS, EJIS,CACM, JAIS, and ACM Computing Sur-veys. In the presence of the aforemen-tioned critics on IS, the question arisesto what extent knowledge on IS busi-ness value has been preserved throughprior reviews. This leads us to the re-search question of this paper:

To what extent have past reviews ad-dressed or neglected key areas in IS businessvalue research?

The importance of this question isleveraged by the argument of Kohli andGrover (2008) who hypothesize that pastresearch on IS business value has eitherdisregarded or underemphasized increas-ingly important research areas and ques-tions.

Business & Information Systems Engineering 4|2010 233

BISE – STATE OF THE ART

The overall goal of this work is to an-swer the research question by (a) identi-fying and describing central findings inkey research areas of IS business valueresearch, and (b) synthesizing what lit-erature reviews have done to preserveknowledge. Through the methodologicallens, this paper is a review of literature re-views, and thus a “meta review”. Thereby,it differs from a recently published reviewon the value of information systems (Ur-bach et al. 2009) in the research methodand in the objects under investigation.

The remainder of this paper is orga-nized as follows: Sect. 2 provides the the-oretical background of IS business valueresearch and literature review method-ology. In Sect. 3, the research frame-work and the methodology of this pa-per are presented. Section 4 uses a taxon-omy to condense main fields in IS busi-ness value research. Section 5 analyzes towhat extent the research fields have beenaddressed in literature reviews. Finally,Sect. 6 concludes this article and presentsspecific high-priority recommendationsfor future research.

2 Theoretical Background

From a methodological point of view, ameta (literature) review is a particulartype of review and can thereby draw onreview methodology. This section drawson this methodology and follows the rec-ommendation of Webster and Watson(2002, p. xv), who suggest that a reviewpaper should provide elaborate defini-tions of key variables of the review andshould set the boundaries on the review.In this paper, key variables are “informa-tion systems” and “IS business value”.

2.1 Information Systems (IS)

The academic field of IS is terminologi-cally pervaded by the usage of syntacti-cally similar notions, such as “informa-tion system (IS)”, “information technol-ogy (IT)” and “information and com-munication technology (ICT)”. However,these notions often lack any precise se-mantic definitions. Reviewing articlespublished in “Information Systems Re-search”, Orlikowski and Iacono (2001)find that the “IT artifact” has not beentheorized and is widely interpreted de-pending on the specific research con-text. The notional fuzziness and hetero-geneous semantics in literature is not sur-prising, because information systems dis-

cipline does not yet provide a broadly-accepted or even standardized ontology.In this review, we adopt the “holistic”view on IS, as described in the ATIS Tele-com Glossary (ATIS 2007) (option 3):“The entire infrastructure, organization,personnel, and components for the collec-tion, processing, storage, transmission, dis-play, dissemination, and disposition of in-formation.”

2.2 IS Business Value

We frame IS business value research bydefining notion and scope and the level,object and time of evaluation.

IS literature offers a variety of no-tions and semantics. For example, earlyworks use the notions “value”, “benefit”,“outcome” or “worth” (Wiseman 1992),Melville et al. (2004) investigate “orga-nizational performance”, and Kohli andGrover (2008) refer to value as the “eco-nomic impact”. This variety in terminol-ogy does not only mirror notional incon-sistencies, it also reflects different under-standings of how to operationalize theeconomic impact of IS. For example, alarge subset of empirical studies applyeconometric approaches by analyzing therelationship between IS investments andeconomic variables, such as productiv-ity (Brynjolfsson and Hitt 1996), “Returnon Sales” (Bharadwaj 2000), or Tobin’sq (Brynjolfsson and Yang 1999). Otherstudies stress that, beyond financial andnon-financial measures, intangible assetscan be affected by IS investments (Irani2002; Kohli and Grover 2008). The dis-cussion becomes even more complicatedwhen researchers also distinguish be-tween what the particular outcome of anIS investment is and how this outcome isinterpreted. The interpretation of a par-ticular outcome depends on the view ofthe particular evaluator (Sylla and Wen2002, 242), on what competitors haveachieved (Dehning and Richardson 2002,23), and what is finally done to exploit it(Alshawi et al. 2003, p. 419). As this re-view is dedicated to the identification ofuncharted territories in IS business valueresearch, it does not exclude any of theaforementioned facets. Rather, they areused to structure research findings.

Literature suggests different levels forthe examination of the economic impactof IS. A widely used classification distin-guishes individual level, firm level, indus-try level and economy level (Bakos 1987;Kauffman and Weill 1989; Brynjolfssonand Yang 1996; Devaraj and Kohli 2000;

Chau et al. 2007). In addition, researchalso analyzes consumer surplus (Bakos1987; Brynjolfsson and Yang 1996; De-varaj and Kohli 2000). This work doesnot exclude any of these levels.

Consistent with the holistic definitionof IS adopted in this paper, we address theeconomic impact of investments in infor-mation technology, in organizational as-sets, and in personnel.

As Kohli and Grover (Kohli and Grover2008, p. 25) stress, research on IS valuecan be of “ex ante” and “ex post” nature.While “ex ante” research is closely relatedto decision making, “ex post” research isdedicated to the control of past expenses.This work includes both streams of re-search.

3 Research Design andMethodology

The methodology used in this paper isbased on the theoretical research frame-work shown in Fig. 1. The bold rectan-gles and arrows indicate those parts thatare focused in this work.

“Literature review” is an establishedresearch methodology (Salipante et al.1982; Cooper and Hedges 1994; White1994). It is of particular importance forIS research, as stressed by Webster andWatson (2002, p. xiii f), who argue thatthe literature review “[. . . ] facilitates the-ory development, closes areas where aplethora of research exists, and uncoversareas where research is needed. [. . . ][T]heliterature review represents the foundationfor research in IS. As such, review arti-cles are critical to strengthening IS as afield of study.” The relevance of litera-ture reviews has also been addressed inrenowned IS journals. For example, sev-eral years ago “MIS Quarterly” launchedits “MISQ Review Department” (Watson2001), a unit dedicated to the publica-tion of literature reviews. Another exam-ple is the journal “WIRTSCHAFTSIN-FORMATIK”, which publishes this liter-ature review in its “State-of-the-Art” col-umn. The journals “European Journal ofInformation System” and the “Journal ofManagement Information Systems” areexamples of renowned journals that ex-plicitly include review papers and surveysin their scope of invited contributions.

Apparently, literature reviews arean appreciated and highly importantmethodology in IS research. This pa-per draws on this importance twofold: itanalyzes those reviews that address the

234 Business & Information Systems Engineering 4|2010

BISE – STATE OF THE ART

Fig. 1 Methodological research framework

domain “IS business value”, and it applies(meta) review methodology by itself.

Meta review methodology is still inits infancy in terms of methodology andapplication. However, a straightforwardapproach is to apply “review methodol-ogy”, which can be regarded as a method-ological generalization of “meta reviewmethodology”. Thus, we apply “reviewmethodology” and draw upon the workof Webster and Watson (2002). They par-ticularly stress the importance of identi-fying relevant literature and structuringthe review.

We performed a title search in per-tinent journal databases, namely Busi-ness Source Premier, MLA Interna-tional Bibliography, EconLit, ScienceDi-rect, IEEE Xplore, the ACM Digital Li-brary, and Web of Science. The logicalsearch string was (“information technol-ogy” OR “information systems”) AND(“value” OR “investment” OR “produc-tivity” OR “competitive” OR “perfor-mance” OR “measurement” OR “evalu-ation” OR “profit” OR “efficiency”). Wefurther scanned the table of contents ofthe following journals (listed in alpha-betical order): Academy of ManagementReview, ACM Transactions on Informa-tion Systems, American Economic Re-view, Communications of the ACM, Eu-ropean Journal of Information Systems,Information Systems Journal, Informa-tion Systems Research, Journal of Man-agement Information Systems, Journalof the AIS, Management Science, MISQuarterly, and Wirtschaftsinformatik.

Regarding the structure of the review,we apply a concept-centric approach,with research fields being the concepts.More specifically, we adapt the matrix

approach of Salipante et al. (1982) by tab-ulating review articles against research ar-eas (concepts).

This article considers 22 literature re-views on IS business value research,which have been published since 1989 inpeer-reviewed journals or peer-reviewedconference proceedings. More specifi-cally, we use the following reviews, whichare listed in chronological order anddescribed in detail in the appendix:(Kauffman and Weill 1989; DeLone andMcLean 1992; Brynjolfsson 1993; Sohand Markus 1995; Brynjolfsson and Yang1996; Potthof 1998; Sircar et al. 1998;Seddon et al. 1999; Bannister and Re-menyi 2000; Chan 2000; Devaraj andKohli 2000; Dehning and Richardson2002; Irani and Love 2002; Sylla and Wen2002; Dedrick et al. 2003; Melville et al.2004; Walter and Spitta 2004; Piccoli andIves 2005; Chau et al. 2007; Wan et al.2007; Kohli and Grover 2008; Pare et al.2008).

In order to avoid confusion between“research papers” and “literature re-views”, it should be noticed that we usethe findings of research papers to definekey research fields in Sect. 4. Literaturereviews are used in Sect. 5 in order to an-alyze the extent to which these fields havebeen covered in literature reviews.

4 Key Areas of IS Business ValueResearch

The literature on IS business value pro-vides a variety of taxonomies, which arerooted in different perspectives of the au-thors. For example, DeLone and McLean(1992) analyze the dependent variable

and suggest categories of IS success, Sed-don et al. (1999) provide a taxonomy thataccounts for the type of IS asset used anddifferent stakeholders, and Irani and Love(2002) focus on IS investment evalua-tion methodology and provide a taxon-omy of investment appraisal techniques.As the goal of this paper is to provide abroad picture of concepts in IS businessvalue research, we do not focus on a sin-gle perspective or taxonomy. We ratheridentify those dimensions that are widelyadopted in the literature. Finally we usethese dimensions to shape the taxonomyon which our meta literature review isbased upon.

We find broad consensus in the liter-ature that important dimensions of ISbusiness value are “performance mea-sure” (DeLone and McLean 1992; Baruaet al. 1995; Dehning and Richardson2002; Melville et al. 2004; Chau et al.2007), the “level of measurement” (Bakos1987; Brynjolfsson 1993; Dehning andRichardson 2002; Pare et al. 2008), the“type of IS asset” (Weill 1992; Mah-mood and Mann 1993; Rai et al. 1997;Seddon et al. 1999; Sircar et al. 2000;Melville et al. 2004), “methods” (Chan2000; Irani and Love 2002; Chau et al.2007; Pare et al. 2008), and “influenc-ing factors” (contextual factors, lag ef-fects, risk) (Weill and Olson 1989; Baruaet al. 1995; Davern and Kauffman 2000;Stiroh 2002; Ko and Osei-Bryson 2004;Melville et al. 2004; Dewan et al. 2007).While these dimensions address the mea-surement of IS performance, researchersalso stress the importance of question-ing what the value of a particular perfor-mance is (Dehning and Richardson 2002;Alshawi et al. 2003). Thus, we add the di-mension “value” to our taxonomy, whichis shown in Fig. 2.

We describe each of these dimensionsin the following subsections and derivekey research fields. Prior to applyingthis procedure, we explain first why weconsider “terminology” an additional re-search field.

4.1 Terminology

For each academic discipline, a consistentterminology is essential to name relevantconstructs, to define its semantics and toresolve potential ambiguities. However,the discussion in Sect. 2 already revealedsome confusion in IS literature. The im-portance of clearly defining the subject ofresearch is pinpointed by Orlikowski andIacono (2001, p. 121): “[. . . ] we propose

Business & Information Systems Engineering 4|2010 235

BISE – STATE OF THE ART

Fig. 2 Taxonomy of ISbusiness value research

that IS researchers begin to theorize specif-ically about IT artefacts, and then incorpo-rate these theories explicitly into their stud-ies.” We define “Research field 1: Terminol-ogy”.

4.2 Performance Measure

Researchers have analyzed a variety ofeconomic measures, such as productivity(Brynjolfsson and Hitt 1996, 2000), pro-duction efficiency (Thatcher and Oliver2001), consumer welfare (Thatcher andPingry 2004), profit ratios (Weill 1992;Barua et al. 1995), and also market-oriented measures (Bharadwaj et al. 1999;Brynjolfsson and Yang 1999). The abun-dance of different aspects of IS successis addressed by researchers who providetaxonomies to organize the diverse re-search (DeLone and McLean 1992; Iraniand Love 2002; Gable et al. 2008). A sim-ple and often applied classification dis-tinguishes between process performanceand firm performance, which subsumesmarket performance and accounting per-formance (Barua et al. 1995; Dehningand Richardson 2002; Melville et al.2004). It is widely agreed that the impactof IS investments on firm performanceis intermediated by process performance(Barua et al. 1995; Soh and Markus 1995;Dehning and Richardson 2002; Kim et al.2006; Mittal and Nault 2009).

Among process performance measures,productivity is most intensively dis-cussed. Some early studies in the late1980s and early 1990s did not find thatIS considerably contributed to produc-tivity and economic growth at economylevel (Baily 1986; Roach 1987; Jorgensonand Stiroh 1995), at industry level (Roach1991; Berndt and Morrison 1995), or atfirm level (Loveman 1994). One impactof these studies was the creation of theterm “productivity paradoxon”. However,with IS becoming a larger share of totalcapital investment (Dedrick et al. 2003,

p. 19), more recent studies find a ma-jor impact of IS investments on pro-ductivity and economic growth in de-veloped countries (Jorgenson and Stiroh2000; Oliner and Sichel 2000; Jorgen-son 2001). At firm level, the pictureseems to be less clear: While some stud-ies (Ko and Bryson 2002; Ko and Osei-Bryson 2004; Lin and Shao 2006b) donot find any evidence of a positive cor-relation or suggest a microeconomic ex-planation (Stickel 1995), opposite resultsare reported by Brynjolfsson and Hitt(1996, 2000), Kelley (1994), Lin and Shao(2006a), Neirotti and Paolucci (2007),Menon et al. (2000), Stiroh (2002), andSwierczek and Shrestna (2003). Thisleads to the definition of “Research field2: Productivity”.

Researchers have shown their inter-est to analyze to what extent IS in-vestments are correlated with increased(stock) market performance of firms.Tam (1998) and Brynjolfsson and Hitt(1996) investigate the impact on “TotalShareholder Return”, Dos Santos et al.(1993) and Im et al. (2001) analyze stockmarket reactions, and Bharadwaj et al.(1999) and Brynjolfsson and Yang (1999)focus on Tobin’s q. Although some stud-ies find a positive correlation, Dedrick etal. (2003, p. 10) argue that this corre-lation is of purely temporal nature, butlacks any causal characteristics, as manymore micro- and macro-economic fac-tors determine market performance. Onthe other hand, Brynjolfsson and Yang(1999) and Brynjolfsson et al. (2002) sug-gest that adjustment costs and intangi-ble assets may provide an explanation forthe high market valuation found for IS.Bharadwaj et al. (2009) adopt the oppo-site perspective by analyzing the effectsof information technology failures on themarket value of firms. Their results revealthat the market responds negatively to ISfailures. To conclude, we define “Researchfield 3: Market performance”.

The impact of IS investments on ac-counting performance in terms of costratios, turnover ratios and profit ratiosis one the most intensively studied re-search areas in IS business value research.Cost ratios are analyzed by Bharadaj(2000) and Santhanam and Hartano(2003). Turnover ratios are investigatedin the studies of Dehning and Stratopou-los (2002) and Barua (1995). Manystudies address profit ratios: IS invest-ments seem to positively affect “Returnon Sales” (Tam 1998; Bharadwaj 2000;Dehning and Stratopoulos 2002; San-thanam and Hartono 2003) and “Oper-ating income to employees” (Bharadwaj2000; Santhanam and Hartono 2003),while the positive impact on “Return onAssets” (Hitt and Brynjolfsson 1996; Raiet al. 1997; Tam 1998; Bharadwaj 2000;Stratopoulos and Dehning 2000; Dehn-ing and Stratopoulos 2002; Santhanamand Hartono 2003), “Return on Invest-ment” (Stratopoulos and Dehning 2000;Hayes et al. 2001; Mahmood and Mann2005), and “Return on Equity” (Alparand Kim 1990; Rai et al. 1997; Tam 1998;Stratopoulos and Dehning 2000) is lessclear. We define “Research field 4: Ac-counting performance”.

While the aforementioned perfor-mance measures address tangible ben-efits, the importance of intangible ben-efits, such as increased capabilities andknowledge at organizational level, orbetter decision making, has often beenacknowledged (Mertens et al. 1982;Soh and Markus 1995; Brynjolfsson andHitt 2000; Irani and Love 2001) andwas recently re-emphasized by Kohli andGrover (2008). Bhatt and Grover (2005)even argue that the quality of IS businessexpertise can form capabilities that havea significant effect on competitive advan-tage. However, only few research papersaddress intangible benefits. To sum up,we define “Research field 5: Intangiblebenefits”.

236 Business & Information Systems Engineering 4|2010

BISE – STATE OF THE ART

4.3 Level of Measurement

Literature suggests different levels forthe examination of the economic im-pact of IS (see Sect. 2.2). Several stud-ies limit their investigations to a par-ticular level. For example, Brynjolfssonand Hitt (1996, 2000) and Mahmood andMann (2005) focus on firm level, Shih etal. (2007) adopt a macro-economic viewat country-level, and Devaraj and Kohli(2000), Brynjolfsson (1996) and Hitt andBrynjolfsson (1996) analyze consumersurplus created by IS investments. Theimportance of taking the level of exami-nation into account is stressed by Dehn-ing and Richardson (2002, p. 8) and byBrynjolfsson (1993), who states that theusage of different levels even contributesto the explanation of the productivityparadoxon. Apparently, the separation ofdifferent levels is useful to structure re-search and to resolve allegedly conflictingresults. But it is also argued that, beyondthe separation of levels, their linkage canprovide useful insights and explanationsof how IS generates value (DeLone andMcLean 1992; Kohli and Grover 2008).We define “Research field 6: Level of mea-surement”.

4.4 Type of IS Asset

It has been widely argued in the lit-erature that better insights in the wayIS investments induce superior businessperformance require a breakdown of ISinvestments into single IS assets (Weill1992; Mahmood and Mann 1993; Rai etal. 1997; Sircar et al. 2000; Melville etal. 2004). IT capital-related studies (Hittand Brynjolfsson 1994; Barua et al. 1995;Rai et al. 1997; Tam 1998; Sircar et al.2000; Mahmood and Mann 2005) find nocorrelation with stock market behavior,mixed results regarding profitability ra-tios, and a positive correlation with prof-itability in terms of “sales” and “valueadded”. Some studies (Kelley 1994; Raiet al. 1997) are even more specializedand analyze the impact of hardware ex-penditures or expenditures based on in-vestments in software (Rai et al. 1997),production-oriented software (Barua etal. 1995), interorganizational informa-tion systems (Schumann 1990), ERP sys-tems (Poston and Grabski 2000; Hayes etal. 2001; Karimi et al. 2007), e-commercesystems (Subramani and Walden 2001),supply chain systems (Kim et al. 2006),knowledge management systems (Maierand Hädrich 2001) or infrastructure (Rai

et al. 1997; Byrd and Turner 2000; Chat-terjee et al. 2002). The studies differ enor-mously in methods, data, time period,and indicators used. This conclusion alsoapplies to studies that are related to ISpersonnel and training expenditures (Sir-car et al. 2000; Chatterjee et al. 2001;Mahmood and Mann 2005). We define“Research field 7: Type of IS asset”.

4.5 Methods

Studies of decision practice indicate thatmanagers often avail themselves of rel-atively simplistic cost-benefit analysis inthe context of traditional capital budget-ing (Bannister and Remenyi 2000; Iraniand Love 2002; Chau et al. 2007). How-ever, beyond traditional capital budget-ing, many more approaches have beenproposed, such as those related to mea-suring accounting or market-based mea-sures (see the discussion above). Theportfolio of proposed methods also in-cludes value analysis (Money et al. 1988)and analysis based on critical success fac-tors (CSF) (Rockart 1979). Overall, theliterature on performance measurementprovides a plethora of different appraisalmethods (Bannister and Remenyi 2000,p. 232). To sum up, we identify “Researchfield 8: Methods”.

4.6 Influencing Factors

It is widely argued in the literature thatthe impact of IS investments on eco-nomic performance is influenced by non-technological factors. Mostly discussedare factors related to economic structures(contextual factors), lag effects, or risk.We briefly discuss each of them.

Contextual factors comprise firm, in-dustry, and economic factors. They havebeen found to affect the economic impactof IS investments (Weill 1992; Bharadwaj2000; Davern and Kauffman 2000; Dehn-ing and Richardson 2002; Ko and Osei-Bryson 2004; Melville et al. 2004; Zhu etal. 2004). Most studies focus on firm fac-tors (Floyd and Wooldridge 1990; Li andYe 1999; Ravichandran and Lertwongsa-tien 2005; Chari et al. 2008). These stud-ies strongly suggest that (a) the align-ment of IS with a firm’s core competen-cies and business planning and (b) closeties between IS investments and uppermanagement are crucial for IS-drivenenhanced firm performance. Competi-tive factors are addressed in the worksof Lin and Shao (2006b), Sircar et al.

(2000), and Melville et al. (2007), macro-environmental factors are analyzed in thecontributions of Swierczek and Shrestha(2003) and Zhu et al. (2004). We define“Research field 9: Contextual factors”.

It is argued in the literature that a mis-measurement of IS investment impactmay be rooted in inappropriate method-ology, when delayed effects need to beconsidered, but are ignored (Weill andOlson 1989; Stiroh 2002). Some empir-ical studies (Santhanam and Hartono2003; Mahmood and Mann 2005) ac-count for this criticism and find thatlags may exist and that several years maypass before an organization’s investmentin IT bears fruit. We consider this phe-nomenon by defining “Research field 10:Lag effects”.

As in the case of many other invest-ments, IS investments bear economicrisks due to the uncertainty of futureand states (McFarlan 1981; Wehrmannet al. 2006). IS investments are regardedeven substantially riskier than non-IS in-vestments, as measured by their relativecontributions to the overall riskiness ofthe firm (Dewan et al. 2007, p. 1829).The (ex ante) evaluation of IS invest-ments is also based on personal expec-tations and risk preferences of decisionmakers (Rose et al. 2004, p. 53). Risk inIS investment decisions is explicitly con-sidered in the papers of Au and Kauff-man (2003), Wehrmann and Zimmer-mann (2005), Wehrmann et al. (2006),Benaroch et al. (2007), and Dewan et al.(2007). As risk is deemed a substantialcomponent of IS investment decisions,we define “Research field 11: Risk”.

4.7 Value

While the economic performance of ISinvestments is usually determined bymeasuring and comparing economic ra-tios, some researchers started question-ing what the value of a particular out-come is. It is argued that the actual valueof an outcome may depend on whatis done with newly generated capabili-ties (Alshawi et al. 2003, p. 419), whatcompetitors have achieved (Dehning andRichardson 2002, p. 23), and what thesubjective preferences of the persons whoperform the evaluation are (Sylla andWen 2002, p. 242).

The distinction between what is mea-sured and how this outcome is finally val-ued has already been substantiated in de-cision theory and utility theory, which

Business & Information Systems Engineering 4|2010 237

BISE – STATE OF THE ART

distinguish between the result of mea-surement (referred to as “outcome”) andthe perceived value.

One of the most intensively discussedtypes of IS value is competitive advan-tage. It is argued that competitive ad-vantage can only be gained if firms ap-ply strategic information management(Zahn 1990), and if IS-based capabili-ties are pretended from being imitated bycompetitors (Feeny and Ives 1990; Carr2003) or if competitors do not fully ben-efit from imitation (Clemons and Row1991). West and Courtney (1993, p. 245)note that any advantage of innovationwill be eroded as the technology becomescommon practice. Hitt and Brynjolfs-son (1996) find that, although IS invest-ments do not lead to competitive ad-vantage, they are necessary to maintaincompetitive parity. An even more posi-tive picture is drawn by Bhatt and Grover(2005), who find evidence in their em-pirical study that the quality of IT busi-ness expertise and the relationship infra-structure have significant effect on com-petitive advantage. Fink and Neumann(2009) show that IS personnel knowl-edge and skills positively affect the rangeof managerial IS infrastructure capabil-ities, which in turn are responsible forperceived competitive impacts. To sumup, the value of IS represents an im-portant research question. It shapes “Re-search field 12: Value”.

5 Analysis

This section analyzes to what extent theresearch areas identified in the previoussection have been addressed in literaturereviews. Table 1 provides an overview ofthe results.

5.1 Terminology

Two reviews (Kauffman and Weill 1989;Melville et al. 2004) briefly investigate lit-erature regarding terminology. The workof Kauffman and Weill (1989), whichreveals inconsistent definitions of inputand output variables, embraces a veryearly period in IS business value research.However, 15 years later we are informedby Melville et al. (2004) that the IS com-munity has still divergent perspectives onthe IT construct, which depend on thespecific context of research. Although thework of Melville et al. (2004) providesonly a brief overview of terminology andperspectives, it is an excellent startingpoint for future literature reviews.

5.2 Productivity

The large interest of researchers in ex-ploring the impact of IS investments onproductivity is also mirrored in the num-ber of literature reviews that address pro-ductivity. DeLone and McLean (1992)provide an early overview on productiv-ity studies. A comprehensive review isconducted by Brynjolfsson (1993), whoconclude that the alleged productivityparadoxon is much due to deficienciesin measurement and methodology, moreprecisely in mismeasurement of inputsand outputs, lags due to learning and ad-justment, redistribution and dissipationof profits, and mismanagement of infor-mation and technology. A further defi-ciency is identified by Sircar et al. (1998),who conclude that many studies thatclaim to inspect productivity rather mea-sure firm performance. Interestingly, Sir-car et al. (1998) also find that the under-lying theory impacts results: while studiesbased on variance theory refute the pro-ductivity paradoxon, those based on pro-cess theory support it.

Brynjolfsson and Yang (1996) prefer toconduct productivity research at firm-level because this helps to control manyproblems from aggregation occurring atindustry level. Overall, they find a pos-itive effect on productivity reported inrecent literature. This conclusion is sup-ported in the more recent reviews of De-varaj and Kohli (2000) and Dedrick etal. (2003), who also admit that the im-pact varies widely among different com-panies. According to Wan et al. (2007),the productivity paradoxon has been re-solved at firm level due to more sophisti-cated and refined data sources, a shift inthe level of analysis towards the firm level,and a refocus on the management of IS.They argue that research has probablybetter accounted for the four problemscited by Brynjolfsson (1993). At the in-dustry level, results are less clear. Devarajand Kohli (2000) find mixed results in theliterature, and Dedrick et al. (2003) iden-tify some positive returns in the form oflabor productivity. Reviewing productiv-ity at economy level, early studies failedto identify positive effects of IS invest-ments. However, in the 1990s more pos-itive results occurred (Brynjolfsson andYang 1996) and seven years later Dedricket al. (2003) find that literature has showna positive relationship between IS invest-ments, growth and national productivity,at least in developed countries.

Literature reviews on productivity haveprovided excellent overviews of produc-tivity at different levels and have synthe-sized the findings of research papers re-garding the question of whether IS invest-ments led to increased productivity ornot. However, this perspective does notallow explaining why the impact differsso much and resolving the conflicting re-sults of studies that found positive resultsand those that did not.

5.3 Market Performance

Literature provides some studies that finda positive correlation of IS investmentsand market performance and that IShave a mediated impact (Brynjolfssonand Yang 1999; Brynjolfsson and Hitt2000). Although there is only few litera-ture available, which makes “market per-formance” less attractive for literature re-views, two early reviews (DeLone andMcLean 1992; Dehning and Richardson2002) analyze literature findings. The re-view of DeLone and McLean (1992) iden-tifies few empirical studies only. The re-view of Dehning and Richardson (2002,p. 19) concludes that market values in-crease by 5 to 20 times the amount spenton IS and that shareholders value strate-gic IS investments. However, since 2003the interest in the investigation of the im-pact of IS investments on market perfor-mance has declined and there is almostno recent research papers to get reviewed.

5.4 Accounting Performance

Interestingly, the abundance of empiricalstudies on accounting ratios has been ad-dressed in detail by two literature reviewsonly. While DeLone and McLean (1992)find too few studies to draw an overallpicture, Dehning and Richardson (2002)find that the relation between IS spend-ing and accounting performance is tenu-ous. However, in contrast to market per-formance, accounting performance con-tinues to attract researchers’ interest (see,for example, the study of Mahmood andMann 2005).

5.5 Intangible Benefits

Although intangible benefits have beenaddressed in research papers only rarely,several literature reviews acknowledgethat the benefit of IS investments en-closes intangibles (DeLone and McLean1992; Soh and Markus 1995; Devaraj andKohli 2000) and that IS is an enabler

238 Business & Information Systems Engineering 4|2010

BISE – STATE OF THE ART

Tabl

e1

Cove

rage

of“IS

busi

ness

valu

e”re

sear

char

eas

inlit

erat

ure

revi

ews

Dim

ensi

onR

esea

rch

fiel

dL

iter

atu

rere

view

s

KW

89D

M92

Br

93SM 95

BY

96Po 98

Si 98Se 99

BR

00C

h00

DK

00D

R02

IL 02SW 02

De

03M

e04

WS

04P

I05

Ch

07W

a07

KG

08Pa 08

–Te

rmin

olog

yX

X

Perf

orm

ance

mea

sure

Pro

duct

ivit

yX

XX

XX

XX

X

Mar

ket

per

form

ance

XX

Acc

oun

tin

gpe

rfor

man

ceX

X

Inta

ngi

ble

ben

efits

X(X

)X

XX

X

Leve

lofm

easu

rem

ent

XX

XX

XX

XX

X

Type

ofIS

asse

t(X

)

Met

hod

sX

XX

XX

XX

XX

Infl

uen

cin

gfa

ctor

sC

onte

xtu

alfa

ctor

sX

XX

X

Lag

effe

cts

XX

X

Ris

kX

Val

ue

XX

X

X:c

over

ed(X

):pa

rtia

llyco

vere

d

KW

89:

Kau

ffm

anan

dW

eill

1989

Si98

:Si

rcar

etal

.199

8IL

02:

Iran

ian

dLo

ve20

02C

h07

:C

hau

etal

.200

7

DM

92:

DeL

one

and

McL

ean

1992

Se99

:Se

ddon

etal

.199

9SW

02:

Sylla

and

Wen

2002

Wa0

7:W

anet

al.2

007

Br9

3:B

ryn

jolf

sson

1993

BR

00:

Ban

nis

ter

and

Rem

enyi

2000

De0

3:D

edri

cket

al.2

003

KG

08:

Koh

lian

dG

rove

r20

08

SM95

:So

han

dM

arku

s19

95C

h00

:C

han

2000

Me0

4:M

elvi

lleet

al.2

004

Pa08

:Pa

reet

al.2

008

BY

96:

Bry

njo

lfss

onan

dYa

ng

1996

DK

00:

Dev

araj

and

Koh

li20

00W

S04:

Wal

ter

and

Spit

ta20

04

Po98

:Po

tth

of19

98D

R02

:D

ehn

ing

and

Ric

har

dson

2002

PI0

5:P

icco

lian

dIv

es20

05

Business & Information Systems Engineering 4|2010 239

BISE – STATE OF THE ART

of organizational changes that can leadto additional productivity gains (Dedricket al. 2003). The review of Sylla andWen (2002) suggest to apply techniquesrelated to multi-objective and multi-criteria analysis, value analysis and crit-ical success factors. Kohli and Grover(2008, p. 33) state that our measurementinstruments are often too blunt to cap-ture intangibles.

5.6 Level of Measurement

The classification of Bakos (1987) iswidely adopted in literature reviews. Forexample, Brynjolfsson and Yang (1996)and Dedrick et al. (2003) use firm, in-dustry, and economy level to analyzeliterature findings on productivity (seeSect. 5.2). Brynjolfsson and Yang (1996)argue in favor of firm level, as going downto this level helps to control many prob-lems from aggregation.

The reviews of Chan (2000), Chau etal. (2007) and Wan et al. (2007) revealthat the firm level has attracted most ofresearchers’ interest in the past (about80% of all studies investigated). Their re-sults also show that very few studies com-bine multiple-level approaches and thatresearch at the individual level has beenparticularly underemphasized. A morebalanced picture is drawn by Pare et al.(2008), who find that 19% of empiri-cal studies focus on group level, 23% onindividual level, and 26% on firm level.Multiple levels are addressed in 14% ofthe investigated studies. However, thecomparison of results of the aforemen-tioned studies is difficult, as they refer todifferent outlets and periods.

5.7 Type of IS Asset

The impact of specific IS assets (or com-binations) has not attracted much atten-tion in review literature, although thereare many research papers available andalthough the impact of particular IS as-sets and combinations in conjunctionwith their use and contextual factors ishighly relevant for IS investment deci-sion makers. The review of Seddon et al.(1999) is a valuable exception. It uses thetype of IS asset to classify IS effectivenessliterature. However, the authors do notclassify and assess literature according tothe particular IS asset investigated.

5.8 Methods

The diversity in methods has been rec-ognized and addressed in some litera-

ture reviews. As early as 1989, Kauff-man and Weill analyzed applied methodsand found that the majority of studiesare exploratory and mostly based on mi-croeconomic theory. Potthof (1998) findsthat many empirical studies show defi-ciencies in terms of data and/or meth-ods used. These deficiencies weaken thesignificance of the overall positive re-sults. Chan (2000) find that the period1993–1998 was methodologically pre-dominated by secondary data and marketdata analyses, and case studies. Analyz-ing a more comprehensive period (1991–2005), but also limiting their analysis tofour leading IS journals, Pare et al. (2008,p. 407) find that experiments, case stud-ies and questionnaire surveys accountfor 74% of all research papers. Schu-mann (1993), Irani and Love (2002), andWalter and Spitta (2004) provide tax-onomies for evaluation techniques. Morerecently, Chau et al. (2007) analyzed ECIS(2000–2005) and PACIS (1993–2005) pa-pers and found a general shift from us-ing objective measures (firm value, ROI)to perceptual measures. Overall, the re-views on research methods provide agood exploratory overview of this re-search field. However, only few publica-tions tell us when to use which method.Exceptions are the works of Walter andSpitta (2004), and Sylla and Wen (2002)who survey methods and propose a con-ceptual framework that helps decisionmakers to choose the most appropri-ate method. The authors discuss variousevaluation techniques for tangible bene-fits, intangible benefits, and risk.

5.9 Contextual Factors

The role of contextual factors to deter-mine the impact of IS investments iswidely discussed in literature and resultshave also been reviewed. Dehning andRichardson (2002) identify the particu-lar role of contextual factors for abnor-mal stock market returns. Dedrick et al.(2003) highlight the importance of orga-nizational capital, such as decentralizeddecision-making systems, job training,and business process redesign. Melville etal. (2004) stress that the organizationaland technological context impacts mag-nitude and type of operational efficien-cies. Ravichandran et al. (2009) find thatthe interaction between IS spending andproduct and geographical diversificationcan have a positive effect on firm perfor-mance. At industry level, Melville et al.

(2004) find that the degree of competi-tion in an industry correlates positivelywith the extent to which firms achieveefficiency gains, but negatively with theextent to which firms are able to cap-ture the benefits of efficiency gains. At themacro environmental level, they identifythe telecommunications infrastructure asimportant factor for the economic valueof interorganizational information sys-tems.

5.10 Lag Effects

The need to take lag effects into ac-count was already stated by Kauffmanand Weill (1989), who concluded in theirreview that time lags are often omittedfrom models. Some years later, Brynjolf-sson (1993) and Brynjolfsson and Yang(1996) even argued in their reviews thatlags due to learning and adjustment havebeen insufficiently considered in produc-tivity studies and that this shortcomingin methodology is one of four explana-tions of the “IT productivity paradoxon”.These reviews provide an excellent anal-ysis of the impact of lag effects on pro-ductivity. Unfortunately, literature find-ings on the relevance of lag effects in re-search fields other than productivity havebeen neglected in past literature reviews.

5.11 Risk

Risk in the context of IS investments hasreceived little attention in research papersand does not provide a fertile area for re-views. “[The] consideration of risk is vir-tually absent in the growing literature onthe returns on IT investment, even thoughthe risks are widely recognized.” (Dewanet al. 2007). However, one review (Syllaand Wen 2002) addresses risk and de-scribes briefly the application of real op-tion, portfolio approach, and Delphi ap-proach in the context of IS risk.

5.12 Value

Researchers have started to allude to thedifference between the economic out-come and the value that is perceivedor derived. Although none of the ana-lyzed reviews systematically addresses lit-erature findings on IS value, three re-views address the competitive advantageinduced by IS. Melville et al. (2004) findthat the degree to which the firm canobtain a sustained competitive advan-tage is determined through the level of

240 Business & Information Systems Engineering 4|2010

BISE – STATE OF THE ART

inimitability of rare organizational re-sources that are complementary to IT,and through lacking substitutes. Kohliand Grover (2008) stress that leverag-ing IS and complementarities can leadto competition-strengthening “differen-tial value”. The review of Piccoli and Ives(2005) provides an excellent synthesis ofwork that examines the role of IS in sus-taining competitive advantage. Accord-ing to this review, the literature has co-alesced around four determinants of sus-tainability of IT-dependent strategic ini-tiatives: 1. IT resources barrier (IT as-sets and IT capabilities), 2. complemen-tary resources barrier, such as organiza-tional structure, governance, or access todistribution channels, 3. IT project bar-rier (technology characteristics and im-plementation process), and 4. preemp-tion barrier (switching costs and valuesystem structural characteristics).

6 Conclusion

Based on a comprehensive literaturesearch, this meta review analyzes to whatextent past literature reviews on IS busi-ness value have covered key research ar-eas and preserved their key findings.The results show that while some ar-eas have been explored extensively, someother crucial areas have been neglectedand should be considered in future re-search. The results also reveal research ar-eas where even primary research is weakand needs to get intensified before litera-ture reviews can be applied to synthesizefindings. More precisely, the main resultsof this paper are as follows:

First, the research fields “level of mea-surement” and “contextual factors” havebeen addressed comprehensively in re-views.

Second, there are research fields wherelarge parts have been covered effectivelyin reviews, but where some subfieldshave been neglected. The field “produc-tivity” has been addressed extensivelyin exploratory reviews. With the excep-tion of two early reviews (Brynjolfsson1993; Brynjolfsson and Yang 1996), re-search still lacks explanatory reviews thatidentify complementary assets and rel-evant contextual factors. Similarly, lit-erature reviews have covered the re-search field “methods” comprehensivelyin terms of which methods have beenused (exploratory perspective). However,only little work has been done to ana-lyze the appropriateness of various tech-niques. With regard to the research field

“lag effects”, past reviews have effectivelysynthesized literature findings that referto productivity, but they have not gonebeyond productivity. While the role of ISin gaining sustainable competitive advan-tage has been considered well in literaturereviews, a more general perspective onthe subtle difference between economicperformance and business value is desir-able. The aforementioned research fieldsprovide fertile areas for future literaturereviews.

Third, we find research fields (“ter-minology” and “intangible benefits”),which have not been extensively investi-gated in research papers. However, someliterature reviews stress the importance ofthe fields and provide excellent startingpoints for future reviews, which wouldbe, in turn, good starting points for pri-mary research.

Fourth, there are fertile research ar-eas that have been (largely) ignored byreviews (“accounting performance” and“type of IS asset”). These fields should beaddressed with high priority.

Fifth, there are important researchfields (“market performance” and “risk”)where no substantial body of researchwas available for literature reviews. Wesuggest that researchers (re)start coveringthese fields.

References

Agarwal R, Lucas HC (2005) The informationsystems identity crisis: focusing on high-visibility and high-impact research. MISQuarterly 29(3):381–398

Alpar P, Kim M (1990) A microeconomic ap-proach to the measurement of informationtechnology value. Journal of ManagementInformation Systems 7(2):55–69

Alshawi S, Irani Z, Baldwin LP (2003) Bench-marking information technology invest-ment and benefits extraction. Benchmark-ing – An International Journal 10(4):414–423

ATIS (2007) ATIS telecom glossary 2007.http://www.atis.org/glossary/

Au YA, Kauffman RJ (2003) What do youknow? Rational expectations in informa-tion technology adoption and investment.Journal of Management Information Sys-tems 20(2):49–76

Baily MN (1986) What has happened to pro-ductivity growth? Science 234:443–451

Bakos JY (1987) Dependent variables for thestudy of firm and industry-level impacts ofinformation technology. In: Proc interna-tional conference on information systems,Pittsburgh

Bannister F, Remenyi D (2000) Acts of faith:instinct, value and IT investment deci-sions. Journal of Information Technology15(3):231–241

Barua A, Kriebel CH, Mukhopadhyay T (1995)Information technologies and businessvalue – an analytical and empirical in-vestigation. Information Systems Research6(1):3–23

AbstractGuido Schryen

Preserving Knowledge on ISBusiness Value

What Literature Reviews Have Done

The economic relevance of informa-tion systems has been studied for manyyears and has attracted an abundanceof research papers. However, the “pro-ductivity paradoxon” of the 1990s,Carr’s widely recognized paper “ITdoesn’t matter”, and several studiesthat do not find a positive correlationbetween IS investments and economicperformance reveal long-lasting dif-ficulties for IS researchers to explain“IS business value”. Business executivesand researchers also continue to ques-tion the value of IS investments. Thisraises the question of whether litera-ture reviews have tapped their poten-tial to address the concerns by cover-ing key research areas of IS businessvalue and preserving their key findings.In order to address this question, thispaper identifies and describes 12 keyresearch areas, and synthesizes whatliterature reviews published in perti-nent academic outlets have done topreserve knowledge. The analysis of 22literature reviews shows that some cru-cial areas have not been (sufficiently)covered. They provide fertile areas forfuture literature reviews. As this workis based on the results of more than200 research papers, it is capable ofdrawing a comprehensive picture ofthe current state-of-the-art in IS busi-ness value research.

Keywords: Business value, Informa-tion systems, Literature review, Metareview

Business & Information Systems Engineering 4|2010 241

BISE – STATE OF THE ART

Benaroch M, Jeffery M, Kauffman RJ, Shah S(2007) Option-based risk management: afield study of sequential information tech-nology investment decisions. Journal ofManagement Information Systems 24:103–140

Berndt ER, Morrison CJ (1995) High-tech cap-ital formation and economic performancein US manufacturing industries: an ex-ploratory analysis. Journal of Econometrics65:9–43

Bharadwaj AS (2000) A resource-based per-spective on information technology capa-bility and firm performance: an empiricalinvestigation. MIS Quarterly 24(1):169–196

Bharadwaj A, Bharadwaj SG, Konsynski BR(1999) Information technology effects onfirm performance as measured by Tobin’s q.Management Science 45(7):1008–1024

Bharadwaj A, Keil M, Mahring M (2009) Effectsof information technology failures on themarket value of firms. The Journal of Strate-gic Information Systems 18(2):66–79

Bhatt GD, Grover V (2005) Types of infor-mation technology capabilities and theirrole in competitive advantage: an empiricalstudy. Journal of Management InformationSystems 22(2):253–277

Brynjolfsson E (1993) The productivity para-dox of information technology. Communi-cations of the ACM 36(12):66–77

Brynjolfsson E (1996) The contribution ofinformation technology to consumerwelfare. Information Systems Research7(3):281–300

Brynjolfsson E, Hitt L (1996) Paradox lost?Firm-level evidence on the returns to in-formation systems spending. ManagementScience 42(4):541–558

Brynjolfsson E, Hitt LM (2000) Beyond compu-tation: information technology, organiza-tional transformation and business perfor-mance. Journal of Economic Perspectives14(4):23–48

Brynjolfsson E, Yang S (1996) Informationtechnology and productivity: a reviewof the literature. Advances in Computers43:179–215

Brynjolfsson E, Yang S (1999) The intangi-ble costs and benefits of computer invest-ments: evidence from the financial mar-kets. In: Proc international conference oninformation systems, Atlanta

Brynjolfsson E, Hitt LM, Yang S (2002) Intan-gible assets: how the interaction of com-puters and organizational structure affectsstock market valuations. Brookings Paperson Economic Activity 33(2002-1):137–198

Byrd TA, Turner DE (2000) Measuring the flexi-bility of information technology infrastruc-ture: exploratory analysis of a construct.Journal of Management Information Sys-tems 17(1):167–208

Carr NG (2003) IT doesn’t matter. HarvardBusiness Review 81(5):41–49

Chan YE (2000) IT value: the great dividebetween qualitative and quantitative andindividual and organizational measures.Journal of Management Information Sys-tems 16(4):225–261

Chari MDR, Devaraj S, David P (2008) Theimpact of information technology invest-ments and diversification strategies onfirm performance. Management Science54(1):224–234

Chatterjee D, Richardson VJ, Zmud RW (2001)Examining the shareholder wealth effectsof new CIO position announcements. MISQuarterly 25(1):43–70

Chatterjee D, Pacini C, Sambamurthy V (2002)The shareholder-wealth and trading-volume effects of information-technology

infrastructure investments. Journal of Man-agement Information Systems 19(2):7–42

Chau PYK, Kuan KKY, Liang TP (2007) Researchon IT value: what we have done in Asiaand Europe. European Journal of Informa-tion Systems 16(3):196–201

Clemons EK, Row MC (1991) Sustaining IT ad-vantage – the role of structural differences.MIS Quarterly 15(3):275–292

Cooper HM, Hedges LV (1994) The handbookof research synthesis. Russell Sage Founda-tion, New York

Davern MJ, Kauffman RJ (2000) Discover-ing potential and realizing value from in-formation technology investments. Jour-nal of Management Information Systems16(4):121–143

Dedrick J, Gurbaxani V, Kraemer KL (2003) In-formation technology and economic per-formance: a critical review of the empir-ical evidence. ACM Computing Surveys35(1):1–28

Dehning B, Richardson VJ (2002) Returns oninvestments in information technology: aresearch synthesis. Journal of InformationSystems 16(1):7–30

Dehning B, Stratopoulos T (2002) DuPontanalysis of an IT-enabled competitive ad-vantage. The International Journal of Ac-counting Information Systems 3(3):165–176

DeLone WH, McLean ER (1992) Informationsystems success: the quest for the de-pendent variable. Information Systems Re-search 3(1):60–95

Devaraj S, Kohli R (2000) Information technol-ogy payoff in the health-care industry: alongitudinal study. Journal of ManagementInformation Systems 16(4):41–67

Dewan S, Shi C, Gurbaxani V (2007) Investi-gating the risk-return relationship of infor-mation technology investment: firm-levelempirical analysis. Management Science53(12):1829–1842

Dos Santos BL, Peffers KG, Mauer DC (1993)The impact of information technology in-vestment announcements on the marketvalue of the firm. Information Systems Re-search 4(1):1–23

Feeny DF, Ives B (1990) In search of sus-tainability: reaping long-term advantagefrom investments in information technol-ogy. Journal of Management InformationSystems 7(1):27–46

Fink L, Neumann S (2009) Exploring theperceived business value of the flexibil-ity enabled by information technology in-frastructure. Information & Management46(2):90–99

Floyd SW, Wooldridge B (1990) Path analy-sis of the relationship between competi-tive strategy, information technology, andfinancial performance. Journal of Manage-ment Information Systems 7(1):47–64

Gable GG, Sedera D, Chan TZ (2008) Re-conceptualizing information system suc-cess: the IS-impact measurement model.Journal of the Association for InformationSystems 9(7):377–408

Hayes DC, Hunton JE, Reck JL (2001) Mar-ket reaction to ERP implementation an-nouncements. Journal of Information Sys-tems 15(1):3–18

Hitt L, Brynjolfsson E (1994) The three facesof IT value: theory and evidence. In: Proc15th international conference on informa-tion systems, Vancouver

Hitt LM, Brynjolfsson E (1996) Productivity,business profitability, and consumer sur-plus: three different measures of infor-mation technology value. MIS Quarterly20(2):121–142

Im KS, Dow KE, Grover V (2001) A reexamina-tion of IT investment and the market valueof the firm – an event study methodology.Information Systems Research 12(1):103–117

Irani Z (2002) Information systems evaluation– navigating through the problem domain.Information & Management 40(1):11–24

Irani Z, Love PED (2001) The propagation oftechnology management taxonomies forevaluating investments in information sys-tems. Journal of Management InformationSystem 17(3):159–175

Irani Z, Love PED (2002) Developing a frameof reference for ex-ante IT/IS investmentevaluation. European Journal of Informa-tion Systems 11(1):74–82

Jorgenson DW (2001) Information technol-ogy and the U.S. economy (Presidential ad-dress to the American Economic Associa-tion). American Economic Review 91(1):1–32

Jorgenson DW, Stiroh KJ (1995) Computersand growth. Economics of Innovation andNew Technology 3:295–316

Jorgenson DW, Stiroh KJ (2000) Raising thespeed limit: US economic growth in the in-formation age. Brookings Papers on Eco-nomic Activity 1(1):125–211

Karimi J, Somers TM, Bhattacherjee A (2007)The role of information systems resourcesin ERP capability building and businessprocess outcomes. Journal of ManagementInformation Systems 24:221–260

Kauffman RJ, Weill P (1989) An evaluativeframework for research on the perfor-mance effects of information technologyinvestment. In: Proc 10th international con-ference on information systems, Boston

Kelley MR (1994) Productivity and informa-tion technology – the elusive connection.Management Science 40(11):1406–1425

Kim D, Cavusgil ST, Calantone RJ (2006) In-formation system innovations and sup-ply chain management: channel relation-ships and firm performance. Journal of theAcademy of Marketing Science 34(1):40–54

Ko M, Bryson KM (2002) A regression treebased exploration of the impact of infor-mation technology investments on firmlevel productivity. In: Proc European con-ference on information systems

Ko M, Osei-Bryson KM (2004) Using regressionsplines to assess the impact of informationtechnology investments on productivity inthe health care industry. Information Sys-tems Journal 14(1):43–63

Kohli R, Grover V (2008) Business value ofIT: an essay on expanding research direc-tions to keep up with the times. Journalof the Association for Information Systems9(1):23–39

Li MF, Ye LR (1999) Information technologyand firm performance: linking with en-vironmental, strategic and managerialcontexts. Information & Management35(1):43–51

Lin WT, Shao BBM (2006b) Assessing the inputeffect on productive efficiency in produc-tion systems the value of information tech-nology capital. International Journal of Pro-duction Research 44(9):1799–1819

Lin WT, Shao BBM (2006a) The business valueof information technology and inputs sub-stitution: the productivity paradox revis-ited. Decision Support Systems 42(2):493–507

Loveman GW (1994) An assessment of theproductivity impact of information tech-nologies. In: Allen TJ, Scott-Morton MS

242 Business & Information Systems Engineering 4|2010

BISE – STATE OF THE ART

(eds) Information technology and the cor-poration of the 1990s: research studies. Ox-ford University Press, Oxford

Mahmood MA, Mann GJ (1993) Measur-ing the organizational impact of infor-mation technology investment: an ex-ploratory study. Journal of Management In-formation Systems 10(1):97–122

Mahmood MA, Mann GJ (2005) Informa-tion technology investments and organi-zational productivity and performance: anempirical investigation. Journal of Organi-zational Computing and Electronic Com-merce 15(3):185–202

Maier R, Hädrich T (2001) Modell für dieErfolgsmessung von Wissensmanage-mentsystemen. WIRTSCHAFTSINFORMATIK43(5):497–509

McFarlan FW (1981) Portfolio approach to in-formation systems. Harvard Business Re-view 59(5):142–150

Melville N, Kraemer K, Gurbaxani V (2004)Review: information technology and or-ganizational performance: an integrativemodel of IT business value. MIS Quarterly28(2):283–322

Melville N, Gurbaxani V, Kraemer K (2007)The productivity impact of informationtechnology across competitive regimes:the role of industry concentration anddynamism. Decision Support Systems43(1):229–242

Menon NM, Lee B, Eldenburg L (2000) Pro-ductivity of information systems in thehealthcare industry. Information SystemsResearch 11(1):83–92

Mertens P, Anselstätter R, Eckhardt T, Nickel R(1982) Betriebswirtschaftliche Nutzeffekteund Schäden der EDV – Ergebnisse des NSI-Projektes. Zeitschrift für Betriebswirtschaft52(2):135–154

Mittal N, Nault BR (2009) Investments in in-formation technology: indirect effects andinformation technology intensity. Informa-tion Systems Research 20(1):140–154

Money A, Tromp D, Wegner T (1988) Thequantification of decision support benefitswithin the context of value analysis. MISQuarterly 12(2):223–236

Neirotti P, Paolucci E (2007) Assessing thestrategic value of information technology:an analysis on the insurance sector. Infor-mation & Management 44(6):568–582

Oliner SD, Sichel DE (2000) The resurgenceof growth in the late 1990s: is informationtechnology the story? Journal of EconomicPerspectives 14(4):3–22

Orlikowski WJ, Iacono CS (2001) Desperatelyseeking the ‘IT’ in IT research. A call to the-orizing the IT artifact. Information SystemsResearch 12(2):121–134

Pare G, Bourdeau S, Marsan J, Nach H, Shu-raida S (2008) Re-examining the causalstructure of information technology im-pact research. European Journal of Infor-mation Systems 17(4):403–416

Piccoli G, Ives B (2005) Review: IT-dependentstrategic initiatives and sustained compet-itive advantage: a review and synthesis ofthe literature. MIS Quarterly 29(4):747–776

Poston R, Grabski S (2000) The impact of en-terprise resource planning systems on firmperformance. In: Proc international confer-ence on information systems

Potthof I (1998) Empirische Studien zumwirtschaftlichen Erfolg der Informations-verarbeitung. WIRTSCHAFTSINFORMATIK40(1):54–65

Rai A, Patnayakuni R, Patnayakuni N (1997)Technology investment and business per-formance. Communications of the ACM40(7):89–97

Ravichandran T, Lertwongsatien C (2005)Effect of information systems resourcesand capabilities on firm performance:a resource-based perspective. Journalof Management Information Systems21(4):237–276

Ravichandran T, Liu Y, Han S, Hasan I (2009) Di-versification and firm performance: explor-ing the moderating effects of informationtechnology spending. Journal of Manage-ment Information Systems 25(4):205–240

Roach SS (1987) America’s technologydilemma: a profile of the information econ-omy. Morgan Stanley, Special EconomicStudy

Roach SS (1991) Services under siege: therestructuring imperative. Harvard BusinessReview 39(2):82–92

Rockart JF (1979) Chief executives definetheir own data needs. Harvard Business Re-view 57(2):81–93

Rose JM, Rose AM, Norman CS (2004) Theevaluation of risky information technologyinvestment decisions. Journal of Informa-tion Systems 18(1):53–66

Salipante P, Notz W, Bigelow J (1982) A ma-trix approach to literature reviews. In: StawBM, Cummings LL (eds) Research in organi-zational behavior. JAI Press, Greenwich

Santhanam R, Hartono E (2003) Issues in link-ing information technology capability tofirm performance. MIS Quarterly 27(1):125–153

Schumann M (1990) Abschätzung von Nutz-effekten zwischenbetrieblicher Informa-tionsverarbeitung. WIRTSCHAFTSINFOR-MATIK 32(4):307–319

Schumann M (1993) Wirtschaftlichkeitsbeur-teilung für IV-Systeme. WIRTSCHAFTSIN-FORMATIK 35(2):167–178

Seddon PB, Patnayakuni R, Bowtell M (1999)Dimensions of information systems suc-cess. Communications of the AIS 2(3):1–61

Shih E, Kraemer KL, Dedrick J (2007) Determi-nants of country-level investment in infor-mation technology. Management Science53(3):521–528

Sircar S, Turnbow JL, Bordoloi B (1998) Theimpact of information technology invest-ments on firm performance: a review of theliterature. The Journal of Engineering Valu-ation and Cost Analysis 1(3):171–181

Sircar S, Turnbow JL, Bordoloi B (2000)A framework for assessing the relation-ship between information technology in-vestments and firm performance. Jour-nal of Management Information Systems16(4):69–97

Soh C, Markus ML (1995) How IT creates busi-ness value: a process theory synthesis. In:Proc 16th international conference on in-formation systems, Amsterdam

Stickel E (1995) WettbewerbsorientierteInformationssysteme und Produktivitäts-paradoxon. WIRTSCHAFTSINFORMATIK37(6):548–557

Stiroh KJ (2002) Information technology andthe US productivity revival: what do the in-dustry data say? American Economic Re-view 92(5):1559–1576

Stiroh K, Botsch M (2007) Information tech-nology and productivity growth in the2000s. German Economic Review 8(2):255–280

Stratopoulos T, Dehning B (2000) Does suc-cessful investment in information technol-ogy solve the productivity paradox? Infor-mation & Management 38(2):103–117

Subramani M, Walden E (2001) The impact ofe-commerce announcements on the mar-ket value of firms. Information Systems Re-search 12(2):135–154

Swierczek FW, Shrestha PK (2003) Informa-tion technology and productivity: a com-parison of Japanese and Asia-Pacific banks.The Journal of High Technology Manage-ment Research 14(2):269–288

Sylla C, Wen HJ (2002) A conceptual frame-work for evaluation of information tech-nology investments. International Journalof Technology Management 24(2–3):236–261

Tam KY (1998) The impact of informa-tion technology investments on firm per-formance and evaluation: evidence fromnewly industrialized economies. Informa-tion Systems Research 9(1):85–98

Thatcher ME, Oliver JR (2001) The impact oftechnology investments on a firm’s produc-tion efficiency, product quality, and pro-ductivity. Journal of Management Informa-tion Systems 18(2):17–45

Thatcher ME, Pingry DE (2004) An economicmodel of product quality and IT value.Information Systems Research 15(3):268–286

Urbach N, Smolnik S, Riempp G (2009) DerStand der Forschung zur Erfolgsmessungvon Informationssystemen – Eine Analysevorhandener mehrdimensionaler Ansätze.WIRTSCHAFTSINFORMATIK 51(4):363–375

Walter SG, Spitta T (2004) Approaches tothe ex-ante evaluation of investments intoinformation systems. WIRTSCHAFTSINFOR-MATIK 46(3):171–180

Wan Z, Fang Y, Wade M (2007) The ten-yearodyssey of the IS productivity paradox –a citation analysis (1996–2006). In: ProcAmericas conference on information sys-tems, Keystone

Watson RT (2001) Introducing MISQ Review– a new department in MIS Quarterly. MISQuarterly 25(1):103–106

Webster J, Watson RT (2002) Analyzing thepast to prepare for the future: writing aliterature review. MIS Quarterly 26(2):XIII–XXIII

Wehrmann A, Zimmermann S (2005) Inte-grierte Ex-ante-Rendite-/Risikobewertungvon IT Investitionen. WIRTSCHAFTSINFOR-MATIK 47(4):247–257

Wehrmann A, Heinrich B, Seifert F (2006)Quantitatives IT-Portfoliomanagement.WIRTSCHAFTSINFORMATIK 48(4):234–245

Weill P (1992) The relationship between in-vestment in information technology andfirm performance: a study of the valve man-ufacturing sector. Information Systems Re-search 3(4):307–333

Weill P, Olson M (1989) Managing invest-ment in information technology: mini caseexamples and implications. MIS Quarterly13(1):3–18

West LA, Courtney JF (1993) The informa-tion problems in organizations – a re-search model for the value of informa-tion and information systems. Decision Sci-ences 24(2):229–251

White HD (1994) Scientific communicationand literature retrieval. In: Cooper HM,Hedges LV (eds) The handbook of researchsynthesis. Russell Sage Foundation, NewYork

Wiseman D (1992) Information economics: apractical approach to valuing informationsystems. Journal of Information Technol-ogy 7(3):169

WITSA (2008) Global ICT spending tops$3.5 trillion: industry experiences subduedspending growth. http://www.witsa.org/press/Digital_Planet_Release_final.doc

Business & Information Systems Engineering 4|2010 243

BISE – STATE OF THE ART

Zahn E (1990) Informationstechnologie alsWettbewerbsfaktor. WIRTSCHAFTSINFOR-MATIK 32(6):493–502

Zhu K, Kraemer KL, Xu S, Dedrick J (2004) In-formation technology payoff in e-businessenvironments: an international perspective

on value creation of e-business in the finan-cial services industry. Journal of Manage-ment Information Systems 21(1):17–54

244 Business & Information Systems Engineering 4|2010