erpimpeval[1]

7/31/2019 erpimpeval[1]

1/12

322 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 51, NO. 3, AUGUST 2004

A Framework for Evaluating ERPImplementation Choices

Wenhong Luo and Diane M. Strong

Abstract A key issue in enterprise resource planning (ERP) im-plementation is how to nd a match between the ERP system andan organizations business processes by appropriately customizingboth the system and the organization. In this paper, we advancea framework for supporting management decision-making aboutcustomization choices and the capabilities required to accomplishthem. In this framework, we identify various customization possi-bilities for business processes as well as ERP systems. We also iden-tify technical and process change capabilities required for systemand process customization. Combining customization options withchange capabilities, we present a useful way for managers to iden-

tify feasible customization options for their particular organiza-tion. Such a framework also helps managers to recognize the gapbetween desired customization options and change capabilities. Acase study is used to illustrate the application of the framework.

Index Terms Case study, change capability, enterprise re-source planning (ERP) systems, process customization, systemcustomization.

I. INTRODUCTION

E NTERPRISE Resource Planning (ERP) software is oneof the fastest growing segments of business computingtoday. According to a report by Advanced Manufacturing

Research, the ERP software market is expected to grow from$21 billion in 2002 to $31 billion in 2006 and the entireenterprise applications market which includes Customer Rela-tionship Management and Supply Chain Management softwarewill top $70 billion [ 1]. The reason behind this phenomenalgrowth is the promise that ERP systems can provide an inte-grated business computing solution and improve a companysability to compete in the marketplace. Often cited benets of ERP systems include the integration of data and applications,replacement of old, fragmented legacy systems, cost advantagesand quicker deployment of packaged systems as compared within-house development, and the adoption of best practices inorganizational processes [ 2], [3].

For individual companies, however, the implementation of ERPsystems presents thegreatest challengeformany MISman-agers today [ 4][8]. Reports of ERPimplementation failures arecommon. Reasons for failures include spending more money

Manuscript received May 1, 2001; revised March 1, 2004. Review of thismanuscript was arranged by Department Editor V. Sambamurthy.

W. Luo is with the Department of Decision and Information Technologies,College of Commerce and Finance, Villanova University, Villanova, PA 19085USA (e-mail: [email protected]).

D. M. Strong is with the Department of Management, Worcester Poly-technic Institute, Worcester, MA 01609 USA (e-mail: [email protected];www.mgt.wpi.edu/People/Strong/).

Digital Object Identier 10.1109/TEM.2004.830862

on ERP than the company can afford, being incompatible withstrategic partners, conicting with its management style andstructure, being overwhelmed by the required organizationalchanges to t the system, and dealing with ever changing ERPtechnology and its infrastructure [ 9][11]. Dow Chemical, forexample, spent seven years and close to a half of a billion dol-lars in implementing a mainframeERP system and then realizedthat a client-server architecture would be more appropriate [ 12].

Any successful ERP implementation requires a t between

the ERP system and the organizational processes it supports[13], [14]. An important characteristic of ERP systems is thatthey are packaged software solutions rather than customizedsystems. As such, they come with built-in assumptions andprocedures about organizations business processes. These as-sumptions and procedures seldom match exactly with thoseof the implementing organizations existing processes [ 15].In traditional information systems development, the computersystem is usually designed to t the organization and its pro-cesses. With packaged software solutions such as ERP systems,it is difcult to completely mold the system to t existingbusiness processes. In fact, many researchers and practitionershave suggested that it is easier and less costly to mold busi-ness processes to ERP systems rather than vice versa [12],[13]. Thus, even for those companies that have successfullyimplemented large-scale information systems projects in thepast, ERP implementation still presents a challenge, becauseit is not simply a large-scale software deployment exercise.ERP implementation is often accompanied by large-scale or-ganizational changes [ 14], [16]. Consequently, a key issue inERP implementation is how to nd a match between the ERPsystem and an organizations business processes by appropri-ately customizing both the system and the organization.

In this paper, we advance a framework for supporting man-agement decision-making about customization choices and the

capabilities required to accomplish them. In this framework,we identify various customization possibilities for businessprocesses, as well as ERP systems. We also identify technicalcapabilities required for technical ERP customization optionsand process change capabilities needed for process customiza-tion. Combining the process and technical customizationoptions with technical and process change capabilities, wepresent a useful way for managers to identify feasible cus-tomization options for their particular organization. Such aframework also helps managers to develop a long-term view of ERP implementation by suggesting that ERP implementationbe viewed as a series of interdependent customization andimplementation projects.

0018-9391/04$20.00 2004 IEEE


2/12

LUO AND STRONG: FRAMEWORK FOR EVALUATING ERP IMPLEMENTATION CHOICES 323

TABLE ILITERATURE ON ERP IMPLEMENTATION ISSUES

II. RELATED LITERATURE

A large body of literature on information technology (IT)implementations has been developed during the past severaldecades [ 17][19]. However, our understanding of the factorsandprocessesthat lead to ERPimplementation successesor fail-ures is still limited because ERP implementation is relativelynew and is different from traditional information systems de-velopment projects [ 11], [20]. Here, we focus on some of theunique characteristics of ERP systems and review the imple-mentation issues related to these characteristics. A summary isprovided in Table I.

One major difference lies with the fact that ERP systemsare packaged software. Carmel and Sawyer [ 21] compared thedevelopment processes of packaged software with traditionalinformation systems at the industry, development, culturalmilieu, and team levels. Their analysis suggests that vendors of

packaged software have to satisfy many customers with varyingneeds and requirements in order to capture the necessary marketshare and pro t to justify their investment. At the same time,there is tremendous time-to-market pressure on the vendorsto come out with their new products to stay ahead of theircompetitors. In addition, packaged software developers areseparated from the users organizationally, as well as physicallyand they usually do not participate in the implementation of the software package. Intermediaries such as sales and cus-tomer support staff and third party consultants often providethe linkage between software developers and users [ 22][24].Consequently, unlike traditional software development projects

where a system is tailor-made to suit the existing business re-quirements and processes, packaged software implementationinvolves the users changing procedures and business processesin order to use the package, changing some of the programsin the package to t their unique requirements, and relying onpackage vendors for assistance and updates to the softwarepackage [ 25].

The disconnect between an organization s information andprocess requirementsandthesolutionsprovided by ERPis espe-cially pronounced due to thecomplex natureof theERPsystems[26]. In their study of ERP implementations in Singapore, Sohet al. [27] illustrates that the so-called industry best practices embedded in ERP systems is hardly universal. They suggest thatthe mists between business requirements and ERP capabili-ties can be company-speci c, industry-speci c, or country-spe-

cic and may be classi ed into three categories: data, process,and output. Davison [ 28] highlighted the cultural implicationsof mists in ERP implementations.

Another difference between ERP and traditional informationsystem development project is that ERP projects, which tendto be enterprise-wide, are typically larger in scope than tradi-tional software development projects, which often focus on oneor more departments or business processes. The risks associ-ated with ERP projects are relatively higher than those tradi-tional projects [ 29]. Enterprise-wide projects affect many moreusers and these users mayhave differentandpossibly con ictingneeds and requirements. In addition, ERP projects require theintegration of data, processes, and operations throughout theenterprise, often on a global scale, which makes ERP projectsfarmore complex than traditional systems developmentprojects[7]. To achieve integration, dissimilar components and indepen-

dent business processesneed to be tted together by conformingto a uniform standard. For example, integration may be accom-plished by changing each business process to t with the systemusing the ERP system as a standard.

The t between business processes and ERP systems andamong businessprocesses is believedto be critical to the successof ERP implementation [ 3], [13], [30], [31]. Hong and Kim [ 31]assessed the impact of data, process, and user t between ERPsystem and organizational requirements on implementation suc-cess. They found a positive correlation between the initial orga-nizational t and implementation success. However, for mostorganizations, such a t can only be achieved through mutualadaptation of the ERP systems and organization processes [ 32].

The adaptation of the ERP systems involves the customiza-tion of the ERP system to t existing or reengineered businessprocesses. Davenport [ 12] identied ERP system customizationas module selection, table con guration, and code modi ca-tion. Similarly, Glass [ 33] classied ERP system customizationinto conguration, extension, and modi cation. Brehm et al.[15] provided a more detailed categorization of ERP systemcustomization, which include nine types of adaptation: con g-uration, bolt-ons, screen masks, extended reporting, work owprogramming, user exits, ERP programming, interface devel-opment, and package code modi cation. Each type may requirechanges to be made at different layers of the ERP system.Consequently, such changes may affect the initial ERP im-plementation, as well as future maintenance, upgrade, and


3/12


conversions. It is worth noting that these frameworks are onlyconcerned with the change of systems to t business processes.

An important stream of ERP implementation research fo-cuses on the adaptation of the organization to ERP systems.This research explores the process of ERP implementationand its associated organizational changes from initiation to

development to completion over time [ 4], [14]. ERP imple-mentation is viewed as a long and complex process with suc-cesses and failures at different stages of the implementation.In a study of ERP implementation in 15 different companies,Ross [34] discovered that most companies go through thefollowing ve stages in their implementation process: design,implementation, stabilization, continuous improvement, andtransformation. Markus and Tanis [ 3] describes a typical ERPimplementation in terms of four phases: the chartering phase,the project phase, the shakedown phase, and the onward andupward phase. Koh et al. [35] applied this process theory toidentify problems and issues of ERP implementation usinga case study. Stages models of IT implementation have alsobeen used in analyzing ERP implementations [ 13], [36]. Al-though these models depict the ERP implementation processdifferently, they share many similar activities and decisionsthat need to be made by managers. One of the key decisionsin the early stage of the implementation process is whether toaccept the assumptions about business processes built into thesystem [ 34]. This decision affects the amount of customizationneeded to the software, as well as to the organization.

The systems development life cycle methodology needs to bemodied for the unique characteristics of ERP implementation[27], [37]. Implementation in the traditional SDLC refers tothe later stage of system development in which a completed

system is installed, deployed, and placed into operation orproduction. The stage also includes the conversion to the newsystem, the training of users, and nal documentation of thesystem [ 38]. ERP implementation involves the understandingof existing business processes and ERP technologies, the cus-tomization of business processes, and ERP modules and tablesto t each other, and the management of large-scale businessprocess change and system integration projects. Hence, analysisphase activities (e.g., understanding of critical organizationprocesses) and design phase activities (e.g., knowledge of theERP software) of SDLC have to be merged for ERP implemen-tation [27]. Brehm and Markus [ 37] stressed the importance

of interaction between vendors and adopters during the ERPimplementation.From a resource-based perspective, the success of ERP im-

plementation is affected by the kind of IT-based resourcespossessed by an organization and how they are assembled,coordinated, and deployed [ 39][42]. IT-based resources canbe classi ed into tangible IT resources (e.g., IT infrastructure),intangible IT resources (e.g., knowledge bases), and humanIT resources (e.g., technical and managerial IT skills) [ 42],[43]. The ability to assemble, coordinate, and deploy IT-basedresources in combination with other resources is referred toas an organization s IT capability. Feeny and Willcocks [ 41]included the following as a set of core IT capabilities: IS/ITleadership, business system thinking, relationship building, ar-chitectureplanning, making technologywork, informed buying,

contract facilitation, contract monitoring, and vendor devel-opment. Organizations have access to different IT resourcesand, thus, possess different IT capabilities. Such differencesmay explain the divergences among organizations in the useof IT and in the bene ts they have gained from the usage[39]. They may also one of the major reasons why organiza-

tions choose different ERP customization options during ERPimplementation.

III. CUSTOMIZATION IN ERP IMPLEMENTATION

The primary goal of customization in ERP implementation isto achieve a t between the ERP system and the process that thesystem supports. Thus, both the system and the process can bechanged or customized to achieve the goal. When the system iscustomized to t the process, we refer to this kind of customiza-tion as technical customization . Similarly, when a process is cus-tomized to t the system, we refer it as process customization .

A. Technical CustomizationWhen installing an ERP system, companies have many

choices about how to change and customize the softwarepackage [ 37]. Most ERP software packages are developed in amodular approach, where each module contains speci c func-tionality and con gurable options [ 44]. In addition, an openor proprietary programming environment is often provided byERPvendors to their customers for modifying the system. Thus,companies have three types of technical customization options:module selection, table con guration, and code modi cation[12]. In module selection, companies choose to implementone or more modules using the default con guration set by

ERP vendors. In this case, technical customization is achievedthrough the company s decision as to which modules to imple-ment. This type of technical customization makes minimumalterations to the system and, by itself, is rarely suf cient inan ERP implementation. Some small companies, however,may take this low cost and low risk approach. For example,Thermacore decided to implement SAP s accounting moduleusing the default charts of accounts without any changes [ 45].

Since most ERP systems are table-driven, another type of technical customization is to select con guration options in thetablesso that thesystem ts organizationalneeds.A key require-ment for table con guration is to understand the meaning and

consequences of each con gurable option in each table. Sincethere are numerous tables in a typical ERP system, this can be avery complex and time-consuming task, especially when inter-dependencies among options across various tables and modulesneed to be considered. For example, Dell Computer spent morethan a year on table con guration alone [ 12]. The bene ts of this type of technical customization include the ability to tailorthe system without coding, the full support from the vendor, andthe ease of future upgrades.

The third type of technical customization is code customiza-tion, where the source code of the ERP system is changed, thefunctionality is augmented, or a new interface is developedto allow the ERP system to interact with other systems [ 46],[47]. Some ERP systems come with a proprietary program-ming environment to support code customization; others can


4/12


interface with higher level programming languages such asC++. Code customization provides companies the greatestexibility in adapting the system to organizational needs andallows companies to integrate the ERP system with any existingproduction systems. On the other hand, it also presents thehighest risks and costs in technical customization. It is quite

expensive to obtain competent technical staff or consultants toperform code customization and there are risks of failures andbudget overruns. Also, too much code customization leads toincompatibility with newer versions of the system and, thus,dif culties in future upgrades. Furthermore, certain integrationbenets built into the original ERP design may be lost as aresult of code customization. Clearly, as we move from modulecustomization to code customization, the costs and risks in-crease; the bene ts, however, may or may not increase.

ERP vendors have a rather different view of technical cus-tomization than the view of adopting organizations [ 46]. Whilemost vendors provide the above programming environmentfor supporting adopting organizations, they consider technicalcustomization as an evolving process where they continuouslyadd modules, extend con guration tables, and improve toolsfor code modi cation to meet the needs of adopting organi-zations. Their goal is to reduce the degree, costs, and risksof customization so that adopting organizations can restricttheir customization efforts to module selection and tableconguration.

B. Process Customization

Fit can also be achieved by changing the process ratherthan the system. Process customization is the degree to which

the business process is changed to t the system. A businessprocess is de ned as a set of logically related tasks that usethe resources of an organization to achieve a de ned businessoutcome [ 48]. This denition indicates that a business processconsists of tasks, resources, the outcome, relationships amongtasks, and relationships among resources and tasks. Basedon the changes made to these elements, we classify processcustomization into three categories: no change, incrementalchange, and radical change.

In the case of no change, process customization involves onlychanges in tasks and resources, but no changes in relationshipsamong tasks and con gurations of resources. An example of such process customization is task automation in which com-puter technologies are substituted for manual labor. Then, theresources used to accomplish the task have been switched frommanual labor to computers but the other elements of the busi-ness process remain the same.

The second category of process customization is incrementalchange in which improvements are made not only in tasks andresources, but also in relationships among tasks and relation-ships among tasks and resources. The nature of the process andits outcome measures, however, has not changed. The focus of the change is solving problems found in the process. Most totalquality management (TQM) initiatives are examples of incre-mental change [ 49].

Radical change is the third category of process customiza-tion. It involves the fundamental rethinking and radical redesign

of the elements in a business process, including the measuresof performance. Literature on Business Process Reengineeringprovides numerous examples of radical change. Organizationsthat are converting from a functional view of business to aprocess view often make radical changes to their businessprocesses as well [ 49].

These three categories of process change represent in-creasing degrees of process customization on a continuumfrom no change to radical change. How much change is neededdepends on how well the new technology ts with the existingprocess. Management has the choice of changing the processto t the system and vice versa.

C. ERP Customization Choices

With technical customization and process customization asdimensions, we derive a table for describing various ERP cus-tomization choices, as illustrated in Table II.

Each cell in the table represents a possible choice forachieving t between the process and the ERP system. Eachrow shows the process change options given a chosen systemconguration option. Each column identi es the system con-guration options to t a particular process change decision.For example, the rst row shows the three available options if managers decide to adopt the process embedded in the systemand make any necessary changes in the business process to tthe system. In this case, the system process is often regardedas the best practice or is deemed as the ideal process for theorganization.

The no customization cell is a valid choice when there isalready a good t between the existing business process and thesystem process. Thus, no process customization is necessary

and a good t can be achieved by selecting appropriate systemmodules. Although rare, it is still possible when, for example,an organization s process was used as a model for developingthe system. This cell involves the lowest implementation risk among all cells provided that the assessment of t is accurate.Managers should ensure that the perceived t between theexisting process and the embedded system process is not justtheir wishful thinking. The cell process adaptation involvesmaking moderate process changes to t system modules. Itassumes that the existing business process is similar to thesystemprocess andincremental changes aresuf cient to achievet. The risk associated with this cell is primarily related to

the organization s ability in improving the existing businessprocess. If the existing business process is drastically differentfrom the system process, then the cell process conversioninvolving radical changes to the current process is necessaryto t the system process. This option is signi cantly riskierthan process adaptation.

The second row assumes that technical customization islimited to conguration of the system through tables. MostERP systems come with con gurable tables that model typicalvariations of business processes. Using the technical changesinvolved in table customization, organizations can tailor thesystem to a near t to the process. Any further changes neededto achieve complete t can be accomplished by changingprocesses. The cell t system to process is an ideal situation,where t has been accomplished through table customization.


5/12


TABLE IICUSTOMIZATION CHOICES

The risk factor for this option is slightly greater than no cus-tomization . Mutual adaptation means that similar amounts of customization are applied to both the system and the process.Consequently, the cell is riskier than t system to processand process adaptation . When there is still a signi cant gapbetween the system and process after table customization, t process to system may be required to redesign the process toachieve t. This option is even riskier than process conversionbecause it not only requires radical process change but alsochanges in the system.

In the third row, system customization is not limited toonly table con guration; changes to the source code are alsoconsidered to be viable. While vendors discourage code cus-tomization, they do provide programming environments tofacilitate such changes because there are variations of businessprocesses that cannot be modeled through table customiza-tion. For various reasons, managers may decide to make no

changes to the process and achieve the t entirely throughchanges to the system. This option is referred to as systemconversion . Organizations need substantially more expertiseto work with the system than table con guration and, hence,system conversion is riskier than the t system to process cell.The system conversion and process adaptation cell involvesmoderate process improvements, as well as code customization.Depending on the nature of process changes, this option maynot be as risky as the system conversion cell because it mayincrease exibility in the process and require fewer codingchanges. Finally, system and process reengineering refers toa scenario in which managers may choose to redesign thebusiness process and make necessary changes to the systemto support the redesigned process. Such changes in both theprocess and the system represent the most risky cell in the table.

IV. CAPABILITY REQUIREMENTS FOR ERP CUSTOMIZATION

The customization options depicted in Table II representpossible customization choices available to managers. No oneoption in Table II is necessarily better than the others perse. Consistent with the resource-based perspective, we argue,however, that some options are better for an organization thanothers and this is determined by the organization s capability tomake necessary system and process changes. Such capabilityrequirements consist of technical change capability and processchange capability .

A. Technical Change Capability

Technical change capability refers to an organization soverall ability to customize ERP systems. The rst abilitywithin technical change capability is concerned with the under-

standing of default ERP system processes, con gurations, andbuilt-in options. It is crucial to recognize the underlying man-agement principles and assumptions made by system vendors[20], [25]. This understanding forms the basis for planning andmaking appropriate changes to the system. Organizations maydiffer in this ability in terms of the scope and depth of theirunderstanding of the adopted ERP system. For example, someorganizations have a good understanding of the entire systemand others may focus on a module or a part of the system. Thatis, the scope of understanding can differ. Similarly, the depth of understanding can vary as well. For example, one organizationknows the processes that are supported by the system, whileanother organization also understands the assumptions andinterconnections within the system well enough to plan anydesired changes.


6/12


TABLE IIICAPABILITY ASSESSMENT

The second technical change capability is the ability to de-velop and modify large-scale software in a networked databaseenvironment. This ability is directly related to the extent towhich organizations are able to make desired system changes.It is crucial that organizations have the ability to use a varietyof software application development tools such as custom toolsprovided by ERP vendors, database management tools, andprogramming languages. The more tools an organization is ableto employ, the broader the scope of their technical change capa-bility. With each development tool, the degree of pro ciency inusing that tool signi es the depth of an organization s technicalchange capability.

The third technical change capability is an organization sability to manage large-scale systems development projects.This includes setting unambiguous and realistic goals for theproject, providing necessary resources to match the goals,maintaining clear communication channels among projectparticipants, monitoring project progress, and detecting andcorrecting any problems within the project [ 38]. ERP im-plementation may be viewed as a series of coordinated ITcustomization and implementation projects. As such, organi-zations would need not only the ability to manage individualprojects, but also the ability to coordinate and integrate multiple

interconnected projects.An organization s technical change capability consists of thescope and depth of its ability to understand the ERP system, itsability to make changes to software, and its ability to managelarge-scale ERP implementation projects. We consider an or-ganization to have high technical change capability if it hasbroad scope and great depth in all three abilities. Conversely, weconsider an organization to have low technicalchangecapabilityif it has narrow scope and limited depth in all three abilities.

B. Process Change Capability

The other change capability that an organization needs is

process change capability, which refers to its overall abilityto customize business processes. Organizations rst need tohave the ability to understand their existing business processesand their business environment. That understanding shouldinclude not only the current state of existing business processesbut also the history and evolution of these processes. SinceERP systems cross functions, enterprise-wide understandingof processes is more valuable than just the understanding of individual functional areas. Therefore, an organization wouldhave higher process change capability if it has enterprise-wideunderstanding of its business processes and their historyand evolution, as well as its products, customers, and otherstakeholders.

Second, organizations must have the ability to design new orchanged businessprocesses, as well as implement these designs.

Creative thinking and process design tools and methodologiesare crucial for designing new or changed business processes.Creative thinking allows organizations to think outside of thebox and generate many alternative designs. The ability to useprocess design toolsand methodologies permits organizations tosimulate and evaluate process design alternatives and to identifythose with signi cant performance improvements.

Third, organizations must be capable of managing and co-ordinating large-scale business process changes. As discussedearlier, ERP implementation can be viewed as a series of cus-tomization projects that involves signi cant business processchanges. As such, managing large-scale business processchanges consists of managing individual projects, as well asintegrating and coordinating multiple projects. To successfullymanage individual projects, organizations need to have the nec-essary skills in project management and organizational changemanagement. Integrating and coordinating multiple projects re-quires the ability to decompose large projects into smaller ones,manage the interfaces among these projects, and integrate theirresults to achieve overall objectives. The advantage of a seriesof projects is the opportunity for organizations to learn. Thatlearning includes the exchange of knowledge within and acrossmultiple project teams. Furthermore, knowledge acquired from

earlier projects can be applied to future projects.We measure an organization s process change capabilitiesin terms of their scope and depth. For instance, the scope of the organization s capability in managing process changesis broad when an organization is capable of managing verylarge-scale projects. Furthermore, we consider an organizationto have in-depth capability if that organization has a largeset of alternative tools and techniques for managing processchanges. We consider an organization to have high businessprocess change capability if it has broad scope and great depthin all three abilities, the ability to understand existing businessprocesses, the ability to design and implement business processchanges, and the ability to manage large scale organizationalchanges. Conversely, we consider an organization to have lowbusiness process change capability if it has narrow scope andlimited depth in all three abilities.

C. Overall Capability

Combining the technical change capability and processchange capability, we can assess an organization s overall capa-bility in implementing and customizing ERP systems. Table IIIshows the four possible combinations of such an assessment.A novice organization has low capabilities in both technicalchange and process change. That places severe constraints onwhat an organization can do to customize its own processes, aswell as the software package.


7/12


TABLE IVFRAMEWORK FOR EVALUATING ERP CUSTOMIZATION CHOICES

A technician organization has high technical change capa-bility, but low process change capability. It has high ability tot the system to the organization s needs, but it may not be able

to take full advantage of the possible process improvements. Atechnician organization should guard against the temptation toover customize the system. An organizer is an organization thathas high process change capability, but low technical changecapability. It is in a better position to take advantage of processimprovement associated with ERP systems. Due to the inabilityto change the system, an organizer may not be able to realize agood t between the process and the ERP system.

An expert organization has high capabilities in both tech-nical change and process change. This kind of organization isin the best position to focus on business strategy without wor-rying about limits in its capability. It can pursue various alter-

natives of ERP implementation. An expert organization mayover-customize both the process and the system so it shouldguard against the temptation to use all available expertise.

In addition to assessing its overall capability, an organizationshould also evaluate its capability with respect to each processand each module within the ERP system. An organization soverall capability may be different from that with respect toindividual processes and modules. For example, an organiza-tion may regard itself as an organizer overall, but consider it-self a novice in manufacturing because it has little experiencein changing its manufacturing processes.

Both technical change and process change capabilities willchange over time as an organization goes through the ERP im-plementation process. A novice organization, for example, mayimprove its technical change ability by employing consulting

rms and sending employees to technical training. Similarly,a technician organization can acquire process change abilityby employing consultants to help it learn from earlier im-

plementation projects. So organizations should periodicallyevaluate their own capability, identify their needs for capa-bility improvements, and direct necessary resources to meetthose needs.

V. A F RAMEWORK FOR EVALUATING ERPCUSTOMIZATION CHOICES

The framework presented in Table IV combines our dis-cussion of process and technical customization options withprocess and technical change capabilities. The table shows thecustomization choices that are feasible for various organiza-

tional capabilities. For instance, a novice has the customizationoptions in the upper left of the framework. An expert, on theother hand, can choose from any of the customization possi-bilities. Thus, an expert has far more options available to itthan a novice. Customization options that require more processchanges are better suited for organizers than for technicians. Onthe other hand, customization options that require ERP systemchanges are appropriate for technicians.

Because both technical capability and process capability of an organization may evolve over time, previously unrealisticoptions may become available as organizations learn from theirexperiences. Process change capability may improve when cer-tain business processes are improved. Technical capability mayimprove when organizations gain experience in project man-agement, acquire more technical talent, and so forth. Because


8/12


capabilities are dynamic, organizations should anticipate andfacilitate capability growth. For example, in its initial selectionof ERP implementation projects and associated customizationoptions, an organization should choose projects and optionsthat will allow it to grow in the future.

VI. FRAMEWORK APPLICATION : A CASE STUDYThe purpose of the proposed framework is to assist the

evaluation of ERP customization options. As such, it providesa methodology for choosing customization options based onan organization s technical and process capabilities. It suggeststhat customization decisions should involve: 1) determiningthe degree of system and process customization desired;2) determining the organization s capabilities to do technicaland process customization; and 3) selecting a feasible cell thatmatches customization options with capabilities.

Given a desired ERP customization option, the framework can help assess the needs for additional technical change and/ororganization change capabilities. Such needs can then be metby obtaining external consultants, offering training to existingemployees, and deploying change agents.

Since in many companies ERP system implementation is aseries of implementation projects, the framework can be used toplan these projects by anticipating and facilitating the growth of technical and process change capabilities and choosing projectsand options that allow for capability growth over time. Takingthis dynamic view, we can envision dif cult customization andimplementation projects becoming feasible over time as capa-bilities build during prior implementation projects.

In the remainder of this section, we illustrate the use of theframework through an analysis of an ERP system implementa-

tion at a private technological university. While this case studyis a post-hoc analysis since the framework did not exist whenthe implementation projects started, the analysis shows the im-plicit technical and process customization choices made, howthese choices changed over time, andhow capabilitieswere builtthrough customization choices.

This private university (PU) has a rather unique undergrad-uate academic curriculum and schedule. Instead of 15-week semesters, PU employs a 7-week term system, where there arefour terms in each academic year. All undergraduate studentsare required to do junior and senior projects. This project-ori-ented curriculum, started in the 1970s, distinguishes it fromother technological universities and is the core of its academic

program. To support university operations, PU adopted andbegan to implement the Banner system as early as 1988.

Banner is an ERP system designed for educational institu-tions [50]. As such, Banner has modules for students, alumniand development, nancial aid, and room scheduling, as wellas modules common to most business ERP systems, such asnance, human resources, and payroll. Over 1300 universi-ties and colleges worldwide have adopted Banner systems,including University of Arizona and Yale University [ 50]. Im-plementing ERP systems, such as Banner, in universities is noless complicated than that in large corporations since it couldinvolve tens of thousands of users from various constituentsincluding current and prospective students, faculty, staff, ad-ministrators, and alumni. Many universities have fragmentedcomputer systems attending different business functions that

have never been integrated. Each university has unique busi-ness processes that distinguish itself from other competitors.Furthermore, ever changing government regulations play amajor role in how state universities operate. In sum, the Bannersystem and ERP implementations in higher education possessthe same general characteristics of ERP implementations usingSAP in large business organizations.

PU selected Banner because it provided a common databaseand integrated applications using this common database. PUwas an early adopter and served as a beta site for the Bannersystem. Over time, PU provided input to the vendor developingBanner, System and Computer Technology (SCT), and hasimplemented Banner modules, as they became ready for betatesting. The student module went online in 1988, the alumniand development module in 1995, the nance module in 1998,the HR/payroll module in 1999, the nancial aid module inearly 2000, and implementation of the scheduling module is inprocess.

The general implementation process for each module was

as follows. First, the department or departments were selected.At PU, ERP implementation has taken primarily a functionalorientation because, for many of the modules, a single pri-mary department is responsible for the functions covered bythe module. Second, members of the selected department andthe implementation team decided how much process reengi-neering and redesign to do. This decision was made from acontinuous improvement viewpoint. In making this decision,the functionality available in the Banner module was consid-ered. Finally, the submodules within the module were selected,conguration options were selected, and code was added orchanged to t the module to the redesigned process. Overall,PU rst made process customization choices with some con-

sideration of the module functionality, and then customizedthe system modules to whatever extent was needed to achievet. Because of PU s technical expertise and its role as thebeta test site, such a sequence was feasible.

Over time, the customization choices made have movedfrom primarily technical customization to primarily processcustomization, but these choices depend on the criticality of the functionality to the mission of PU. We illustrate the natureof these choices by discussing the implementation of threedifferent modules, student, nance, and scheduling. Table Vshows the framework as applied to PU s customization choicesfor these three modules.

A. Student Module

The student module implementation began in 1988. At thattime, the implementation team viewed ERP implementation inthe context of traditional systems design and development. Theteam determined what the primary user, the registrar s of ce,wanted and what the process was. They then worked withSCT to acquire technical expertise that allowed them to makemajor changes to the student module in order to t the existingprocess. That is, they chose the system conversion cell, whichis feasible only if the implementation team possesses technicalchange capabilities (labeled Student Module 1 in Table V).

Technical customization of the student module is necessarybecause educational programs are the core advantage andcompetency of an educational institution. In particular, PU has


9/12


TABLE VFRAMEWORK APPLIED TO PUS ERP IMPLEMENTATION CHOICES

a project-oriented undergraduate program. Although studentstake courses, their curriculum is driven by the completion of three signi cant projects, a humanities suf ciency project, aproject involving the interaction of technology and society, and

a project demonstrating competency in their major area of study.The last two projects have credit equivalent to three courses. Inaddition, the academic year is divided into seven-week terms(half semesters) and students may be off-site for one termduring the year at one of PU s global project centers in London,Bangkok, etc. This unique educational design is consideredPUs competitive advantage, and cannot be changed simplybecause the ERP system does not support such functionality.

The heavily customized student module, however, requiressignicant resources to maintain because new versions of the student module need to be implemented. For example,the recent implementation of the new web-based registrationsubmodule required signi cant customization for registration

in seven-week terms and registration for projects.Through the implementation of the student module, the teamlearned the costs and bene ts of relying on PU s techniciancapabilities. The bene ts are a well-functioning customizedmodule that supports PU s project-oriented educational mis-sion. The cost is over customization through overuse of itstechnician capabilities. It was realized that not all technicalchanges to the ERP system provided strategic bene t to PU.Current implementations of student sub-modules take a morebalancedapproach between technicalcustomization andprocesscustomization (labeled Student Module 2 in Table V).

B. Finance Module

The implementation of the nance module took a very dif-ferent approach. The process by which PU does its accounting

and manages its nances was similar to most other colleges anduniversities and was not considered as a competitive advantagefor PU. Learning from the costs of technical customization of the student module, the implementation team decided to mini-

mize technical customization, and focused on training users forthe nancial process embedded in Banner s nancial module.That is, the t process to system cell was selected, whichrequires expert capabilities, i.e., both technician and organizerabilities (labeled Finance Module in Table V).

PUs technical change capabilities were well developed, butits organizer abilities were lacking. To develop the organiza-tional change capability, PU initiated a reengineering project toimprove the operation of the process. The reengineering wasdone in cooperation with consultants from SCT so that the re-sulting process would be a close t to the process embedded inthe Banner system. To t the process to the system, PU mademany changes in its nancial processes, including changing itsentire chart of accounts to t Banner, changing retirement de-duction calculations to the method used in Banner, and trainingall users that normal account balances were now negative. Thechanges to the process required negotiations with the user com-munity and extensive training, both of which have served tobuild PUs organizer capabilities.

C. Scheduling Module

Decisions about implementing the scheduling module are un-derway now. The implementation team knows that the changesrequired are at least those of the mutual adaptation cell, thatis, at least incremental process change and table customization.A concern is that the process changes may be radical rather


10/12


than incremental and the system changes may require signi -cant coding. There is a major difference in project dif culty andrisk between a mutual adaptation project and a system andprocess reengineering project (labeled Scheduling Module inTable V). On the system side, there is concern that Banner sscheduling module does not have adequate functionality and,

thus, implementation may require coding. On the process side,the PU team is deciding how much reengineering is needed toimprove thescheduling process. Forexample, one issue is whichrooms shouldbe centrally scheduled. Departments like the exi-bilityof having local scheduling control over various conferencerooms andother space near their department. Central schedulingof such space, however, may provide better utilization of lim-ited space. Making decisions about whether and how to changescheduling of space on campus is a critical part of the ERP im-plementation process. Since scheduling decisions affect mostdepartments and individuals in these departments, changing thescheduling process radically could be risky.

These decisions about reengineering the scheduling processdetermine the process customization choice, that is, whetherimplementing the scheduling module requires incremental orradical process change, which in turn affects the organiza-tional capabilities required and the size, scope, and risk of theimplementation project. Through the implementation of thestudent module and the nance module, PU has built expertlevel ERP implementation capabilities. Thus, deciding to doradical process reengineering and signi cant code customiza-tion is feasible, which gives PU the exibility to choose anycustomization option. PU or any other organization must, how-ever, avoid projects that are more complex than needed.

The above discussion illustrates the dynamic use of the

framework for evaluating ERP customization choices. At PU,the implementation of each ERP module, and sometimes eachsubmodule, is an implementation project in itself. Each projectinvolves decisions about how much to change the associatedbusiness process and, simultaneously, how much module cus-tomization to allow. These choices must be made within thecontext of organizational capabilities. Since organizational ca-pabilities develop and change over time, the sequencing of implementation projects is important, so that each project isfeasible from the perspective of organizational capabilities atthe time it is initiated.

VII. CONCLUSION

The framework developed in this paper identi es nine cus-tomization options based on the degree of changes undertakenfor both the ERP system and the business process. It is de-signed to help organizations understand which customizationoptions are available and which of these are feasible given anorganization s capabilities. The applicability of the framework was illustrated through a case study of an organization thatemployed a phased implementation of several ERP modules.This phased implementation also illustrated the growing capa-bility of an organization as it undertook ERP implementationsinvolving technical and organizational changes and how addi-tional customization options became feasible with increasedorganizational capabilities.

A. Contributions to Practice

This framework allows ERP implementers to examine manyimplementation possibilities rather than simply following theconventional wisdom of tting processes to the system. Theframework shows nine customization options that depend onhow much to change the business process and how much to

change the ERP system. To illustrate the contribution of thisframework to the practice of engineering management, we de-scribe how a typical organization implementing an ERP systemwould use the framework.

The rst step in using the framework is to assess how well theselected ERP system matches the business process. Typically,this must be assessed for multiple ERP modules and multiplebusiness processes. Then, the organization can use the frame-work to think about the possibilities of changing the system andchanging the business process to provide better t between thesystem and the process. As a result, the organization will havea set of options, some involving more process change and some

involving more system change, for achieving better t.The next step is to assess the organization s ability to makethese changes. The paper describes organizational capabilitiesas technical change capabilities and process change capabilities.Depending on the results of assessing these capabilities, the listof options developed in the previous step can be evaluated forfeasibility for that organization. The nal step is to considera longer-term plan depending on which options are the mostfeasible for the organization over time.

We have suggested that ERP system implementation shouldbe viewed as a series of implementation projects. This viewhas several implications for engineering management. First, or-ganizations have overall process change and technical changecapabilities that may differ from their change capabilities withrespect to an individual project. Second, these capabilities canchange over time as organizations learn from previous projects.Third, it is important to plan thepath of implementation projectsso that more dif cult projects become feasible through learning.That is, an organization should not only focus on the customiza-tion options for the current projects but should also consider op-tions for capability growth.

The result of such an approach is to view ERP implemen-tation as a portfolio of projects. Different projects may requiredifferent levels of effort, resources, and expertise. Thus, theyshould be managed in their own ways. The framework provides

a way of thinking about the implementation choices to be madeand helps engineering managers understand and evaluate thesechoices.

B. Contributions to Research

The conventional wisdom in the literature is that orga-nizations should change their business processes to t theERP system, which embeds best practices. Any review of case studies of ERP system implementation will nd manyexceptions to the conventional wisdom. Managers need betteradvice from the literature.

To date, there are frameworks in the literature describingthe possibilities for technical changes, that is, options foradapting the ERP system to the business process, e.g., [ 12],


11/12


[15]. Our framework goes beyond these in several ways.First, we consider both technical and process changes, whichare the options managers actually have available to them.Second, we consider these customization options in light of an organization s capabilities to make these changes. Third, wepromote a dynamic view of ERP implementation that assesses

the building of organizational capabilities through a series of ERP implementations and how this increases the feasibility of various customization options over time. Finally, the framework can be used to generate testable hypotheses of the relationshipsbetween customization choices and organization capabilities.

C. Limitations

The development of the framework was motivated by prac-ticing engineering managers trying to make decisions aboutERP customization choices. The framework itself was devel-oped from the literature, and extends existing frameworks. Wehave demonstrated the applicability of the framework througha case study. This case study, however, is not a full validationof the framework; it only illustrates its applicability in oneorganization. ERP customization is only one of the importantissues in ERP implementation. Other issues such as planning,management support, and change management can be equallyimportant. Therefore, this framework should be used togetherwith others tools to address all important issues in ERP imple-mentation by the project team.

Furthermore, the framework does not determine decisions formanagers; rather it provides the possibilities for customizationand indicates the level of technical and organizational changecapabilities needed to implement each possible customizationoption. Theframework is designed to help managers think aboutthe feasible options, and should be used in that way.

D. Future Research

Engineering managers can use the framework, as is, in themanner suggested above. The framework, however, requiresfurther validation by testing it both in experiments and in orga-nizations. A future research study could compare the decisionsof managers using the framework with those not using theframework in a controlled laboratory study. Once further under-standing of the framework in use is developed, the framework should be tested in a similar way in multiple organizations tovalid the framework more fully than was done in this study.

REFERENCES[1] (2002) AMR Research predicts enterprise applications market will

reach $70 billion in 2006. AMR Research. [Online]. Available:www.amrresearch.com

[2] C. Brown and I. Vessey, ERP implementation approaches: Toward acontingency framework, in Proc. 20th Int. Conf. Information Systems ,Charlotte, NC, Dec. 13 15, 1999, pp. 411 416.

[3] M. L. Markus and C. Tanis, The enterprise systems experience Fromadoptionto success, in Framing the Domains of IT Research: Glimpsingthe Future through the Past , R. W. Zmud, Ed. Cincinnati, OH: Pin-naex Educational Resources Inc., 2000, pp. 173 207.

[4] M.-C. Boudreau and D. Robey, Organizational transition to enterpriseresource planning systems: Theoretical choices for process research, inProc. 20th Int. Conf. Information Systems , Charlotte, NC, Dec. 13 15,1999, pp. 291 299.

[5] C. H. Deutsch, Software that canmakea grown company cry, TheNewYork Times , Nov. 1998.

[6] H. Akkermans and K. van Helden, Vicious and virtuous cycles in ERPimplementation: A case study of interrelations between critical successfactors, Eur. J. Inform. Syst. , vol. 11, no. 1, 2002.

[7] M. L. Markus, C. Tanis, and P. C. van Fenema, Multisite ERP imple-mentations, Commun. ACM , vol. 43, no. 4, 2000.

[8] C. Brown and I. Vessey, Managingthe nextwave of enterprise systems:Leveraging lessons from ERP, MIS Quart. Executive , vol. 2, no. 1, pp.6577, 2003.

[9] J. E. Scott, The FoxMeyer Drugs bankruptcy: Was it a failure of ERP?, in Proc. 5th Americas Conf. Information Systems , Milwaukee,WI, Aug. 13 15, 1999, pp. 223 225.

[10] E. Nelson and E. Ramstad, Hersheys biggest dud is its new computersystem, The Wall Street J. , Oct. 1999.

[11] O. Volkoff, Usingthe structurational model of technology to analyze anERP implementation, in Proc. 5th Americas Conf. Information Systems ,Milwaukee, WI, Aug. 13 15, 1999, pp. 235 237.

[12] T. H. Davenport, Putting the enterprise into the enterprise system, Harv. Bus. Rev. , pp. 121131, July-August 1998.

[13] C. P. Holland and B. Light, A critical success factors model for ERPimplementation, IEEE Software , pp. 3036, May/June 1999.

[14] D. Robey, J. W. Ross, and M.-C. Boudreau, Learning to implemententerprise systems: An exploratory study of the dialectics of change, J. Manage. Inform. Syst. , vol. 19, no. 1, 2002.

[15] L. Brehm, A. Heinzl, and M. L. Markus, Tailoring ERP systems: Aspectrum of choices and their implications, in Proc. 34th Annu. Hawaii Int. Conf. Systems Sciences , HI, 2001.

[16] M. Hammer, Up the ERP revolution, Informationweek , p. 186, Feb.1999.

[17] H. C. Lucus, Implementation: The Key to Successful Information Sys-tems . New York: Columbia Univ. Press.

[18] R. P. Cerveny and G. L. Sanders, Implementation and structural vari-ables, Inform. Manage. , vol. 11, no. 4, pp. 191 198, 1987.

[19] V. Sambamurthy and L. J. Kirsch, An integrative framework of the in-formation systems development process, Decision Sci. , vol. 31, no. 2,pp. 391411, 2000.

[20] S. Sawyer, Packaged software: Implications of the differences fromcustom approaches to software development, Eur. J. Inform. Syst. , vol.9, pp. 4758, 2000.

[21] E. Carmeland S. Sawyer, Packagedsoftwaredevelopmentteams: Whatmakes themdifferent?, Inform. Technol. People ,vol.11,no.1,pp.7 19,1998.

[22] R. K. Lynch, Nine pitfalls in implementing packaged applications soft-ware, J. Inform. Syst. Manage. , vol. 2, no. 2, pp. 88 92, 1985.

[23] M. Keil and E. Carmel, Customer-developer links in software develop-ment, Commun. ACM , vol. 38, no. 5, pp. 33 44, 1995.

[24] D. Gefen, Nurturing clients trust to encourage engagement successduring the customization of ERP systems, Omega , vol. 30, no. 4, 2002.

[25] H. C. Lucas Jr., E. J. Walton, and M. J. Ginzberg, Implementing pack-aged software, MIS Quart. , vol. 12, no. 4, pp. 88 92, 1985.

[26] K. Kumar and J. V. Hillegersberg, ERP experiences and evolution, Commun. ACM , vol. 43, no. 4, pp. 23 26, 2000.

[27] C. Soh, S. S. Kien, and J. Tay-Yap, Cultural ts and mis ts: Is ERP auniversal solution?, Commun. ACM , vol. 43, no. 4, 2000.

[28] R. Davison, Cultural complications of ERP, Commun. ACM , vol. 45,no. 7, pp. 109 111, 2002.

[29] L. M. Applegate, F. W. McFarlan, and J. L. McKenney, Corporate In- formation Systems Management: Text and Cases , 5th ed. New York:

McGraw-Hill, 1999.[30] Y. Van Everdingen, J. Hilergersberg, and E. Waarts, ERP adoption byeuropeanmidsize companies, Commun.ACM , vol. 43, no. 2,pp. 27 31,2000.

[31] K. Hong and Y. Kim, The critical success factors for ERP implemen-tation: An organizational t perspective, Inform. Manage. , vol. 40, no.1, 2002.

[32] K. S. Lassila and J. C. Brancheau, Adoption and utilization of com-mercial software packages: Exploring utilization equilibria, transitions,triggers, and tracks, J. Manage. Inform. Syst. , vol. 16, no. 2, pp. 63 90,1999.

[33] R. L. Glass, Enterprise resource planning Breakthrough and/or termproblems?, Database for Advances Inform. Syst. , vol. 29, no. 2, pp.1416, 1998.

[34] J. Ross, The ERP Revolution: Surviving Versus Thriving , MIT SloanWorking Paper, Center for Information Systems Research, 1998.

[35] C. Koh, C. Soh, and M. L. Markus, A process theory approach toERP implementation and impacts: The case of Revel Asia, J. Inform.Technol. Cases and Applicat. , vol. 2, no. 1, pp. 4 23, 2000.


12/12


[36] P. Rajagopal, An innovation-diffusion view of implementation of en-terprise resource planning(ERP) systems and development of a researchmodel, Inform. Manage. , vol. 40, no. 2, pp. 87 114, 2002.

[37] L. Brehm and M. L. Markus, The divided software life cycle of ERPpackages, presentedat the 1st Global Information TechnologyManage-ment World Conf., Memphis, TN, 2000.

[38] J. L. Whitten and L. D. Bentley, Systems Analysis and Design Methods ,4th ed. New York: McGraw-Hill, 1998.

[39] E. K. Clemons and M. C. Row, Sustaining IT advantage: The role of structural differences, MIS Quart. , pp. 275292, 1991.[40] F. J. Mata, W. L. Fuerst, and J. B. Barney, Information technology

and sustained competitive advantage: A resource-based analysis, MIS Quart. , vol. 19, no. 4, pp. 487 505, 1995.

[41] D. F. Feeny and L. Willcocks, Core IS capabilities for exploiting infor-mation technology, Sloan Manage. Rev. , pp. 921, 1998.

[42] A. S. Bharadwaj, A resource-based perspective on information tech-nology capability and rm performance: An empirical investigation, MIS Quart. , vol. 24, no. 1, pp. 169 196, 2000.

[43] N. B. Duncan, Capturing exibility of information technology infra-structure: A study of resource characteristics and their measure, J. Manage. Inform. Syst. , vol. 12, no. 2, pp. 37 57, 1995.

[44] D. Sprott and D. , Componentizing the enterprise application pack-ages, Commun. ACM , vol. 43, no. 4, 2000.

[45] N. Engler, ERP heats up thermacore, Comput. Reseller News , pp.6869, May 1999.

[46] D. A. Leishman, Solution customization, IBM Syst. J. , vol. 38, no. 1,pp. 7697, 1999.

[47] J. E. Scott and L. Kaindl, Enhancing functionality in an enterprise soft-ware package, Inform. Manage. , vol. 37, no. 3, p. 111, April 2000.

[48] V. Grover, Business Process Change: Concepts, Methods and Technolo-gies , W. J. Kettinger, Ed. Harrisburg, PA: Idea Group, 1995.

[49] M. Hammer, Beyond Reengineering: How the Process-Centered Organ-ization is Changing Our Work and Our Lives . New York: Harperbusi-ness, 1996.

[50] (2003) SCT Industry Solutions for Education. SCT Corporation.[Online]. Available: http://www.sct.com/Education/Products/Banner/ index.html

Wenhong Luo received the Ph.D. degree in manage-ment information systems from University of Ken-tucky, Lexington.

He is an Associate Professor in the Departmentof Decision and Information Technologies, Collegeof Commerce and Finance at Villanova University,Villanova, PA. His publications have appeared inleading journals such as Communications of the

ACM , European Journal of Information Systems , Information and Management, The InformationSociety, Computer Supported Cooperative Work ,

and Group Decisions and Negotiation . His research interests include electroniccommerce and information system project management.

Diane M. Strong received the Ph.D. degree in in-formation systems from Carnegie Mellon University,Pittsburgh, PA.

She is an Associate Professor in the Departmentof Management at Worcester Polytechnic Institute,Worcester, MA, and Director of the MIS program.Her publications have appeared in leading journalssuch as Communications of the ACM , Communica-tions of the AIS , Journal of Management InformationSystems , ACM Transactions on Information Systems , Journal of Database Management , Journal of Sys-

tems and Software and Information and Management . Her research centers ondata and information quality and on the organizational impacts of MIS applica-tions systems, especially enterprise systems.

Dr. Strong is serving on the Association for Information Systems (AIS)Council and was Program Co-Chair for Americas Conference on InformationSystems (AMCIS), Boston, MA, and for the International Conference onInformation Quality.

erpimpeval[1]

Documents