a model to measure the business value of information ... · a model to measure the business value...
TRANSCRIPT
A Model to Measure the Business Value of Information Technology: The Case of Project and Information Work
Indranil Bardhan The University of Texas at Dallas
Vish Krishnan The University of Texas at Austin
Shu Lin The University of Texas at Dallas [email protected]
Comments on an earlier version of this paper from Gregory Dess, V. Sambamurthy, participants at the 2004 INFORMS and DSI conferences, and seminar participants at the University of Texas at Dallas and the University of Texas at Austin are gratefully acknowledged. _________________________________________________________________________
ABSTRACT
The impact of information technology (IT) on improving the productivity of information work has been mixed. While some firms have realized gains, many others have found the benefits to be elusive. Prior research has primarily focused on the impact of IT spending on firm-level performance. To further this line of research, it is necessary to understand the operational and process-level changes and isolate the impact of specific types of IT applications on organizational dynamic capabilities which mediate the impact of IT on firm performance. We measure the impact of different types of enterprise information technologies on firm performance based on two types of outcomes: (a) business process-level measures such as project/program quality, cycle times, cost, and completion rates, and (b) firm-level financial performance measures such as gross margin, return on assets, and return on equity. The central research contribution is the development of a research framework to improve our understanding of the operational impact of information work on the development of dynamic organizational capabilities which explains differences in process- and firm-level performance across a cross-section of companies.
Keywords: Information work productivity, Dynamic capabilities, Effectiveness, Efficiency,
Project outcomes, Firm performance.
1
1.0 INTRODUCTION
Improving information and knowledge work productivity is the next frontier for
information technology (IT) offerings and applications. Technology has delivered significant
and documented productivity improvements in the manufacturing sector of the economy
(Weill, 1992; Barua et al., 1995). Information work – defined as the “creation, coordination,
integration and management of information within a firm and its value network (Conner and
Prahalad, 1996)” - is in its rudimentary stages of being mapped, measured, and managed.
The intangibility and diversity of information work and its outputs presents unique
challenges in its description, measurement, and improvement. Prior research has focused
primarily on the aggregate impact of IT spending on firm performance (Barua et al., 1995;
Brynjolfsson and Hitt, 1996; 1998; Kohli and Devaraj, 2003). The deeper, operational and
process-level impact of software applications are only currently being examined in detail.
To develop a finer and a more nuanced understanding of the business value of IT, we
focus on the improvement in project work productivity and performance. In the information
economy, projects are important vehicles for firms to complete work and realize value. Tom
Peters (Peters, 1999) goes as far as to state that, “In the new economy, all work is project
work.” Focusing on projects or programs (i.e., collection of projects) allows us to study the
impact of IT on information work in multiple firms while using a common vocabulary and
drawing on a large body of knowledge on project management.
Most prior studies, dealing with relationships between IT and organizational
performance, evaluate the business value of IT by lumping IT-related spending under one
category termed “IT capital.” While IT capital is a useful measure of firm-level IT
investments, it needs to be complemented with additional insights into the usage and impact
2
of different types of IT, including software capabilities, at the business process level. Since
IT investment decisions are often made at the application level, it is important to trace the
impact of different classes of IT applications on organizational capabilities and firm
performance (Mukhopadhyay et al., 1997).
We focus on the role of information work, the relationships between three classes of
major IT applications used by managers to plan, control, and manage projects, and their
impact on organizational capabilities and performance. We draw on the theory of dynamic
capabilities to explain how firms create competitive value by creating relevant organizational
routines to leverage their IT assets (Zhu and Kraemer, 2002). We propose and validate a
model that enhances our understanding of the impact of IT applications on project and firm
performance. Our research questions can be summarized as follows: (a) How do we measure
the value of information work defined in terms of organizational capabilities that are enabled
by IT?, (b) what are the mechanisms by which IT has an impact on project and process-level
outcomes? and (c) what is the impact of IT-enabled capabilities on firm performance?
We validate our research using survey data from project and program managers and
executives drawn from a large cross-section of US firms as well as financial data for a subset
of the publicly traded firms that participated in the survey. Our results suggest that IT-
enabled dynamic capabilities have a significant impact on improvement in four project-level
outcomes (cycle time, cost, quality, and on-time completion rate), and two of the four firm-
level financial measures. Our research integrates two distinct bodies of literature: one that
draws on the strategic management literature, dealing with the relationships between IT
usage and dynamic capabilities, and the other that draws on the IT productivity literature
dealing with the impact of IT on information work.
3
2.0 BACKGROUND
One of the difficulties in managing information work productivity is the challenge in
measuring the effects of implementing IT. In industrial work, the effect of technology was to
increase the output and decrease the inputs needed for processing. IT may have more
intangible effects on the output than merely increasing the quantity produced (Fichman,
2004). A close case-based examination of the literature and industrial practice reveals that
besides improving output, technology also can help improve quality and consistency of
information work and help the organization create better products/services through improved
market intelligence and customer insight. The networked digital environment enables a firm
to keep the customer engaged throughout the project, learn third-party customer information
from online media, leverage the work of global suppliers, and reuse knowledge assets from
other organizational entities (Barua et al., 2004).
We draw on the theory of dynamic capabilities which explains how firms create value
by creating relevant organizational processes and routines to leverage their IT assets (Teece
et al., 1997). Rooted in the resource-based view (RBV) of the firm, dynamic capabilities
represent the organizational and strategic routines that help firms respond to changing market
conditions and create sustainable advantage by assembling inimitable resources to create
organizational capabilities (Wernerfelt, 1984; Barney, 1991, Eisenhardt and Martin, 2000,
Iansiti and Clark, 1994).
Dynamic capabilities encompass different types of identifiable routines that have
been the subject of extensive empirical research (Pavlou and El Sawy, 2004). For example,
effective product development projects are characterized by the use of cross-functional teams
13 The four dimensions of effectiveness are highly correlated, with pairwise correlations exceeding 0.75. Hence, our approach to combine them into one meta-construct is validated by the results of the EFA.
4
that leverage different sources of expertise which enhances the breadth of available
information, and enable coordination and concurrency of manufacturing, marketing, and
design activities (Eisenhardt and Martin, 2000). Similarly, strategic decision making is an
example of a dynamic capability for integrating project resources wherein project managers
pool their diverse expertise to make project investment choices that are aligned with the
strategic direction of the firm (Eisenhardt, 1989). Other dynamic capabilities associated with
project management include effective external communication capabilities (Clark and
Fujimoto, 1991), knowledge transfer and reuse capabilities which include routines for
replication and brokering (Hansen, 1999, Hargadon and Sutton, 1997), and patching or
continual realignment of business processes, resources, and organization to changing market
conditions and customer needs (Eisenhardt and Brown, 1999). Dell’s constant segmentation
of its operating units to better align with shifting customer requirements, organizational
efforts to leverage supplier and internal knowledge assets, and practices to capture and use
customer feedback are examples of these dynamic capabilities (Magretta, 1998). Effective
alliance and acquisition processes for bringing new resources into the firm also represent
important dynamic capabilities, since they enable managers to assemble advanced technical
know-how for superior performance (Zollo and Winter, 2002).
IT assets can provide the building blocks for business processes to enhance their
dynamic capabilities (Amit and Schoemaker, 1993; Tallon and Kraemer, 2004). Although IT
assets may be viewed as tangible resources, which are mobile and imitable, they enable
unique organizational routines and are often bundled with an organization’s commitment to
specific business processes (Wade and Hulland, 2004). Hence, firms can develop these IT-
enabled capabilities through multiple paths, and independently of other firms. Eisenhardt
5
and Martin (2000) argue that, even though there may be commonalities in the key features of
effective dynamic capabilities, they can actually be different in terms of many details related
to the resources used and trajectory of their adoption paths. Therefore, we draw upon
dynamic capabilities theory to provide the theoretical foundation for studying the context in
which information technologies may lead to better project and firm performance.
3.0 RESEARCH FRAMEWORK
Project and program management work may typically involve a broad range of
activities, including product management, coordination and integration management, and
process management (Harter and Slaughter, 2003). Also, a broad range of information
technologies are used in a variety of industries to improve project performance. To be able to
measure information work productivity, we first develop a typology to distinguish among
different types of information technologies that are typically used for project/program
management. This typology helps us relate various technology types to different kinds of
improvement possible in project and process performance.
3.1 Technology Typology
Our classification is driven by the nature of interdependencies that exist in different
projects. While there is enormous diversity in the types of information technologies observed
in organizations, the nature of task interdependencies inherent in different types of project
management activities entails usage of different types of IT. There are different levels of
task interdependencies (Thompson, 1967). Pooled interdependencies are common in
activities that are coordinated by common standards and/or rules-based mechanisms and
where project teams share common IT resources but are otherwise independent (Kumar and
van Dissel, 1996). Sequential interdependencies are common where the tasks of each
6
module are distinct and serially structured so that the output of one unit is likely to affect
subsequent downstream tasks. Documentation tasks in project and product management
activities are sequential since they receive their primary inputs from the development teams
(Harter and Slaughter, 2003). Reciprocal interdependencies occur when there is mutual
exchange of information and tasks require ongoing adjustments and adaptation between work
units. Project teams that are engaged in tasks with reciprocal interdependencies are tightly
coupled to the timing of others. Hence, the nature of interdependencies drives the level of
usage of different types of IT, since tasks that are tightly coupled may require a specific class
of collaboration technologies compared to others that are more independent.
Building upon this body of prior research, we now classify the information
technologies typically used for project/program management into three groups: Core
Communication Technologies (CCT), Enterprise Computing Technologies (ECT), and Group
Collaboration Technologies (GCT). CCTs, which include basic technologies, such as e-mail
and web portals, support tasks that entail pooled interdependencies where IT resources are
shared by all entities. ECTs institutionalize sequential interactions between work units and
support structured sequential interactions between users which enable them to access and
exchange data in a structured format. ECTs encompass technologies, such as enterprise
resource planning (ERP) software, which facilitates information exchange where output from
one functional area serves as the input to other functional areas. GCTs support reciprocal
interdependencies where communication involves information that is exchanged, processed,
and adapted by users. These systems involve collaboration technologies such as groupware,
online teamspaces and discussion databases, instant messaging and video-conferencing,
which enable teams to communicate in real time (Carte and Chidambaram, 2004).
7
3.2 Efficiency versus Effectiveness of Information Work
An examination of the research literature and field studies of industrial practice
suggests that the impact of IT on information work can be classified based on the benefits
within the process and the interface with the customers, suppliers, and the extended
organization. While firm productivity is generally defined as a ratio of firms’ outputs
divided by their inputs, the measurement of information work productivity is complicated by
the fact that both the inputs and outputs associated with information work are often
intangibles. Most of the prior IT literature has focused on the efficiency aspects of
information work productivity which studies the quantity of outputs produced and the costs
associated with this effort. We propose a broader definition of information work productivity
that captures the impact of IT on the effectiveness and efficiency dimensions of information
work (Tallon et al., 2004). While efficiency refers to “doing things right,” effectiveness,
which refers to “doing the right things,” signifies achievement of specific organizational
objectives and is manifested in the attainment of sustained competitive advantage (Ray et al.,
2004; Melville et al., 2004). We now discuss these capabilities in greater detail.
3.2.1 Efficiency
Efficiency refers to the degree to which an organization is able to more cost- and
time-effectively achieve its targets by automating, streamlining, and accelerating its business
processes. IT improves efficiency in a number of ways including re-engineering of business
processes, reducing the latency and wait times of project tasks, and in the time spent in
saving/retrieving business information. Hence, the impact of IT on project efficiency can be
measured based on several factors including the time spent in searching and waiting for
information, project task durations, and elimination of rework and non-value added tasks.
8
3.2.2 Effectiveness
Effectiveness encompasses dynamic capabilities that pertain to the organization’s
ability to achieve more and better customer-perceived outcomes with fewer and lower-cost
resources. Specifically, effectiveness can be measured in terms of the following indicator
variables: (a) quality-consistency of the project outcome, (b) customer and market relevance
of the outcome (Zeithaml and Bitner, 1996), (c) degree to which the project effort is
structured to leverage the efforts of suppliers and other value chain partners (Yu et al., 2003),
and (d) the alignment of the effort with the firm’s and business unit’s strategy (McGrath and
Iansiti, 1999; Tallon et al., 2004; Nambisan, 2002).
Quality, a key component of effectiveness, is particularly difficult to measure for
information work, in contrast to manufacturing where it is measured as conformance to
specifications. One measure of quality we use is consistency, which refers to the extent to
which IT users are able to achieve targeted business results in a predictable manner and
achieve their stated goals. Consistency is also an indicator of the visibility the organization
has into its business results and improvement efforts. By helping to aggregate business
information across the enterprise, IT should improve a firm’s ability to track progress, spot
and correct deviations, and consistently execute on business plans and initiatives.
Alignment denotes the organization's ability to streamline its multiple functions,
departments, and initiatives towards a set of common business plans and goals. Centralized
IT infrastructure and communication technologies helps align projects and programs with the
company's business strategy, rationalizing project and product portfolio, and eliminate
redundant initiatives (McGrath and Iansiti, 1999). Another dimension of the effectiveness of
information work is the potential for IT to improve project/program relevance, which
9
measures the degree to which the users and their organizations are customer- and market-
oriented in terms of their ability to meet current and future customer needs (Barua et al.,
2004). Used appropriately, IT should help the firm build “higher bandwidth” communication
channels with its lead users and mainstream customers and capture tacit and emerging
customer information (Zeithaml and Bitner, 1996).
3.3 Research Model
The term “IT business value” refers to the impact of IT on organizational
performance, which is typically measured using intermediate process-level measures as well
as firm-level measures that represent both financial and operational metrics. The dynamic
capabilities theory provides a valid framework to analyze the impact of IT on organizational
performance due to its focus on resource attributes and its relevance for measuring IT and
non-IT resources (Peteraf and Barney, 2003; Wheeler, 2002). Improvements in firm
capabilities, in turn, drive improvements in business process performance as well as firm-
level measures such as profit margins. In our research context, the impact of IT-enabled
dynamic capabilities on improvements in business process performance is measured by
studying their impact on project outcomes, such as reduction in project cycle time, cost, and
on-time completion rates. Since projects represent important vehicles for firms to realize
business value, we also examine the impact of IT-enabled dynamic capabilities on firm
financial measures, such as gross margins and return on assets (Bharadwaj, 2000).
Our conceptual research model is shown in Figure 1. Our model examines the role of
dynamic capabilities in mediating the impact of different types of IT resources on project-
and firm-level measures of performance. We integrate the information processing view of IT
resources with the dynamic capabilities perspective to provide a sharper understanding of (a)
10
the differential impact of different types of IT and their usage, and (b) the role of
effectiveness and efficiency in mediating the impact of IT on process outcomes and firm
financial measures. Specifically, we incorporate the role of different types of software when
conceptualizing the IT constructs which has been not been addressed in prior studies
(Melville et al., 2004, pp. 312). Our model provides a sharper distinction of effectiveness
and efficiency, which enables us to measure the differences in their impact on performance.
3.4 Research Hypotheses
Impact of IT on Efficiency and Effectiveness
While the impact of aggregate IT resources on process efficiency has been
extensively studied in prior research, less attention has been given to studying the differential
impact of specific types of IT resources (Barua et al., 1995; Brynjolffson and Hitt, 1998).
We measure the impact of IT on process efficiency based on several factors, including the
time spent in searching and waiting for project information, task durations, and elimination
of rework and non-value added tasks. In a project/program management setting, basic
communication technologies (CCTs), such as e-mail and web portals, can improve efficiency
by reducing the latency of project tasks and in the time spent in searching for retrieving
information. Similarly, GCTs such as instant messaging and vide-conferencing technologies
enable teams to exchange project information in real-time and thereby reduce the wait times
involved in information transfer and making project decisions (Carte and Chidambaram,
2004). ECTs, such as document management systems and knowledge management systems,
provide efficient electronic storage and enable quick information retrieval which improves
process efficiency. Hence, we expect that usage of IT resources is associated with
improvements in the efficiency of organizational capabilities.
11
H1: Greater usage of IT resources is associated with an improvement in the efficiency of
organizational capabilities.
H1a: Greater CCT usage is associated with improvements in project/program efficiency.
H1b: Greater ECT usage is associated with improvements in project/program efficiency.
H1c: Greater GCT usage is associated with improvements in project/program efficiency.
We argue that measuring the effectiveness of organizational capabilities may be a
better approach to empirically test resource-based logic since it is possible for firms to
appropriate the economic profits generated by a firm’s processes before these profits are
reflected in the firm’s gross profit margins (Ray et al., 2004). ECTs, such as project
management software and business intelligence systems, enable firms to improve the
consistency of project execution by facilitating greater visibility to project data, allowing
managers to track progress more easily and identifying project risks. Similarly, customer
relationship management (CRM) software enable managers to identify and disseminate
customer requirements in a timely manner and allow greater customer involvement in project
decisions. Knowledge management systems and business intelligence software enable
project managers to identify emerging market trends and develop relevant product
enhancements and new market opportunities.
CCTs, such as project portals and e-mail communication technologies, enable project
teams to solicit customer and supplier involvement in key project decisions which can lead to
improvements in the quality of project decisions. They also facilitate re-use of product data
and other knowledge assets within and across project teams. Similarly, GCTs such as mobile
computing technologies (handheld computers, PDAs) facilitate quicker exchange of mission-
critical data, and thereby enable more effective alignment of distributed project teams. Other
12
technologies, such as groupware and instant messaging software, enable easier tracking of
projects and improve project visibility even though project teams may not be co-located.
Hence, we hypothesize that IT usage is associated with an improvement in the effectiveness
of organizational capabilities.
H2: Greater usage of IT resources is associated with an improvement in the effectiveness
of organizational capabilities.
H2a: Greater CCT usage is associated with improvements in project effectiveness.
H2b: Greater ECT usage is associated with improvements in project effectiveness.
H2c: Greater GCT usage is associated with improvements in project effectiveness.
We control for project characteristics, such as team size, extent of co-location of
project teams, project duration, and a firm’s propensity to use IT, that may also have an
impact on the development of firm dynamic capabilities. Team size dictates the extent of
information processing that is required to disseminate information across project teams. The
increasing use of virtual project teams, where team members may be distributed across
different offices in different countries, also has an impact on organizational capabilities. A
firm’s propensity to use IT, as measured by the level of adoption of emerging technologies,
may influence the types of IT resources that are used and thereby impact the effectiveness
and efficiency dimensions of organizational capabilities.
Impact of Human IT Capital
The “human IT capital” component of IT resources plays an important role in
determining the impact of IT on organizational performance (Bharadwaj, 2000; Ross et al.,
1996). It includes technical expertise skills, such as application development, maintenance,
and integration, as well as managerial expertise such as the ability to provide project
13
leadership, and provide the necessary financial and human resources (Melville et al., 2004).
Here, we focus on the managerial expertise component since prior researchers have argued
that successful application of IT needs to be accompanied by complementary organizational
changes such as organizational culture and workplace practices (Mata et al., 1995;
Brynjolffson and Hitt, 2000).
We draw upon the concept of organizational IQ, a construct that was proposed by
Mendelson (2000) to measure human IT capital within organizations. It measures the extent
of adoption of advanced management practices for project planning, execution and
management, which include competitive benchmarking of best practices, senior executive
leadership, and appropriate training in the use of work methods, processes and IT. Hence,
human IT capital represents the degree to which management facilitates knowledge
transparency, information awareness and decision-making capabilities within the firm
(Boynton et al., 1994; Armstrong and Sambamurthy, 1999). We expect that human IT
capital will be associated with improvements in dynamic organizational capabilities.
H3: Greater levels of human IT capital are associated with an improvement in firm dynamic
capabilities.
H3a: Greater human IT capital is associated with improvements in project effectiveness.
H3b: Greater human IT capital is associated with improvements in project efficiency.
Organizational Performance
Since a limitation of RBV is that it assumes that resources are applied in their best
uses, saying little about how this is done, we expand our research framework to better
understand how IT is applied to improve process performance. Although the impact of IT on
operational efficiency and competitive advantage has been supported in the IT business value
14
literature (Melville et al., 2004; Barua et al., 1995; Soh and Markus, 1995), the research
question that is not well-understood is “what are the mechanisms by which IT enables firms
to achieve efficiency and competitive advantage?” Recent work by Ray et al. (2004, pp. 35)
suggest that it is important to develop a better understanding of the relationship between a
firm’s resources and the effectiveness of its organizational routines. We argue that IT
resources enable improvements in dynamic capabilities which, in turn, lead to improvements
in process-level outcomes and firm-level performance.
Improvements in process efficiency, due to a reduction in project latency (wait times)
and ability to undertake concurrent development of multiple project tasks, is associated with
a reduction in project cycle times and improved project completion rates. Similarly, a
reduction in rework associated with projects work units/modules and removal of non-value
added tasks is associated with improvements in project quality and costs, since defects/errors
that are identified and corrected in later stages of the project life-cycle are costlier to fix.
Improvements in process effectiveness capabilities have an impact on project
outcomes. Improved consistency of project execution, through better transparency and risk
management, can reduce operating costs and increase project completion rates.
Improvements in a firm’s ability to leverage its existing network partners to manage its
sourcing processes is associated with reduction in project cycle times and costs. Similarly,
achieving better alignment through closure of functional gaps and distributed project teams,
can lead to improved project quality and cycle times. Hence, we expect that improvements
in firm effectiveness capabilities are associated with improvements in project outcomes.
H4: Improvements in firm dynamic capabilities are associated with improvements in
project-level outcomes.
15
H4a: Improvements in process efficiency are associated with improved project outcomes.
H4b: Improvements in the effectiveness of organizational capabilities are associated with
improvements in project outcomes.
Improvements in dynamic capabilities not only drive project outcomes, but are
associated with improvements in firm-level performance, since these capabilities proliferate
across projects and project teams. Improvements in a firm’s capabilities to effectively
leverage its suppliers to improve its sourcing processes, and thereby reduce its cost of sales,
is associated with improvements in gross margins. By improving their customer focus, firms
can extract higher premiums and create new markets, which are associated with
improvements in firm margins and return on assets (ROA). Similarly, improvements in
firms’ consistency of execution may be associated with better capabilities to leverage their
investments into more productive assets, which in turn drive higher returns on equity (ROE).
H5: Improvements in dynamic capabilities are associated with improvements in firm
financial performance.
We control for the impact of other organizational characteristics on project- and firm-
level performance. For instance, firm size may have an impact on the ability of firms to
realize improvements in process-level outcomes, since smaller firms may be more agile and
better able to respond to shifting market trends compared to larger firms. We also control for
the impact of project autonomy, process maturity, and the number of projects that are
concurrently managed, on organizational performance.
Since organizational dynamic capabilities link the impact of IT resources to
improvements in project- and firm-level performance, we frame our hypothesis as:
16
H6: Dynamic capabilities mediate the impact of technological IT resources on
improvements in project outcomes and firm performance
4.0 RESEARCH DATA
4.1 Data Collection
A cross-sectional survey was employed for data collection. We designed a research
survey to capture the organizational work characteristics, information technology usage, and
project performance data, as shown in the Appendix. The survey was administered online in
June 2004, through an independent, professional survey firm, to a random sample of project
and product managers drawn from US private- and public-sector organizations. A first phase
of pilot data was followed by a full-fledged data collection effort with 780 responses. The
final dataset of 625 respondents represents completed surveys where the respondents met the
survey selection criteria of “being a project/program/business manager in their firm.” Table
1 shows the profiles or respondents based on firm and project characteristics.
About 42% of the survey respondents were from publicly traded firms while the
remainder belonged to not for profit organizations, state or federal government agencies, and
privately held firms. About a third of the respondents were responsible for managing 4-5
projects concurrently, while 15% responded that they managed between 6-10 projects and
12% managed more than 10 projects concurrently. While 36% of the respondents indicated
that their projects lasted less than one year on average, an equal number responded that their
projects lasted between one and five years.
4.2 Construct Measurement
Constructs that represent usage of different types of IT resources and the
organizational dynamic capabilities have been discussed in the previous section.
17
Effectiveness is measured using a twelve item-scale where each of its dimensions are
represented by three indicator variables. We have proposed several variables to control for
the impact of project and firm characteristics on the measurement of IT productivity. Project
duration is measured as the average length of a typical project/program (measured in years),
while Team size represents the size of the project team measured as the number of full-time
equivalent (FTE) of project staff. Co-location measures the extent of dispersion among team
members. Higher values represented dispersed project teams while lower values represent
teams that are located in one location.
Autonomy measures the extent to which project teams have autonomy from
management in making project decisions (Mendelson, 2000). Process maturity represents
the degree of implementation of advanced process capabilities, and includes practices such as
cross-functional teams, robust project management processes, and customer feedback for
continuous process improvement (Harter and Slaughter, 2003). This construct was adapted
from the Capability Maturity Model–Integrated Product Development (IPD) framework.
The propensity of a firm to use IT is an important indicator of a firm’s willingness to
deploy emerging technologies to its business, and we use this variable to evaluate whether
firms are early adopters, mainstream users or late adopters of IT. We also collect data on the
number of concurrent projects managed since this is an important indicator of managers’
roles and their dedication to current projects. Firm size was used as a control variable to
study the impact of IT on firm-level performance. We measured firm size in two ways: the
number of firm employees and the natural logarithm of the dollar value of firm assets for the
publicly traded firms in our sample.
18
We measure organizational performance at two levels: change in project-level
outcomes and firm performance. We define four project outcomes which represent changes
in project cycle time, cost, quality, and on-time completion rate, during the prior one-year
period at the time of the survey. ∆(Cycle Time) represents the change in project cycle times,
defined as the total time elapsed from project initiation to completion. ∆(Cost) represents the
change in project costs, defined as the total program/project lifecycle cost incurred from
initiation to completion. ∆(Quality) represents the change in project quality, measured as the
total number of errors, defects, and rework associated with the project. Finally, ∆(On-time
completion rate) represents the change in the projects completed on time. These perceptual
measures represent outcomes for a typical (or average) project in the manager’s portfolio.
The three firm financial measures that we examined are: (a) Gross Margin, which is
defined as (Sales minus Cost of Sales) / Sales, (b) ROA, defined as (Net Income / Assets),
and (c) ROE, defined as (Net Income / Common Equity). Gross profit margins are
influenced by two factors: the price premium that a firm’s products command in the
marketplace, and the efficiency of the firm’s procurement and production processes. ROA
measures how much profit a firm is able to generate for each dollar of assets invested (Palepu
et al.,1997; Santhanam and Hartono, 2003). On the other hand, ROE is a more
comprehensive indicator of firm performance because it measures how well managers deploy
the resources invested by shareholders to generate returns. Hence, it provides a good
indicator of the extent of financial leverage of the firm.
4.3 Construct Validity and Reliability
Exploratory factor analyses (EFA) was first conducted to check if the proposed
factors are consistent with our survey data. The factor structures suggested by EFA are
19
consistent with the factors identified in our research model and they account for 62% of the
variance in the data. Since our survey data are self-reported, we performed a Harmon’s one-
factor test to check for common methods bias (Podsakoff and Organ, 1986). The items
measuring the independent variables and those measuring project outcomes loaded on
different factors, which do not indicate common methods bias. The Cronbach alpha values
for these factors range from 0.68 to 0.94 which indicates good internal consistency.3
We also performed a confirmatory factor analysis (CFA) to establish the reliability of
our constructs. Composite reliability reflects the internal consistency of the indicators and it
exceeded the recommended threshold of 0.7 for six of the seven constructs shown in Table 2
(Werts et al., 1974; Nunnally, 1978). The t-statistics for all factor loadings were significant
and suggest that our indicator measures satisfy convergent validity (Phillips and Bagozzi,
1986). To establish the discriminant validity of the constructs, we calculated the average
variance extracted (AVE), a ratio of the construct variance to the total variance among
indicators, which exceeds the threshold of 0.5 for all factors (Fornell and Larcker, 1981). We
calculated several goodness-of-fit statistics which confirm that the measurement model
provides a satisfactory explanation of the observed data.
5.0 EMPIRICAL RESULTS
Since our research framework represents a recursive model, we estimated the model
using ordinary least squares (OLS) regressions which provide consistent and efficient
estimates (Johnston, 1984, pp. 468-469). We also estimated the model using a structural
equation model (SEM) and these results are generally consistent with our OLS estimates.
20
5.1 Impact of IT Resources on Dynamic Capabilities
The impact of IT on the effectiveness and efficiency dimensions of dynamic
capabilities is shown in Table 3. The results indicate that there is a distinct divide in the
impact of information technologies on these two dimensions of information work. Both ECT
and CCT have a positive impact on the effectiveness of information work, in terms of their
ability to improve the consistency of project execution and leverage knowledge assets.
Although the impact of GCT on project effectiveness is positive, it is not statistically
significant. Hence, our results support hypotheses H1a and H1b, but do not support H1c.
On the other hand, GCTs have a significant impact on the efficiency component of
information work. GCTs support real-time collaboration among project teams, and provide a
communications infrastructure for project teams to collaborate across multiple locations. By
enabling concurrent work among project teams, GCTs improve project efficiency by
reducing the time searching/waiting for information and eliminating non-value added tasks.
Hence, our results support hypothesis H2c, but do not support hypotheses H2a and H2b.
Our results indicate that IT human capital has a positive impact on improvements in
project effectiveness and efficiency. Although, the impact of IT human capital on efficiency
is positive, it is statistically significant at a p-value < 0.10. These results suggest that
managerial IT expertise plays an important role in improving business process capabilities, in
terms of their effectiveness and efficiency. Hence, our results support hypothesis H3.
Our results suggest that a firm’s propensity to use IT plays an important role in terms
of its impact on organizational capabilities. Firms which are more likely to adopt emerging
technologies are likely to realize improvements in their process effectiveness, and to a lesser
extent, on process efficiency.
21
5.2 Impact of IT on Project Performance
Table 4 shows the impact of IT and IT-enabled capabilities on the four project
performance measures: change in cycle time, cost, quality, and on-time completion rate.
Odd-numbered columns represent regressions where we treat the performance variable as
dependent on the effectiveness and efficiency capabilities and the firm controls, which
include project autonomy, process maturity, firm size, and the number of concurrent projects
managed. Hence, the regression model is specified in equation (1) as:
Project performance = f (Effectiveness, Efficiency, Firm controls) (1)
Equation (1) is consistent with our research framework which suggests that the
impact of IT on project performance will be completely mediated through organizational
dynamic capabilities. We observe that project effectiveness capabilities and process maturity
dominate other variables in terms of their ability to explain the variation in project outcomes.
Effectiveness has a significant impact on all four project outcomes while efficiency has a
significant impact all outcomes except on-time completion rates. Hence, our data support
hypothesis H4a and H4b.
To validate whether our model provides a satisfactory explanation of the relationship
between IT and project performance, we test the mediation model against a competing model
which included direct paths from the IT variables to the four project performance variables.
Hence, each direct model has three paths more than its corresponding mediation model,
which represents the direct links between the three IT variables and the performance variable.
Therefore, our regression model is specified in equation (2) as
Project performance = f (Effectiveness, Efficiency, ECT, GCT, CCT, Human IT capital,
Firm controls) (2)
22
These regression results are provided in the even numbered columns of Table 4. To
compare the two models, we compared the R2 values which represent the percentage of
variance explained by the predictor variables. Since the models are nested, we can examine
whether the mediation model provides an adequate explanation of the variance in project
outcomes, through an F-test (Subramani, 2004; Venkatraman, 1988). We performed an F-
test on the difference in R2 and report these F-statistics at the bottom of Table 4.
We observe that differences in R2 values are very small for each of the four pairs
(direct and mediation) of regression models. We note that the F-test is statistically
significant for the ∆(Cycle Time) and ∆(Quality) models, while it is not significant for the
∆(Cost) and ∆(On-time Completion Rate) models. Hence, our findings suggest that dynamic
capabilities completely mediate the impact of technological IT resources (ECT, GCT, CCT)
on project cost and completion rates, while they partially mediate IT impact on project cycle
times and quality. The results imply that dynamic capabilities may mediate the impact of IT
on different types of project outcomes in different ways, and may serve only as partial
mediators in some cases.
A closer look at Table 4 suggests that higher levels of ECT usage is associated with
improvements in project on-time completion rates, which implies that usage of enterprise
systems and project management software have a tangible impact on project timeliness. Our
results also indicate that GCT usage is associated with improvements in project cycle time,
which implies that collaboration technologies are useful in coordinating project schedules
and resources. We observe that the IT human capital and process maturity constructs are
associated with improvements in all project performance measures, which is consistent with
findings from prior IS research (Mendelson, 2000; Harter and Slaughter, 2003).
23
5.3 Impact of IT on Firm Performance
A unique characteristic of this study is our ability to go beyond project performance
measures as reported by the survey respondents. We also examined objective financial
measures of the public firms to which respondents belong. Financial data for a subset of 147
companies was collected from the Standard & Poors Compustat database (S&P).
We regressed the firm-level financial measures against dependent variables such as
IT usage, the dynamic capabilities of effectiveness and efficiency, and organizational and
process variables. Our regression results are shown in Table 5. In a manner similar to our
approach described in the previous section, the odd-numbered columns denote results where
we regressed the relevant financial performance measure in year t (FPt) against the dynamic
capabilities, firm control variables, and past firm performance in year t-1 represented by
“Fin. Perft-1”.4 The mediation model is specified as,
FPt = f (Effectiveness, Efficiency, Firm controls, Fin. Perft-1) (3)
The even-numbered columns represent regressions of FPt against the dynamic
capabilities as well as the IT and control variables. That is,
FPt = f (Effectiveness, Efficiency, ECT, GCT, CCT, IT Human Capital, Firm controls,
Fin. Perft-1) (4)
The results indicate that effectiveness has a significant impact on firms’ gross margins
while efficiency has a significant impact on firms’ ROE. These results indicate that quality of
information work, measured in terms of a firm’s ability to improve its consistency, leverage,
alignment and relevance, is associated with improvements in gross margins as firms are able
to do a better job of reducing the cost of sales and improve price premiums by making their
4 Our approach is similar to the one used by Zhu and Kraemer (2002) to control for the effect of prior performance.
24
product/services more relevant to customer needs. In a similar vein, we note that efficiency
is associated with improvements in ROE as it enables firms to leverage their shareholders
investments into valuable assets which generate profits.
To compare the two models, we once again compare the R2 values of the mediation
and direct models. Clearly, the F-test results in Table 5 show that the incremental direct
impact of IT on firm performance is not statistically significant. These results confirm that
the impact of IT on firm performance is mediated through effectiveness and efficiency, but
through firm-level data. Hence, our results support hypothesis H5 and partially support H6,
since dynamic capabilities partially mediate the impact of IT on project outcomes.
6.0 DISCUSSION AND CONCLUSIONS
Improving information work (IW) productivity has been a major challenge due to the
intangibility of organizational outcomes. As a first step in gaining a more nuanced
understanding of the role and business value of IT, we focused on information work in a
project context. Our research objectives were to understand (a) how different types of IT
resources impact dynamic capabilities within organizations, and (b) how to gauge and
measure the value of IT in an information work setting. We have extended prior research by
(a) improving our understanding of the differential impact of alternate types of IT resources
as well as their usage, (b) incorporating the role of software when conceptualizing the IT
resource, which has been missing from most IT productivity research, (c) operationalized two
salient dimensions of dynamic capabilities, effectiveness and efficiency, which explain a
significant portion of the variation in firm performance, and (d) provided empirical support to
test the dynamic capabilities framework with data collected from an information work setting
25
where technologies are used to manage projects, programs, and people. Furthermore, our
model measures a key construct, the level of IT usage, and links it to dynamic capabilities
which mediate the impact on firm performance (Devaraj and Kohli, 2003).
Our observed results using perceptual measures of project outcomes were further
enhanced through the collection of financial data for a subset of 147 public firms. Indeed,
our results suggest that prior studies that have attempted to measure the productivity of
information work may be inadequate since they mainly capture the efficiency dimension of
information work. Our results indicate that quality of firm outputs (effectiveness) may be
even more important in terms of its impact compared to the quantity of outputs (efficiency).
In this manner, our research framework supports the arguments for extending IT innovation
research as proposed by Fichman (2004).
Our study has several limitations. First, it does not account for the use of information
technologies by individual contributors or non-managerial personnel who may have different
IT usage patterns from project/program managers. Second, while our study contains a large
number of data points, it can be enhanced further by including respondents in non-US
organizations to account for country- or culture-specific differences in their IT usage. Third,
our survey findings must be validated with more field studies and additional data collected in
industry-specific settings which will allow us to examine the impact of industry
characteristics and IT usage on the development of specific dynamic capabilities in different
environments. Future research should also include data on project complexity, team diversity
and uncertainty which are important project management variables.
26
The overall results suggest that IT needs to be implemented with the intent of
maximizing project and financial outcomes, in a manner that fundamentally improves the
dynamic capabilities of the firm, which, in turn, will improve project and firm-level
performance. Organizations will not realize significant performance improvements if IT is
used to only improve quantity of outputs and inputs, ignoring the effectiveness or quality of
the outcomes. Effectiveness should be as much of a focus as efficiency, which has hitherto
driven IT investment decisions. Our study represents an early effort to study the impact of IT
specifically focused on project and information work (Nambisan, 2003). Our findings
amplify the need for firms to strengthen their organizational dynamic capabilities after
making investments in information technology (Dorgan and Dowdy, 2004). While we have
experienced many anecdotally-driven discussions regarding whether IT matters or not, our
data shows that IT matters to the extent that it can extend organizational dynamic
capabilities.
REFERENCES
Amit, R. and P.J.H. Schoemaker, “Strategic assets and organizational rent,” Strategic Management Journal 14(1), 1993, 33-46.
Armstrong, C.P., and V. Sambamurthy, “Information technology assimilation in firms: the influence of senior leadership and IT infrastructure,” Information Systems Research 10(4), 1999, 304-327.
Barney, J.B., “Firm resources and sustained competitive advantage,” Journal of Management 17(1), 1991, 99-120.
Barua, A., C. Kriebel, and T. Mukhopadhyay, “Information technology and business value: An analytic and empirical investigation,” Information Systems Research 6(1), 1995, 3-23.
Barua, A., P. Konana, A. Whinston, and F. Yin, “An empirical investigation of net-enabled business value,” MIS Quarterly 28(4), 2004, 585-620.
Bharadwaj, A.S., “A resource-based perspective on information technology capability and firm performance: An empirical investigation,” MIS Quarterly 24(1), 2000, 169-196.
Boynton, A. C., R.W. Zmud, and G.C. Jacobs, “The influence of IT management practice on IT use in large organizations,” MIS Quarterly 18(2), 1994, 299-318.
27
Brynjolfsson, E., and L. Hitt, “Paradox lost? Firm-level evidence on the returns to information systems spending,” Management Science 42(4), 1996, 541-558.
Brynjolfsson, E., and L. Hitt, “Beyond the productivity paradox,” Communications of the ACM 41(8), 1998, 49-55.
Brynjolfsson, E., and L. Hitt, “Beyond Computation: Information technology, organizational transformation, and business performance,” Journal of Economic Perspectives 14(4), 2000, 23-48.
Carte, T., and L. Chidambaram, “A capabilities-based theory of technology deployment in diverse teams: Leapfrogging the pitfalls of diversity and leveraging its potential with collaborative technology,” Journal of the Association for Information Systems 5(11-12), 2004, 448-471.
Chatterjee, D., R. Grewal, and V. Sambamurthy, “Shaping up for e-commerce: Institutional enablers of the organizational assimilation of web technologies,” MIS Quarterly 26(2), 2002, pp. 65-89.
Clark, K., and T. Fujimoto, Product Development Performance: Strategy, Organization and Management in the World Auto Industries, 1991, Harvard Business School Press, Boston, MA.
Conner, K.R., and C.K. Prahalad, “A resource based view of the firm: Knowledge versus opportunism,” Organization Science 7(5), 1996, 477-501.
Devaraj, S., and R. Kohli, “Performance impacts of information technology: Is actual usage the missing link?” Management Science 49(3), 2003, pp. 273-289.
Dorgan, S.R., and J.J. Dowdy, “When IT lifts productivity,” The McKinsey Quarterly (4), 2004.
Eisenhardt, K., “Making fast strategic decisions in high-velocity environments,” Academy of Management Journal 32(3), 1989, 543-576.
Eisenhardt, K., and S.L. Brown, “Patching: Restitching business portfolios in dynamic markets,” Harvard Business Review 77(3), 1999, 72-82.
Eisenhardt, K., and J. Martin, “Dynamic capabilities: What are they?” Strategic Management Journal 21, 2000, 1105-1122.
Fichman, R.G., “Going beyond the dominant paradigm for information technology innovation research: Emerging concepts and methods,” Journal of the Association for Information Systems 5(8), 2004, 314-355.
Fornell, C., and D. Larcker, “Evaluating structural equation models with unobservable variables and measurement errors,” Journal of Marketing Research 18(1), 1981, pp. 39-50.
Gefen, D., D.W. Straub, M.C. Boudreau, “Structural equation modeling and regression,” Communications of the Association for Information Systems 4(7), 2000, 1-70.
Hansen, M.T., “The search-transfer problem: The role of weak ties in sharing knowledge across organizational subunits,” Administrative Science Quarterly (March) 44, 1999, 82-111
Hargadon, A, and R.I. Sutton, “Technology brokering and innovation in a product development firm,” Administrative Science Quarterly 42(4), 1997, 716-749.
Harter, D.E., M.S. Krishnan, S. Slaughter, “Effects of process maturity on quality, cycle time, and effort in software product development,” Management Science 46(4), 2000, 451-466.
Harter, D.E., S. Slaughter. 2003. Quality improvement and infrastructure activity costs in software development: A longitudinal analysis. Management Science. 49(6), 84-800.
Hitt, L.M., D.J. Wu, X. Zhou, “Investment in enterprise resource planning: Business impact and productivity measures,” Journal of Management Information Systems 19(1), 2002, 71-98.
28
Iansiti, M., and K.B. Clark, “Integration and Dynamic Capability: Evidence from Product Development in Automobiles and Mainframe Computers,” Industrial and Corporate Change 7(3), 1994, pp. 557-605.
Johnston, J., Econometric Methods, 1984, Third Edition, McGraw-Hill, Inc., New York, NY.
Kohli, R., and S. Devaraj, “Measuring information technology payoff: A meta-analysis of structural variables in firm-level empirical research,” Information Systems Research 14(2), 2003, 127-145
Kumar, K., and H.G. van Dissel, “Sustainable collaboration: Managing conflict and cooperation in inter-organizational Systems,” MIS Quarterly 20(3), 1996, 279-300.
Magretta, J., “The power of virtual integration: An interview with Dell Computer’s Michael Dell,” Harvard Business Review 76(2), 1998, 72-84.
Mata, F.J., W.L. Fuerst, and J. Barney, “Information technology and sustained competitive advantage: A resource-based analysis,” MIS Quarterly 19(4), 1995, 487-505.
McGrath, M., and M. Iansiti, “Envisioning IT-enabled innovation” Insight Magazine, 1998, 2-10.
Melville, N, K.L. Kraemer, and V. Gurbaxani, “Information technology and organizational performance: An integrative model of IT business value,” MIS Quarterly 28(2), 2004, 283-322.
Mendelson, H., “Organizational architecture and success in the information technology industry,” Management Science 46(4), 2000, 513-529
Mukhopadhyay, T., S. Rajiv and K. Srinivasan, “Information technology impact on process output and quality,” Management Science 43(12), 1997, pp. 1645-1659.
Nambisan, S., “Designing virtual customer environments for new product development: Toward a theory,” Academy of Management Review 27(3), 2002, 392-413.
Nambisan, S., “Information systems as a reference discipline for new product development,” MIS Quarterly 27(1), 2003, 1-18.
Nunnally, J.C., and I.H. Bernstein, Psychometric Theory, 1994, McGraw Hill, New York, NY.
Palepu, K.G., V.L. Bernard, and P.M. Healy, Introduction to Business Analysis and Valuation. 1997, South-Western Publishing, Cincinnati, OH.
Pavlou, P.A., and O.A. El Sawy, “Understanding the ‘black box’ of dynamic capabilities: A missing link to the strategic role of IT in turbulent environments,” Working Paper, 2004, University of California at Riverside, CA.
Peteraf, M.A., and J.B. Barney, “Unraveling the resource-based tangle,” Managerial and Decision Economics 24(4), 2003, 309-323.
Phillips, L., and R. Bagozzi, “Assessing measurement error in key informant reports: A methodological note on organizational analysis in marketing,” Journal of Marketing Research (18), 1986, pp. 395-415.
Podsakoff, P.M., D.W. Organ. 1986. Self-reports in organizational research: Problems and prospects. Journal of Management. 12(4), pp. 531-544.
Peters, T., “The Wow Project”, Business 2.0, May 1999, Issue 24, Page 116.
Ray, G., J.B. Barney, and W.A. Muhanna, “Capabilities, business processes, and competitive advantage: Choosing the dependent variable in empirical tests of the resource-based view,” Strategic Management Journal 25, 2004, 23-37.
29
Ross, J.W., C.M. Beath, and D.L. Goodhue, “Develop long-term competitiveness through IT assets,” Sloan Management Review 38(1), 1996, 31-42.
Sambamurthy, V., “Business Strategy in Hyper-competitive Environments: Rethinking the Logic of IT Differentiation,” in Framing the Domains of IT Management: Projecting the Future through the Past, Zmud, R.W., (ed.), 2000, Pinnaflex, Cincinnati, OH, 245-261.
Santhanam, R., and E. Hartono, “Issues in linking information technology capability to firm performance,” MIS Quarterly 27(1) 2003, 125-153.
Soh, C., and L.M. Markus, “How IT creates business value: A process theory synthesis,” in Proceedings of the 16th International Conference on Information Systems, J.I. DeGross, G. Ariav, C. Beath, R. Hoyer, and C. Kemerer (eds.), Amsterdam, the Netherlands, 1995, 29-41.
Subramani, M., “How do suppliers benefit from information technology use in supply chain relationships?” MIS Quarterly 28(1), March 2004, 45-73.
Tallon, P.P., and K.L. Kraemer, “The impact of IT capabilities on firm performance: Perspectives on the mediating effects of strategic alignment,” Working paper, 2004.
Tallon, P.P., K.L. Kraemer, V. Gurbaxani, and J. Mooney, “A multi-dimensional assessment of the contribution of information technology to firm performance,” CRITO Working Paper #ITR-105, 2004, University of California at Irvine, Irvine, CA.
Teece, D.J., G. Pisano, A. Shuen, “Dynamic Capabilities and Strategic Management,” Strategic Management Journal 18(7), August 1997, 509-533.
Thompson, J.D., Organizations in Action, 1967, McGraw Hill, New York, NY.
Venkatraman, N., “The concept of fit in strategy research: Toward verbal and statistical correspondence,” Academy of Management Review 14(3), 1989, 423-444.
Wade, M., and J. Hulland, “The resource-based view and information systems research: Review, extension, and suggestions for future research,” MIS Quarterly 28(1), 107-142.
Wernerfelt, B., 1984, “A resource-based view of the firm” Strategic Management Journal 5(2), pp. 171-180.
Werts, C., R. Linn, and K. Joreskog, “Interclass reliability estimates: Testing structural assumptions,” Educational and Psychological Measurement 34(1), 1974, pp. 25-33.
Weill, P. "The relationship between investment in information technology and firm performance: A study of the valve manufacturing sector," Information Systems Research 3(4) 1992, pp 307- 333.
Wheeler, B.C., “NEBIC: A dynamic capabilities theory for assessing Net-enablement” Information Systems Research 13(2), 2002, pp. 125-146.
Yu, Seunghee, T. Mukhopadhyay, and S. Slaughter, “Organizational capabilities and the assimilation of electronic procurement in service industries,” Proceedings of the Twenty-Fourth international Conference on Information Systems, 2003, 975-981.
Zeithaml, V., and M.J. Bitner, Services Marketing, 1996, McGraw-Hill, New York, NY.
Zhu, K., and K.L. Kraemer, 2002, “e-Commerce metrics for net-enhanced organizations: Assessing the value of e-Commerce to firm performance in the manufacturing Sector,” Information Systems Research 13(3), 275-295.
Zollo, M., and S.G. Winter, “Deliberate learning and the evolution of dynamic capabilities,” Organization Science 13(3), 2002, 339-351.
30
Figure 1: Conceptual Research Model
Table 1: Survey Respondent Firms’ Profiles
Firm Revenues Number of Respondents Percent (%) $10 million to $49.9 million 90 14.4 $50 million to $99.9 million 62 9.92 $100 million to $499.9 million 74 11.84 $500 million to $999.9 million 48 7.68 $1 billion to $9.9 billion 103 16.48 $10 billion or more 103 16.48 Not for profit 64 10.24 Don’t Know or not at liberty to disclose 81 12.96 Industry Communications 51 8.16 Energy 18 2.88 Financial Services 77 12.32 Healthcare 67 10.72 Public Sector 104 16.64 Manufacturing 121 19.36 Retail/Wholesale 54 8.64 Travel and Transportation 25 4.00 Other 108 17.28 Project Team Size 1 - 10 people 354 57% 11 - 20 people 127 20% 21 - 50 people 82 13% 51 - 100 people 30 5% > 100 people 32 5% TOTAL 625
Firm Financial Measures
• Gross Margin• Return on Assets• Return on Equity
• Effectiveness• Efficiency
• Enterprise Computing
• Group CollaborationTechnologies (GCT)
• Core Communication Technologies (CCT)
Organizational Capabilities
Information TechnologiesProject Outcomes
•• Efficiency
• Enterprise Computing Technologies (ECT)
•Technologies
• Core Communication Technologies (CCT)
Change in
Organizational Controls
• Project Cycle Time• Project Cost• Project Quality• On-time Completion Rate
Organizational Performance
• Team Size• Co-location• Propensity to Use IT• Project Duration
Project Controls
•• Process Maturity•• Number of Concurrent
Projects• Firm Size
• Autonomy
Human ITCapital
Firm Financial Measures
• Gross Margin• Return on Assets• Return on Equity
• Effectiveness• Efficiency
• Enterprise Computing
• Group CollaborationTechnologies (GCT)
• Core Communication Technologies (CCT)
Organizational Capabilities
Information TechnologiesProject Outcomes
•• Efficiency
• Enterprise Computing Technologies (ECT)
•Technologies
• Core Communication Technologies (CCT)
Change in
Organizational Controls
• Project Cycle Time• Project Cost• Project Quality• On-time Completion Rate
Organizational Performance
• Team Size• Co-location• Propensity to Use IT• Project Duration
Project Controls
•• Process Maturity•• Number of Concurrent
Projects• Firm Size
• Autonomy
Human ITCapital
31
Table 2: Confirmatory Factor Analyses of the Model Variables
Construct Indictor Variable
Standardized Loading
t-statistic Composite Reliability
Average Variance Extracted
(AVE) Knowledge Management Software
0.789 23.29
Document Management Software
0.660 18.20
Enterprise Application Software
0.681 18.97
CRM Software 0.762 22.11 Content Management Systems
0.788 23.25
Project Management Software
0.717 20.33
ENTERPRISE COMPUTING TECHNOLOGIES (ECT)
Business Intelligence Software
0.681 18.95
0.89 0.73
Video Conferencing and Voice over IP
0.693 19.00
Groupware and Online Teamspaces
0.722 20.09
Mobile Computing 0.718 19.92
GROUP COLLABORATION TECHNOLOGIES (GCT)
Instant Messaging Software
0.624 16.61
0.78 0.70
Internet Search Engines and Web Portals
0.657 15.09
Email 0.503 11.37
CORE COMMUNICATION TECHNOLOGIES (CCT) Mobile Communication
Technologies 0.657 15.10
0.64 0.61
EFFECT-1 0.745 21.69 EFFECT-2 0.740 21.47 EFFECT-3 0.719 20.63 EFFECT-4 0.725 20.90 EFFECT-5 0.689 19.50 EFFECT-6 0.760 22.29 EFFECT-7 0.747 21.78 EFFECT-8 0.754 22.06 EFFECT-9 0.736 21.33 EFFECT-10 0.768 22.65 EFFECT-11 0.692 19.58
EFFECTIVENESS
EFFECT-12 0.703 20.02
0.94 0.73
(continued on the next page)
32
EFFICIENCY-1 0.737 19.72 EFFICIENCY-2 0.800 21.82
EFFICIENCY
EFFICIENCY-3 0.750 20.13
0.81 0.76
ORGIQ-1 0.784 22.98 ORGIQ-2 0.782 22.91 ORGIQ-3 0.706 19.82 ORGIQ-4 0.662 18.20 ORGIQ-5 0.655 17.96 ORGIQ-6 0.592 15.81
HUMAN IT CAPITAL
ORGIQ-7 0.718 20.30
0.88 0.70
PROCM-1 0.634 17.04 PROCM-2 0.741 20.96 PROCM-3 0.781 22.57 PROCM-4 0.802 23.48
PROCESS MATURITY
PROCM-5 0.717 20.04
0.86 0.74
Table 3: Impact of IT on Organizational Dynamic Capabilities Independent Variable Dependent Variables Effectiveness Efficiency
Intercept 0.14 (0.31)
0.15 (0.33)
ECT 0.30*** (0.0001)
0.03 (0.31)
GCT 0.05 (0.19)
0.17*** (0.0001)
CCT 0.23*** (0.0001)
0.05 (0.17)
Human IT Capital
0.25*** (0.0001)
0.08* (0.06)
Project Controls Project Duration -0.004
(0.89) 0.03
(0.44) Team Size -0.01
(0.67) 0.03
(0.37) Team Co-location 0.01
(0.61) -0.03 (0.34)
Propensity to Use IT 0.07** (0.05)
0.07** (0.05)
Adjusted R2 0.28 0.07 *** indicates p < 0.01, ** indicates p < 0.05, and * indicates p < 0.10.
Table 4. Impact of IT and Dynamic Capabilities on Project Outcomes Independent Variable
∆(CYCLE TIME ) ∆(COST) ∆(QUALITY) ∆(ON-TIME COMPLETION RATE)
(1) (2) (3) (4) (5) (6) (7) (8) Intercept 3.37***
(0.000) 3.39*** (0.000)
3.57*** (0.000)
3.60*** (0.000)
3.69*** (0.000)
3.67*** (0.000)
3.57*** (0.000)
3.59*** (0.000)
Effectiveness 0.34*** (0.000)
0.32*** (0.000)
0.33*** (0.000)
0.33*** (0.000)
0.37*** (0.000)
0.34*** (0.000)
0.33*** (0.000)
0.30*** (0.000)
Efficiency 0.15*** (0.000)
0.13*** (0.000)
0.13*** (0.000)
0.12*** (0.000)
0.10*** (0.001)
0.09*** (0.004)
0.02 (0.452)
0.01 (0.707)
ECT _ 0.05 (0.129)
_ 0.01 (0.708)
_ 0.02 (0.570)
_ 0.07** (0.043)
GCT _ 0.09*** (0.006)
_ 0.06 (0.063)
_ 0.02 (0.545)
_ 0.05 (0.136)
CCT _ 0.05 (0.108)
_ -0.02 (0.505)
_ 0.09*** (0.006)
_ 0.03 (0.355)
Human IT Capital 0.05 (0.172)
0.02 (0.496)
0.14*** (0.000)
0.13*** (0.000)
0.12*** (0.001)
0.10*** (0.003)
0.18*** (0.000)
0.16*** (0.000)
Autonomy 0.01 (0.789)
0.00 (0.907)
-0.01 (0.695)
-0.02 (0.597)
0.00 (0.940)
-0.00 (0.955)
0.02 (0.547)
0.01 (0.606)
Process Maturity 0.11*** (0.001)
0.09** (0.012)
0.14*** (0.000)
0.13*** (0.001)
0.16*** (0.000)
0.15*** (0.000)
0.22*** (0.000)
0.20*** (0.000)
Firm Size (Employees) 0.00 (0.895)
0.00 (0.997)
-0.00 (0.994)
-0.00 (0.816)
-0.00 (0.825)
-0.00 (0.992)
-0.00 (0.854)
-0.01 (0.740)
Number of Concurrent Projects Managed
0.02 (0.430)
0.02 (0.427)
-0.05 (0.055)
-0.05 (0.052)
-0.00 (0.986)
-0.00 (0.990)
0.00 (0.985)
0.00 (0.946)
Adjusted R2 0.245 0.254 0.274 0.276 0.315 0.320 0.317 0.320
∆(Adj. R2) 0.009 0.001 0.005 0.003 F-statistic# 3.59**
(0.014) 1.39
(0.24) 2.64** (0.05)
2.03 (0.11)
*** indicates p < 0.01 and ** indicates p < 0.05. p-values are shown in parentheses. # The F-test was computed based on the ∆(R2) values of the complete mediation and full models with df1 = 3 and df2 = 617.
34
Table 5. Impact of IT and Dynamic Capabilities on Firm Financial Performance
Independent Variable GROSS MARGIN ROA ROE
(1) (2) (3) (4) (5) (6) Intercept -0.057
(0.057) -0.054 (0.074)
-0.018 (0.58)
-0.012 (0.71)
-0.091 (0.67)
-0.058 (0.79)
Effectiveness 0.013** (0.03)
0.013* (0.08)
0.007 (0.33)
0.006 (0.47)
-0.002 (0.95)
-0.004 (0.93)
Efficiency -0.007 (0.20)
-0.008 (0.14)
0.004 (0.46)
0.005 (0.37)
0.068* (0.09)
0.083** (0.04)
ECT - 0.002 (0.69)
- 0.006 (0.37)
- 0.033 (0.45)
GCT - 0.007 (0.20)
- -0.006 (0.35)
- -0.082* (0.05)
CCT - -0.002 (0.72)
- -0.002 (0.75)
- -0.010 (0.82)
Propensity to use IT
0.001 (0.89)
0.000 (0.98)
0.007 (0.22)
0.009 (0.14)
0.059 (0.13)
0.079** (0.05)
Human IT Capital - -0.006 (0.37)
- -0.007 (0.28)
- -0.029 (0.48)
Process Maturity -0.006 (0.36)
-0.008 (0.15)
0.001 (0.84)
0.000 (0.99)
0.008 (0.86)
-0.003 (0.94)
Autonomy 0.000 (0.95)
-0.001 (0.88)
0.004 (0.51)
0.004 (0.49)
0.025 (0.49)
0.031 (0.40)
Log(Assets) 0.006*** (0.01)
0.006** (0.02)
0.004 (0.11)
0.004 (0.13)
0.015 (0.38)
0.017 (0.35)
Fin. Perft-1 0.891*** (0.000)
0.881*** (0.000)
0.343*** (0.000)
0.346*** (0.000)
0.773*** (0.000)
0.767*** (0.000)
Adjusted R2 0.780 0.784 0.235 0.241 0.324 0.333
∆(Adj. R2) 0.004 0.006 0.009
F-statistic 0.685 0.664 1.657
p-values are shown in parentheses. *** indicates p < 0.01 and ** indicates p < 0.05.
APPENDIX: SURVEY QUESTIONNAIRE6
The objective of this research survey is to develop an understanding of the business processes, management practices and information technologies (IT) in use in your firm. Your responses will help the research team understand the impact of IT and organizational processes on information work productivity. The first section asks for specific information about your role, work characteristics and your firm. In the second section, data is collected about the business impact you have observed. In the third section, data about project performance outcomes are measured.
1. Select the generic role below that best describes your role within your current company: 1=Individual Contributor, 2=Project Manager, 3=Program Manger, 4=Product Manager, 5=Business
Manager 2. Is your company…? (Please select only one response.)
0=Publicly traded, 1=Privately held. Provide your company name. 3. FIRM SIZE: How many regular full-time employees are in your company? (Please select only one)
1 = < 100 employees (TERMINATE); 2 = 100 to 249; 3 = 250 to 499; 4 = 500 to 999; 5 = 1,000 to 4,999; 6 = 5,000 to 9,999; 7 = 10,000 to 24,999; 8 = 25,000 or more; 9 = Not Sure
4. NUMBER OF CONCURRENT PROJECTS MANAGED: What is the typical number of projects you
manage concurrently (at any one time)? 1 = 1 Program/project; 2 = 2 - 3 Programs/projects; 3 = 4 - 5 Programs/projects 4 = 6 - 10 Programs/projects; 5=More than 10 Programs/projects
5. PROJECT DURATION: How long does your typical program or project last?)
1 = Less than 1 Year; 2 = 1 - 2 Years; 3 = 3-5 Years; 4 = 6 - 10 Years; 5 = > 10 Years 6. TEAM SIZE: How many people are typically involved in your programs or projects?
1 = 1 - 10 People; 2 = 11 - 20 People; 3 = 21 - 50 People; 4 = 51 - 100; 5 = > 100 People 7. CO-LOCATION: Describe the extent of co-location of your team. Most of the program/project team is in
the: 1 = Same Building; 2 = Same City; 3 = Same Time Zone; 4 = Different Time Zones; 5 = Different Country and Time Zone
8. PROPENSITY TO USE IT: How would you describe your organization's use of IT?
1 = Late Adopter; 2 = Conservative Adopter; 3 = Mainstream User; 4 = Early User; 5 = Leading Edge User
9. AUTONOMY: Project/program targets/goals are primarily set by (choose one):
1= Business Unit Head; 2 = Program Sponsor; 3 = Project Manager; 4 = Project Team 10. HUMAN IT CAPITAL
Please comment on the following program/project related statements as they relate to your organization. (Please provide a rating for each) 1 = Strongly Disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly Agree
i. We benchmark our performance against best-practices from other industries ii. Project personnel are well-trained in the use of work methods, and practices iii. Information technology training is a high priority
6 The question headings (in bold) were not shown on the survey questionnaire and questions were arranged in random order. They are shown here to improve the clarity of the model variables and constructs to the reader.
36
iv. New or updated methods, practices and Information systems are used to keep project teams in touch with customers
v. Information systems are used to support communications between project teams vi. Senior management is involved in championing my programs/projects vii. Senior management provides the necessary financial and human resources to ensure project success
[USE THE FOLLOWING SCALE FOR question 11]
1 = Never; 2 = Rarely; 3 = Sometimes; 4 = Frequently; 5 = All the Time
11. PROCESS MATURITY Please comment on the following program/project related statements as they relate to your organization. i. We use cross-functional project teams ii. Projects are structured to enable knowledge transfer among project teams iii. We have well-defined processes for program/project execution iv. Quantitative targets are used to manage program/ projects v. Metrics are automatically collected for process management vi. Customer feedback is collected, tracked, and used for continuous process improvement
12. Please rate the following IT applications on their degree of usage in your current job/role: 1 = No Use; 2 = Limited Use; 3 = Medium Use; 4 = High Use; 5 = Heavy Use
CORE COMMUNICATION TECHNOLOGIES (CCT) i. Email ii. Internet Search Engines and Web Portals iii. Mobile Communication (WLAN, Voice, Cell phones, Wireless laptops) GROUP COLLABORATION TECHNOLOGIES (GCT) i. Instant Messaging Software ii. Mobile Computing (e.g. Tablets, PDA, Blackberry hand-helds) iii. Video-Conferencing Technologies (e.g., Webex) iv. Groupware and Online Teamspaces (e.g. Sharepoint, LiveLink, eRoom)
ENTERPRISE COMPUTING TECHNOLOGIES (ECT) i. Enterprise Application Software (ERP, Supply chain management software) ii. Knowledge Management Software iii. Customer Relationship Management Software (CRM, Collaboration Tools) iv. Project Management Software v. Business Intelligence (e.g. SAS, Hyperion, Cognos) vi. Content Management Systems/Project Portals vii. Document Management Solutions/Systems
13. Please evaluate the impact of IT applications for each statement listed below. Please respond based on
what you would consider your typical program/project over the past 12 months. 1 = Significant Negative Effect; 2 = Moderate Negative Effect; 3 = Mild Negative Effect; 4 = No Effect; 5 = Mild Positive Effect; 6 = Moderate Positive Effect; 7 = Significant Positive Effect
EFFICIENCY i. Time spent searching and waiting for information ii. Program/project task durations and enabling concurrent work iii. Rework and non-value added tasks
37
EFFECTIVENESS i. Making current program/project information (schedule, goals) visible throughout the team ii. Ease of tracking and monitoring programs/projects throughout the company iii. Identifying and mitigating program/project and business risks iv. Identification and facilitation (management) of qualified suppliers and partners v. Ability to lower procurement cost of goods vi. Re-use of information and corporate knowledge assets across the enterprise vii. Making the program/project fit the firm's strategic direction and project/business portfolio viii. Identification and closure of functional gaps within and across program/project teams ix. Alignment of virtual and distributed program/project teams x. Identifying and disseminating customer needs and requirements to the program/project teams xi. Continual involvement of customers in programs/projects xii. Creating new markets, product enhancements, and business opportunities
14. PROJECT OUTCOMES
Evaluate the change in the following project outcomes in the previous one year (2003 to 2004): 1 = High level of negative benefit; 2 = Some negative benefit; 3 = No positive or negative benefit; 4 = Moderate level of benefit; 5 =High level of benefit
i. Change in project cycle/completion times, defined as the total time elapsed from project initiation to
completion. ii. Change in program or project costs, defined as the total program/project cost incurred from initiation
to completion (including labor, equipment, services) iii. Change in program or project quality, measured as the total number of errors, defects, and rework
associated with the project work * iv. Change in % of programs or projects completed on-time, measured as the deviation between actual
and expected project completion time *