Choice and Change of Measures in Performance Measurement Models

Mary A. MalinaNaval Postgraduate SchoolFrank H. SeltoLeeds School of BusinessUniversity of Colorado at BoulderJune 2004

AbstractThis paper uses management control, resource-based systems-based and contingency-based strategy theories to describe a large U.S. manufacturing companys efforts to improve profitability by designing and using a performance measurement model (PMM). This PMM includes multiple performance measures relevant to its distribution channel for products, repair parts and maintenance services. The PMM is intended to reflect the companys understanding of performance relations among strategic resources, operational capabilities, and desired financial outcomes. The PMM also reflects its intended distribution strategy, the types of performance necessary to achieve that strategy by its distributors, and its desired financial outcomes. Furthermore, the company uses the model to evaluate its North American distributors and intends to use these evaluations as a partial basis for annual and long-term rewards. Thus, the PMM embodies the measurable portion of the firms management control system of its distribution channel.

1. IntroductionImproving performance measurement at key parts of the value chain is one of management accountings major roles. Valid performance measurement allows a firm to effectively describe andimplement strategy, guide employee behavior, assess managerial effectiveness, and provide the basis for rewards. Managers and researchers from diverse disciplines have sought to improve management of the value chain by building and using performance measurement models (PMM). PMM are comprehensive models of the firm as a system, which reflect organizational knowledge of the relations among various value-chain performance measures. Many organizations reportedly have created PMM that model performance relations among key value-chain activities and valued outcomes [e.g., the balanced scorecard of Kaplan and Norton, 1996]. Consulting reports, normative studies, and descriptive theories predict that these comprehensive models lead to superior performance. Magretta [2002] argues that business models are essential to tying insights to financial results. Furthermore, knowledge-based and systems-based theories of the firm hypothesize that superior performance results from systemic management policies, rather than myopic focus on elements of the value chain [e.g., Huff and Jenkins, 2003; Morecroft, 2002; Sanchez and Heene, 1996]. Empirical evidence supporting normative claims or theoretical hypotheses is scant and usually is uncritical self-reports [e.g., Rucci et al., 1998; Barabba et al. 2002].

Systematic management requires a comprehensive management control system (MCS), but not all ofa MCS need be measurable. However, the portion that is feasibly measured should be considered for the PMM; otherwise the organization might lose valuable performance information. The choice ofperformance measures is critical in reflecting the organization as a system. Since an organization isalways adapting to its environment, it must be able to change its performance measures to reflect current conditions. This study describes the determinants of a particular PMM and investigates the relatively unexplored issue of choice and change of a functioning PMM.

1.1 Knowledge-based and systems-based theories and PMMTheories that explain management policies based on strategic resources, capabilities, learning, andsystems offer guidance and predictions for the choice of performance measures. Recent strategicmanagement literature has evolved the concept of a firms product strategy beyond Porters [1985]depiction of managing the value chain to achieve competitive advantage (e.g., through product cost leadership or differentiation). Porters work on the importance of strategic positioning has greatly influenced later work that seeks to explain how firms might use their resources to attain strategic positioning. Research that followed Porter explains how firms reach and maintain the positions of strategic advantage that he described.

Barney [1991] argues that successful firms achieve competitive advantage by acquiring and usingunique resources to build inimitable capabilities that create strategic advantages [see also Amit andSchoemaker, 1993 and Kogut and Kulatilaka, 2001]. Organizational learning theory by Nonaka andTakeuchi [1995] and Senge [1990] predicts that successful firms create strategic advantages by learning dynamically to use their resources effectively. This learning is realized through development and deployment of the firms capabilities, processes, or competencies to use resources [e.g., Prahalad and Hamel, 1990]. Morecroft et al. [2002] hypothesize that successful firms manage strategic resources and capabilities through holistic management systems; that is, creating and maintaining strategic advantages are enhanced by systemic management. Our accounting interpretation of current management theories is that firms create and maintain strategic advantages or positions by efficiently creating, deploying, and using performance-based MCS. Furthermore, the measurable part of the MCS should itself be systemic, in the form of a PMM.

1.2 Prior work on PMMThe DuPont ROI formula is an early and enduring PMM that disaggregates financial performanceinto manageable elements [e.g., Zimmerman, 1997; p. 187]. EVA is a similar, more current and complex approach to identifying the incremental contribution to shareholder wealth and the manageable elements of periodic income [e.g., Adimando et al, 1994]. Rappaports [1999] approach to building shareholder value recognizes incentive effects of over-reliance on periodic financial results and seeks to mitigate disincentives. Because all of these models focus primarily on financial outcomes, they do not qualify as systems models; that is, they do not model the determinants of financial performance even within the boundaries of the firm.\More comprehensive PMM include Otleys [1999] performance management model, Ittner andLarckers [2001] value-based management model, Epstein et al.s [2000] APL model, Kanjis business scorecard [Kanji and Moura e Sa, 2002] and the balanced scorecard (BSC) [Kaplan and Norton, 1996, 2001]. These models describe links among business decisions and outcomes, and serve to guide strategy development, communication, implementation, and feedback at multiple points along the value chain. Because these comprehensive PMM are business models, reflecting inputs and both intermediate and final outputs, they generally include measures of operational, strategic, financial and non-financial performance. These models do represent efforts to use organizational knowledge to model the firm as a system and implement management control.

Four research questions:RQ1: Are measure attributes important considerations for performance measure choice?RQ2: Does the importance of attributes differ according to firm strategy?RQ3: Does the importance of attributes for design and use differ according to firm strategy?RQ4: Does a company trade-off some individual attributes for others?describe the companys distribution PMM and (2) interviews with top managers to understand the nature of the business and the objectives and dynamic structure of the PMM.

2. Performance Measure AttributesManagement control theory argues that MCS, which include PMM, are intended to insure thatemployees (1) know what is expected of them, (2) will exert effort to do what is expected, (3) are capable of doing what is expected, and (4) accomplish what is expected [e.g., Merchant, 1998]. For more than 30 years, researchers have known that firms choose a portfolio of controls and performance measures [e.g., Khandwalla, 1972]. However, subsequent research on firms choices of performance measures often has focused on broad dichotomies of measures, such as financial vs. non-financial measures and mechanistic vs. organic controls. The theory commonly used in that research likewise characterizes the contingencies affecting choices of measures and controls as broad dichotomies (e.g., high vs. low environmental uncertainty; old vs. new technology).1

One particularly popular research stream predicts that firms operating in complex and riskyenvironments rely heavily on qualitative controls and non-financial performance measures and to a much lesser degree (if at all) on quantitative, financial-performance measures. Contingency research on choice of performance measures has yielded mixed results, perhaps because most of the reported studies are based on cross-sectional survey data, which can obscure the idiosyncrasies of firm-level definitions and implementations of performance measurements [e.g., Anderson and Young, 1999; Chenhall, 2003; Luft and Shields, 2002b]. Enough evidence exists, however, to suggest that most firms rely to some degree on financial performance measures and many use both quantitative and qualitative controls. In other words, firms evidently have great flexibility to choose the portfolio of measures and controls (especially when characterized as broad dichotomies) that they expect to work best in their situations. This equivocal result provides some motivation to search for additional theoretical explanation for the choice of performance measures. 2

Recent management control research addresses specific factors that might explain firms choices ofperformance measures to achieve and maintain strategic advantages. Laboratory experiments [e.g., Libby et al, 2002; Lipe and Salterio, 2000; Luft and Shields, 2001, 2002a] and surveys of management control practice [e.g., Ittner et al. 2002; Cavaluzzo and Ittner, 2002; Ittner and Larcker, 1998] have identified attributes of performance measures that are associated with use, usefulness, and performance. When combined with current resource-based and systems-based strategy theories, what emerges is a focus on performance measures attributes that supercede the popular financial vs. non-financial dichotomy. In all cases, the literature cited in the following subsections presumes that the organization seeks to improve performance relative to its strategic goals.

2.1 Measures are diverse and complementaryFirms management controls can benefit from greater diversity of performance measures (i.e.,operational, strategic, financial, and non-financial measures) if operational measures reflect the current drivers of future financial performance and are early in the value chain [Ittner and Larcker, 2002]. Milgrom & Roberts [1995] argue that, if a diverse set of performance measures is a complete and complementary set (or system), using a subset of measures leads to inferior performance. From a similar systems perspective, Warren [2002] argues that successful management policies (e.g., PMM) reflect resource interdependence and complementarity.

2.2 Measures are objective and accurateIjiri [1967] long ago re-established the theoretical importance of (accounting) performance measureaccuracy and objectivity. This topic has not lost relevance.3 More recently, Libby et al. [2002] find that experimental subjects in management-control tasks rely on performance measures that have been verified by third parties, which might create demand for accurate and objective measures. Other studies have found that low-quality measurement is associated with low MCS use or impact [Cavaluzzo and Ittner, 2002; Ittner and Larcker, 1998]. However, it is unclear ex ante if investing in measurements is superior to measuring the wrong things or the right things poorly, or avoiding unreliable measures altogether [e.g., Cavaluzzo and Ittner, 2002; Gates, 1999]. Objectivity (or verifiability) and accuracy (or error free) are theoretically independent concepts yet are often coincident in practice with reference to performance measurement.

2.3 Measures are informativePerformance measures that differentiate managers facing similar, uncontrollable factors are informative. Informative measures can improve evaluations, even if they are not completely controllable by managers [e.g., Antle and Demski, 1988]. In particular, early value-chain measures can be valuable if they are informative about managers leading actions [Ittner and Larcker, 2001] in sufficient time to take corrective control actions.

2.4 Benefits outweigh costs of collectionMonitoring employee behavior through a PMM is a costly activity. Generating, organizing, andreporting performance information consume scarce company resources [Merchant, 1998; Simons, 2000]. As management accounting researchers have known since the early days of the field [e.g., Horngren, 1967], the perceived benefits of using performance measures should outweigh the associated costs.

2.5 Measures reflect system causalitySome academics and consultants have prescribed forms of causal PMM [e.g., Kaplan and Norton,1996; Epstein et al., 2000; Kanji and Moura e Sa, 2002]. Regardless of the sources of business models, causal relations among firms multiple performance measures often are neither specified nor measured well [Ittner et al., 2002]. Quantifying cause-and-effect relations between actions and outcomes at key points in the value chain could help predict future effects of current actions [e.g., Eccles, 1991]. A functioning causal PMM also might free managers to focus more on strategy and evaluation issues [e.g., Kaplan and Norton, 2001] than on information processing. Furthermore, a comprehensive, causal PMM might reduce the cognitive complexity of understanding and using multiple measures of performance [Luft and Shields, 2002a]. Strategy theorists predict significant benefits from building causal models of firms strategic resources and capabilities. Huff [1990] and Huff and Jenkins [2002] describe these models as knowledge-based, cognitive maps, which can connect and organize dispersed organizational knowledge.

2.6 Measures communicate strategyModels such as PMM facilitate communication, learning, and creation of new knowledge and can bethe key tool to building a learning organization [Huff and Jenkins, 2003]. The right performance measures align actions and strategy by reducing managers financial myopia [McKenzie and Schilling, 1998], and effectively communicate strategy [Kaplan and Norton, 2001; Malina and Selto, 2001]. Systemic management understands and exploits knowledge of dynamic interrelations among resources and capabilities. The elements of a PMM are intended to reflect the strategic use of resources and deployment of efficient processes [e.g., Sanchez et al., 2002].

2.7 Measures create incentives for improvementUsing performance measures that capture inherent time delays between certain decisions (e.g.,investing in R&D and employee development) can lead to improved incentives [e.g., Rappaport, 1999; Cloutier and Boehlje, 2002]. Ittner and Larcker [2001] also observe that operational measures, which have good line of sight, can increase the expectancy of rewards based on those measures [e.g., Green,1992].

2.8 Measures improve decision-makingOrganization of measures into distinct categories can affect decision-making, perhaps by reflectingthe structure of knowledge about the firms value chain [Lipe and Salterio, 2000]. Measures with tangible connections to processes being managed also might activate more knowledge and promote better learning and decision-making compared to relying on financial measures alone [Luft and Shields, 2001, 2002a]. Huff and Jenkins [2002] argue that models (e.g., PMM) organize and express the rationale of complex systems, which aid planning and evaluation activities. Furthermore, such models can represent micro- or macro-levels of knowledge of activities, processes, and systems, thus aiding individuals at all levels of the organization. PMM might improve decision-making by identifying actions and impacts that heretofore have been hidden by traditional measurement systems [e.g., Huff and Jenkins, 2002]. Management control and strategy theories identify eight desirable attributes of performance measures. Measures should be:A1 Diverse and complementaryA2 Objective and accurateA3 InformativeA4 More beneficial than costlyA5 Causally relatedA6 Strategic communication devicesA7 Incentives for improvementA8 Supportive of improved decisions

References

Abernethy, M.A. and P. Brownell. 1999. The role of budgets in organizations facing strategic change: an exploratory study. Accounting, Organizations, and Society. 24(3), 189-204.

Adimando, C., R. Butler, S. Malley, S. A. Ravid, R. Shepro, R. H. Vernon, J. Willett and J.

Zimmerman. 1994. Stern Stewart EVA Roundtable, Journal of Applied Corporate Finance, 7(2), 46-70.

Amit, R. and P. Schoemaker. 1993. Strategic assets and organizational rent. Strategic Management Journal, 14: 33-46.

Anderson, S.W. and S.M. Young. 1999. The impact of contextual and process factors on the evaluation of activity-based costing systems. Accounting, Organizations and Society 24: 525-559.

Antle, R. and J. S. Demski. 1988 The controllability principle in responsibility accounting. The Accounting Review 63(4): 700-718.

Barabba, V., C. Huber, F. Cooke, N. Pudar, J. Smith, M. Paich. 2002. A multimethod approach forcreating new business models: The General Motors OnStar project. Interfaces, 32(1): 20-34.

Bouwens, J. and M. Abernethy. 2000. The consequences of customization on management accounting system design. Accounting, Organizations and Society, 25(3), 221-259.

Cavaluzzo, K. and C. Ittner. 2002. Implementing performance measurement innovations: Evidence from government. Georgetown University working paper.

Chenhall, R.H. 2003. Management control systems design within its organizational context: Findings from contingency-based research and directions for the future. Accounting, Organizations and Society, 28: 127-168.

Chenhall, R.H. and D. Morris. 1995. Organic decision and communication processes and management accounting systems in entrepreneurial and conservative business organizations. Omega, International Journal of Management Science, 23(5): 485-497.

Cloutier, LM. and M. Boehlje. 2002. Innovation management under uncertainty: A system dynamics model of R&D investments in biotechnology. In Morecroft et al. (eds). Systems Perspectives on Resources, Capabilities, and Management Processes. Amsterdam: Pergamon.

Dent, J.F. 1990. Strategy, organization and control: Some possibilities for accounting research. Accounting, Organizations and Society, 15, 3-25.

Eccles, R. 1991. The performance measurement manifesto. Harvard Business Review 69: 131-7 Economics 19: 45-48.

Epstein, M. J., P. Kumar and R. A. Westbrook. "The Drivers Of Customer And Corporate Profitability: Modeling, Measuring, And Managing The Causal Relationships," Advances in Management Accounting, 2000, v9(1), 43-72.

Gates, S. 1999. Aligning Strategic Performance Measures and Results. The Conference Board, New York, NY.

Green, T. B. 1992. Performance and Motivation Strategies for Today's Workforce: A Guide to Expectancy Theory Applications. Westport, CT: Greenwood Publishing Group.

Govindarajan, V. and A.K. Gupta. 1985. Linking control systems to business unit strategy: impact on performance. Accounting, Organizations and Society, 51-66.

Guilding, C. 1999. Competitor-focused accounting: an exploratory note. Accounting, Organizations and Society, 24(7), 583-595.

Gupta, A.K. and V. Govindarajan. 1984. Business unit strategy, managerial characteristics, and business unit effectiveness at strategy implementation. Academy of Management Journal, 25-41.

Huff, A. and M. Jenkins (eds.) 2003. Mapping Strategic Knowledge. Thousand Oaks, CA: Sage Publications.

Horngren, C. T. 1967. Cost Accounting: A Managerial Emphasis. 2d ed., Englewood Cliffs, N.J.: Prentice-Hall.

Ijiri, Y. 1967. The Foundations of Accounting Measurement. Houston, TX: Scholars Book Co.

Ittner, C.D. and D.F. Larcker. 1996. Measuring the impact of quality initiatives on firm financial performance. In Advances in the Management of Organizational Quality, Vol. 1. Eds. S. Ghosh and D. Fedor. Greenwich, CT: JAI, 1-37.

Ittner, C.D. and D.F. Larcker. 2001. Assessing empirical research in managerial accounting: A valuebased management perspective. Journal of Accounting and Economics. v32 (1-3,Dec): 349-410.

Ittner, C.D. and D.F. Larcker. 2002. Determinants of performance measure choices in worker incentive plans. Journal of Labor Economics 20(2): S58.

Ittner, C.D. and D.F. Larcker. 1998. Are non-financial measures leading indicators of financial performance? An analysis of customer satisfaction. Journal of Accounting Research 36 (supplement): 1-35.

Ittner, C.D., D.F. Larcker and T. Randall. 2002. Performance implications of strategic performance measurement in financial services firms. Wharton School working paper.

Kanji, G. and P. Moura e Sa. 2002. Kanjis business scorecard. Total Quality Management. 13(1): 13

Kaplan, R.S. and D.P. Norton. 2001. The Strategy-Focused Organization. Boston, MA: Harvard Business School Press.

Kaplan, R.S. and D.P. Norton. 1996. The Balanced Scorecard. Boston, MA: MA: Harvard Business School Press.

Kogut, B. and N. Kulatilaka. 2001. Capabilities as real options. Organization Science, (12)6: 744-58.

Langfield-Smith, K. 1997. Management control systems and strategy: A critical review. Accounting Organizations and Society, 22(2): 207-232.

Libby, T., S. Salaterio, and A. Webb. 2002. The balanced scorecard: The effects of assurance and process accountability on managerial judgment. Wilfred Laurier University working paper.

Lipe, M. and S. Salterio. 2000. The balanced scorecard: Judgmental effects of common and unique performance measures. The Accounting Review 75: 283-98.

Lipe, M. and S. Salterio. 2002. A note on the judgmental effects of the balanced scorecards information organization. Accounting, Organizations and Society 27:531-40.

Luft, J. and M. Shields. 2001. Why does fixation persist? Experimental evidence on the judgment performance effects of expensing intangibles. The Accounting Review. 76(4): 561-87.

Luft, J. and M. Shields. 2002a. Learning the drivers of financial performance: Judgment and decision effects of financial measures, nonfinancial measures, and statistical models. Michigan State University working paper.

Luft, J. and M. Shields. 2002b. Mapping management accounting: Graphics and guidelines for theoryconsistent empirical research. Forthcoming Accounting, Organizations and Society.

Magretta, J. 2002. Why business models matter. Harvard Business Review, 80(5).

Malina, M.A. and F.H. Selto. 2001. Communicating and controlling strategy: An empirical study of the effectiveness of the balanced scorecard. Journal of Management Accounting Research 13: 48 -90.

McKenzie, F.C. and M.D. Shilling. 1998. Avoiding performance measurement traps: Ensuring effective incentive designs and implementation. Compensation and Benefits Review 30(4): 57-65.

Merchant, K. 1998. Modern Management Control Systems: Text and Cases. Upper Saddle River, NJ: Prentice Hall.

Miles, M.B. and A.M. Huberman. 1994. Qualitative data analysis. Thousand Oaks, CA: Sage Publications.

Miles, R.W. and C.C. Snow. 1978. Organizational strategy, structure and process. New York: McGraw Hill.

Miller, D. and P.H. Freisen. 1982. Innovation in conservative and entrepreneurial firms: two models of strategic momentum. Strategic Management Journal, 1-25.

Morecroft, J. 2002. Resource management under dynamic complexity. In Morecroft et al. (eds). Systems Perspectives on Resources, Capabilities, and Management Processes. Amsterdam: Pergamon.

Morecroft, J., R. Sanchez, and A. Heene (eds). 2002. Systems Perspectives on Resources, Capabilities, and Management Processes. Amsterdam: Pergamon.

Nonaka, I. and H. Takeuchi. (1995). The Knowledge-Creating Company. New York: Oxford University Press.

Norreklit, H. 2000. The balance on the balanced scorecard A critical analysis of some of its assumptions. Management Accounting Research 11: 65-88.

Otley, D.T. 1999. Performance management: A framework for management control systems research. Management Accounting Research: 363-382.

Porter, M. 1980. Competitive strategy. New York: The Free Press.

Porter, M. 1985. Competitive advantage: Creating and sustaining superior performance. New York : Free Press.

Prahalad, C. and G. Hamel, 1990. The core competence of the corporation. Harvard Business Review, 90(3): 79-91.

Rappaport, A. 1999. New thinking on how to link executive pay to performance. Harvard Business Review. Mar-Apr: 91-101.

Rucci, A.J., S.P. Kirn and R.T. Quinn. 1998. The employee-customer-profit chain at Sears. Harvard Business Review 76(1): 82-97.

Sanchez, R., A. Heene and H. Thomas (eds.). 1996. Dynamics of Competence-Based Competition. Oxford: Pergamon.

Senge, P. 1990. The Fifth Discipline: The Art and Practice of the Learning Organizaation. New York: Doubleday.

Simons, R. 1987. Accounting control systems and business strategy: An empirical analysis. Accounting, Organizations and Society 12: 357-374.

Simons, R. 2000. Performance Measurement & Control Systems for Implementing Strategy: Text & Cases. Upper Saddle River, NJ: Prentice Hall.

Van de Ven, A., and Drazin. 1985. The concept of fit in contingency theory. Research in Organizational Behavior 7: 333-65.

Warren, K. 2002. Operationalizing the impact of competence building on the performance of firms resource systems. In Morecroft et al. (eds). 2002. Systems Perspectives on Resources, Capabilities, and Management Processes. Amsterdam: Pergamon.

Yin, R.K. 1994. Case Study Research Design and Methods. Newbury Park, CA: Sage Publications, Inc.

Zimmerman, J. 1997. Accounting for Decision-making and Control. Burr Ridge, IL: Irwin-McGraw-Hill.