documentp1

International Journal of Physical Distribution & Logistics ManagementEmerald Article: Logistics Performance: Definition and MeasurementGarland Chow, Trevor D. Heaver, Lennart E. Henriksson

Article information:

To cite this document: Garland Chow, Trevor D. Heaver, Lennart E. Henriksson, (1994),"Logistics Performance: Definition and Measurement", International Journal of Physical Distribution & Logistics Management, Vol. 24 Iss: 1 pp. 17 - 28

Permanent link to this document: http://dx.doi.org/10.1108/09600039410055981

Downloaded on: 24-11-2012

References: This document contains references to 50 other documents

Citations: This document has been cited by 51 other documents

To copy this document: [email protected]

This document has been downloaded 6975 times since 2005. *

Users who downloaded this Article also downloaded: *

Chris Caplice, Yossi Sheffi, (1995),"A Review and Evaluation of Logistics Performance Measurement Systems", The International Journal of Logistics Management, Vol. 6 Iss: 1 pp. 61 - 74http://dx.doi.org/10.1108/09574099510805279

Stephen M. Rutner, C. John Langley, Jr, (2000),"Logistics Value: Definition, Process and Measurement", The International Journal of Logistics Management, Vol. 11 Iss: 2 pp. 73 - 82http://dx.doi.org/10.1108/09574090010806173

Alain Halley, Alice Guilhon, (1997),"Logistics behaviour of small enterprises: performance, strategy and definition", International Journal of Physical Distribution & Logistics Management, Vol. 27 Iss: 8 pp. 475 - 495http://dx.doi.org/10.1108/09600039710182644

Access to this document was granted through an Emerald subscription provided by NARODNA BIBLIOTEKA SRBIJE For Authors: If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service. Information about how to choose which publication to write for and submission guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.

About Emerald www.emeraldinsight.comWith over forty years' experience, Emerald Group Publishing is a leading independent publisher of global research with impact in business, society, public policy and education. In total, Emerald publishes over 275 journals and more than 130 book series, as well as an extensive range of online products and services. Emerald is both COUNTER 3 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation.

*Related content and download information correct at time of download.

IntroductionLogistics research may be defined as the systematic andobjective search for, and analysis of, information relevantto the identification and solution of any problem in thefield of logistics[1]. A great deal of logistics research isconducted around the premiss that a relationship existsbetween a particular course of action and logisticsperformance (or effectiveness). Unfortunately, drawingbroad inferences from the work that has been done isfrustrating because of the great variety of ways in whichperformance has been defined in the literature. Thedefinition of performance is a challenge for researchers inany field of management because organizations havemultiple and frequently conflicting goals[2]. Some definegoals in terms of profits. Others may choose goals such ascustomer service or sales maximization. Also difficult arethe tasks of selecting and developing adequate measuresfor the chosen definition. “Hard” measures (such as netincome or accounting figures) and “soft” measures (such

as customer satisfaction ratings) each have strengths andweaknesses associated with them.

The purpose of this article is to examine the definitionand measurement of performance in logistics research.We begin with a literature review which includes anexamination of the various ways in which “performance”has been defined. Data collection methods, sources, andthe measures that have been used are also identified.Next, potential sources of performance data are identifiedand discussed. Recommendations arising from the reviewcomplete the article.

Literature ReviewThe contents of five leading logistics journals between1982 and 1992 (International Journal of LogisticsManagement, International Journal of PhysicalDistribution & Logistics Management and itspredecessors, Journal of Business Logistics, Logistics andTransportation Review and Transportation Journal) werereviewed for studies addressing performance. A numberof studies from other sources were selected forexamination on an ad hoc basis.

Summary of the LiteratureThe literature may be divided into six categories. Theyare:

(1) Conceptual works;

(2) Performance definition;

(3) Performance measurement;

(4) “Leading edge” literature;

(5) Performance as an outcome variable;

(6) Mathematical/economic analyses.The first category is composed of three articles and threevolumes. These conceptual works are listed andsummarized briefly in Table I. Armitage studied howmanagement accounting techniques could be used toimprove productivity analysis in distributionoperations[3], while Mentzer and Konrad reviewedperformance measurement practices from an efficiencyand effectiveness perspective[4]. Rhea and Shrockpresented a framework for the development of measuresof the effectiveness of distribution customer serviceprogrammes[5].

While some empirical work is reported in Byrne andMarkham’s volume, a key contribution lies in itsconceptual treatment of measurement issues, particularlyperformance indicators[6]. The volume by La Londe et al.focuses on customer service[7]. This excellent workincludes an ambitious survey of shippers, carriers andwarehouse executives, and provides several ideasrelevant to measuring customer service performance. A

17LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT

Data collection methods, sources and themeasures used are identified.

LogisticsPerformance:Definition andMeasurement

International Journal of Physical Distribution & Logistics Management, Vol. 24 No. 1, 1994, pp. 17-28 © MCB University Press, 0960-0035

Garland Chow, Trevor D. Heaver andLennart E. Henriksson

The authors gratefully acknowledge the financial supportprovided by the Social Sciences and Humanities ResearchCouncil of Canada (SSHRC). An earlier version of this articlewas presented at the 1993 International Logistics Congress inToronto, Canada.

particular contribution of the book lies in its emphasisupon the inter-temporal and multi-dimensional nature ofcustomer service. The volume by the Nevem WorkingGroup is a comprehensive description and application of“hard” performance indicators in the logistics setting[8].

A variety of empirical studies comprise the remainingcategories. Tables II – VI list the studies by category, andidentify the data source and collection method used, howperformance is defined, and whether the measures usedfor performance are soft or hard.

18 IJPD & LM 24,1

Article/book Summary

Armitage[3] Focuses on the use of management accounting techniques to measure (andimprove) efficiency and effectiveness in distribution operations

Mentzer and Konrad[4] Reviews measurement practices and suggests methods for improvement

Rhea and Shrock[5] Defines “physical distribution effectiveness” and discusses some implications formeasurement

Byrne and Markham[6] Focus is on quality; a key value of this book lies in its treatment of performanceindicators for various dimensions of logistics

La Londe, Cooper and Noordewier[7] Focus on customer service and how it may be measured

Nevem Working Group[8] Comprehensive review of performance indicators in logistics

Table I. Conceptual Articles

Data collectionArticle/book method Data source Measures Definition of performance/remarks

Read and Miller[9] Mail survey Managers Soft Quality: (total customer satisfaction, on-timedelivery, zero defects, employee awareness ofquality importance, reduction of the cost ofquality, best-in-class practices, human resourceexcellence, satisfaction of industry regulations,employee education, other)

Harrington, Lambert Archival Firm Hard Vendor performance (criteria developed usingand Christopher[10] brainstorming method): promised lead-time, lead-

time variability, fill rate, discrepancies, totalpurchases

Gassenheimer, Mail survey Executives Soft Logistical performance: length of promised order Sterling and cycle times for base-line/in-stock products,Robicheaux[11] manufacturer’s performance in meeting

promised delivery dates, fill rate onbase-line/in-stock items, advance notice onshipping delays, accuracy of manufacturer inforecasting and committing to estimated shippingdates on contract/project orders, manufacturer'sadherence to special shipping instructions,accuracy in filling orders

Cooper, Browne Personal European-based Soft Study focused upon performance indicators and and Peters[12] interviews companies found significant variation in logistics efficiency

depending upon which performance indicatorswere used

Table II. Empirical Studies: How Performance May Be Measured

Four empirical studies focus primarily upon howperformance could be measured: their content issummarized in Table II. Read and Miller conducted anexploratory study of quality in logistics, using a sampleof firms on a consultant’s mailing list[9]. One of the mostimportant findings of their study is “a clear gap betweenthe importance given to the components of logisticsquality, and the measures being used” (p. 36). Harringtonet al. developed an interesting model for vendorperformance evaluation in a logistics context[10]. A

brainstorming method was used to identify the variousdimensions of vendor performance, and to evaluate thepriority that each deserved. The model was subsequentlytested in a health-care setting. Gassenheimer et al. on theother hand, employed factor analysis to identify thedimensions of “performance”[11]. Their analysis revealedfive key dimensions: “logistical”, “boundary personnel”,“product/product support” [sic], “flexibility andinnovative” [sic] and “inventory assistance” (p. 20). Aprincipal contribution of the paper by Cooper et al. lies in



Bowersox, Daugherty, Mail survey Firm Soft Common attributes index (performance Dröge, Rogers and and interview antecedents). Appendices include attempts to Wardlow[15] associate firm CAI measures with performance;

results revealed few associations that werestatistically significant

Daugherty, Stank and Mail survey Firm Soft Service capabilities of firm (various logistics and Rogers[17] support services)

Daugherty, Sabath and Mail survey Firm Soft Firm’s ability to accommodate special requests: Rogers[19] special service requests, programme support,

customized service, modification while in logisticssystem

Dröge and Mail survey Firm Soft Firm’s ability to accommodate product Germain[20] introduction, product phaseout, product recall and

customization of service levels to specific marketsor customers

Germain[21] Mail survey Firm Soft Ability of firm’s logistics system to accommodate:supply disruption, production schedule changes,in-stream product modification, services levelcustomization, incentive programmes, specialcustomer requests, product introduction, productphase-out, product recall and computerbreakdown

Table IV. Empirical Performance Studies: “Leading-edge” Literature


Clarke[13] Mail survey Executives Soft Productivity (study sought to determine whichmeasures used)

Yavas, Luqmani Mail survey Managers Soft Efficiency (study provides data on efficiency and Quraeshi[14] measure usage, and attempts to correlate this

with a measure of purchasing sophisticationdeveloped by the authors)

Table III. Empirical Studies :Which Performance Measures Are Used

its use of performance indicators for the purpose ofmeasuring logistics efficiency and effectiveness[12].

Two studies, summarized in Table III, report the resultsof surveys which sought to identify the measures used bydecision-makers in assessing logistics performance.Clarke reports how physical distribution productivitywas measured by South Carolina distributionexecutives[13]. Yavas et al. focused a similar study on thepurchasing area. They collected data on the usage ofefficiency measures in Saudi Arabian firms and foundthat utilization of measures was associated withpurchasing sophistication[14]. Given the differentorientations of the two studies, the findings areunderstandably quite dissimilar.

In Leading-edge Logistics, Bowersox and his colleaguesdistinguish between emergent, norm and leading-edgeorganizations on the basis of common attributes index(CAI) scores [15]. However, few significant relationshipswere reported between CAI scores and publicly-availablefinancial performance measures [15, Appendix E, 16].

Four performance-relevant studies have been foundwhich build on the Leading-edge material. A summary ofthese studies (along with the original volume) ispresented in Table IV. An important limitation of the CAI

approach is that it measures potential, that is, servicesoffered rather than actual logistics performance. Forexample, Daugherty et al. suggest that the ability of thelogistics system to accommodate the customization ofservice levels to specific markets or customers is animportant dimension of performance[17]. They measuredperformance by asking respondents to identify which“expanded services” were offered to customers (e.g.customer billing/collection; freight bill audit and paymentand so on). The authors conclude that “firms wishing toimprove logistical performance are well advised toconcentrate on formalizing selected processes” (p. 57).Because the dependent variable is potential rather thanactual service delivery, this conclusion is best regarded astentative and in need of further empirical validation. Thelimitations of outcome variables which may have little orno direct linkage with actual performance are well-known[18, p. 543]. For, as Cooper and his colleaguesnoted, “there can be a large gulf between what companiessay they do and what they actually do” [12, p. 30].

Daugherty et al. found that firms that could bedistinguished as “leading edge” by virtue of severalstructural characteristics performed better thanothers[19]. The authors defined performance as “ease ofaccommodating special requests”, such as sales andmarketing incentive programmes. Again, it is potential,

20 IJPD & LM 24,1


Fawcett and Closs[22]; Mail survey Managers Soft “Competitive position” of firm or profit centre: Fawcett[23] (rated own five perceptual items rating from nontraditional

organization’s measures of cost and customer service to more performance) traditional measure of growth in sales and growth

in return on assets

Perry[24] Site visits, Managers Soft “Operating performance” of specific logisticsquestionnaires (rated own dimensions: vendor relations, material and interviews firm’s acquisition lead-time, purchasing method,

performance) material invoice, transport management/comparison between JIT and non-JIT firms

Rhea and Mail survey Key decision Soft “Distribution effectiveness”: adequacy,Shrock[25, 26] makers consistency, accuracy, timeliness, initiative,

responsiveness

Fawcett[28]; Mail survey Managers Soft A single logistics function, carrier performance:Fawcett and on-time performance, transit time, rates andVellenga[29] tariffs, accuracy, equipment co-ordination,

documentation, information, loss and damage

Marr[30] Mail survey Executives Soft “Distribution service performance”/one-itemperformance measure

Table V. Empirical Performance Studies: Performance as an Outcome Variable or Basis for Comparison

not actual performance that is being measured. Similaroutcome variables were used in Dröge and Germain’sstudy of the effects of formalization[20] and Germain’sstudy of the effects of customization andstandardization[21].

A summary of the studies in which performance waseither investigated as an outcome variable or used as abasis for comparison is provided in Table V. Threestudies investigated variables associated withperformance in a fairly direct fashion. Fawcett and Clossused a promising application of path analysis todemonstrate linkages between “perceived globalization”,“manufacturing”, “logistics”, and the outcome variable,“competitive position”[22]. In another study using thesame database, Fawcett found that firms that emphasizedlogistics issues at a higher level in the early stagesof the co-production decision outperformed theircounterparts[23, p. 39]. In his study of firm behaviour andoperating performance, Perry conducted a qualitativestudy utilizing site visits, questionnaires and interviewswith managers. He found differences between just-in-time(JIT) and non-JIT firms in a number of performance-related areas: vendor relationships, material acquisitionleadtimes, purchasing methods and materialsinventories[24].

Rhea and Shrock utilized a combination of previousresearch and interviews to identify six key elements oflogistics distribution effectiveness[25,26]. They foundthat significant differences exist between “effective” and“ineffective” organizations in the predicted direction. Astrength of this study lies in its use of “customer” foodbroker managers to rate the performance of “seller”producers. While the methodology for collecting the datais quite defensible, a limitation arises from the fact thateach respondent determined both the overall assessmentof four firms as effective or ineffective, and then evaluated

each one on the various elements. The likelihood for aconsistency bias is consequently quite high[27].

Fawcett and Vellenga compared the performance ofmaquiladora and domestic transport operations[28,29].The survey respondents were managers who rated theperformance of transport firms with whom they dealt.The analysis revealed that transport service performancewas less in the transnational operating environment.Marr found “management sophistication” to besomewhat helpful in predicting “distribution serviceperformance”, although a limitation in his findings lies inthe one-item measure used to measure “level of overalldistribution service”[30].

Four studies used mathematical or economic models withhard performance measures, as summarized in Table VI.The papers by Clarke and Gourdin[31] and Kleinsorge etal.[32,33] used data envelopment analysis (DEA) toevaluate the efficiency of logistics practices. Essentially,DEA uses linear programming methods to measureefficient combinations of inputs and outputs for a set ofdecision-making units (DMUs) and provides relativeperformance ratings for all the DMUs in the data set[34].Writers in logistics and other fields have pointed out thatthe advantages of DEA include the less restrictiverequirement for data input in comparison to many other



Clarke and Archival Firm Hard DEA – efficiency/productivity (also includedGourdin[31] evaluation of DEA using questionnaire with soft

measure)

Kleinsorge, Schary Archival Firm Hard DEA – efficiencyand Tanner[32, 33]

Diewert and Smith[37] Archival Firm Hard Total factor productivity

Gomes and Mentzer[38] Simulation Previous Hard Profitability: also examined order cycle time, research variance in order cycle time and order fill rate

Table VI. Empirical Performance Studies: Mathematical/Economic Analyses

Qualitative variables such asattitudes or perceptions

can be included

methods of quantitative analysis. In particular,qualitative variables such as attitudes or perceptions canbe included, thereby allowing the injection of essentialintangibles into the system analysis. DEA has the abilityto isolate variables for measurement while still includingthe effects of interaction among both input and outputvariables[33, p. 39; 35, p. 529). This is in marked contrastto “conventional” performance indicator analyses whichare only incomplete measures of performance[34].Disadvantages of DEA include the frequent difficulty ofobtaining data [36, p. 443]. It should also be noted thatthus far, most applications of DEA have been in thepublic sector[36, p. 443].

The paper by Diewert and Smith is a promising attemptat applying total factor productivity measurement in adistribution setting[37]. Using data from a large applianceparts distributor in Western Canada, the authorsconclude that large productivity gains were possible inthe distribution sector of the economy thanks to thecomputer revolution, which allows a firm to track itspurchases and sales of inventory items and to use thelatest computer software to minimize inventory holdingcosts (p. 16). Finally, Gomes and Mentzer employ asimulation to explore the influence of JIT systems ondistribution system performance[38]. They found thatJIT systems in the physical distribution and materialsmanagement contexts were associated with significantlymore favourable profit and service results in comparisonto non-JIT systems. System-wide JIT, on the other hand,produced favourable service results, but unfavourableprofit results (p. 47).

ImplicationsWith the exception of the mathematical/economic studies,almost all of the empirical studies utilized soft measuresfor the outcome variable. Nevertheless, both soft and hardmeasures are associated with strengths and weaknesses(as a later section of this article discusses in detail). Thislimits a researcher’s ability to infer the existence ofrelationships between logistics performance and itsantecedents.

One limitation common to several of the studies is thatthe respondents appraised their own performance. Thisbecomes a particular concern in cases where theperformance dimension under consideration is betterassessed by another source, such as the customer. Also,very few studies we reviewed adequately captured themultiplicity of goals that must be included in anymeaningful evaluation of performance at either thelogistics or firm level. To be sure, keeping a study withina feasible domain will often involve limiting theexamination to one or more dimensions of “performance”.However, unsupported extrapolations of the findings ofsuch studies to unmeasured dimensions are difficult tojustify. For example, a researcher who finds a significantrelationship between the utilization of total quality

management programmes and customer satisfactionshould indicate that the findings of the study do notnecessarily generalize to dimensions of performance thatwere not included in the study.

The predominance of the mail survey as a data collectionmethod in the logistics studies reviewed raises someconcern in light of its inherent limitations[39]. To theircredit, however, most authors disclosed these limitationsto one extent or another. A few offered especially coherentdiscussions of remedial or assessment measures that hadbeen completed, such as those suggested by Lambert andHarrington[40].

The examination of these studies reveals that an immensevariety of operational definitions and measures exists forlogistics performance. This is the result of the varyinginterests of the researchers and the complexity ofperformance that have been alluded to earlier. Anothersource of the variety in performance measures is thedomain to which the measure is relevant. Several of thestudies measure performance at a logistics activity level(e.g. transport or warehousing). Other studies measureperformance at the logistics function level and severalattempt to measure the firm’s performance.

None of these studies examines logistics performance inthe context of supply chain management. This issignificant. The supply chain comprises all companiesthat participate in transforming, selling and distributingthe product from raw material to the final consumer. Ithas two implications for research in logisticsperformance. First, some research might be orientedtowards measurement of supply chain management, thatis, performance involving multiple organizations. Second,research oriented to members of a channel shouldrecognize the relationship between channel structure andmember functions. For example, distributors in thehealth-care supply chain are increasingly holdinginventory and performing sorting functions traditionallyperformed by hospitals. This integration has resulted inlower inventory and handling costs in the whole supplychain, although specific members of the chain mayexhibit higher inventory and sorting costs than mightotherwise be expected.

22 IJPD & LM 24,1

Distributors in the health-caresupply chain are increasingly

holding inventory

Unfortunately, the variety of performance measuresmake it difficult to draw broad inferences from theliterature about the relationship between a given logisticspractice and performance. Meta-analysis, or aggregatingthe findings of several studies[41], is frustrated by the useof diverse measures. Differences in findings between onestudy and another may be attributable solely to themeasures used.

Defining Logistics PerformanceConceptually, logistics performance may be viewed as asubset of the larger notion of firm or organizationalperformance. The latter has attracted a large volume ofdiverse research over the years[2,35] and illustrates thefutile nature of the search for the “one best way” ofdefining performance. For example, Gleason and Barnumchose to distinguish between effectiveness and efficiency.They defined effectiveness as “the extent to which anobjective has been achieved”, while efficiency was definedas “the degree to which resources have been usedeconomically”[42, p. 380]. Simply put, efficiency is “doingthings right”, while effectiveness is “doing the rightthing”[42, pp. 3,4]. Sink and his colleagues, on the otherhand, defined seven dimensions in order to capture theirconception of “what performance means”: they areeffectiveness, efficiency, quality, productivity, quality ofwork life, innovation and profitability/budgetability[43,pp. 266-7].

Both of these examples have strengths and weaknesses.A principle strength of Gleason and Barnum’s work is itssimplicity. This simplicity may be somewhat deceptive,however: a convincing case could be made about theimportance of efficiency as one dimension ofeffectiveness. As for the dimensions identified by Sinkand his colleagues, they are intriguing because they meetthe need for a broad, comprehensive framework.However, this comes at the price of some overlap betweenthe various dimensions, as the authors themselvesinfer[43,p. 268]. In particular, the distinction betweenperformance and effectiveness is somewhat unclear.

Given the lack of any universally-accepted definition forperformance in the organizational literature, it should not

be surprising that extant literature offers many ideasabout the dimensions that ought to be incorporated into aconceptualization of “logistics performance”. One of thebest examples is the framework presented by Rhea andShrock, where physical distribution effectiveness isdefined as “the extent to which distribution programmessatisfy customers”[5, p. 35]. They note, however, thatother goals remain important under this definition:

In taking this orientation, decision makers do not ignore thewell-documented need to control costs. Rather, theyincorporate this objective within a customer-orientedmanagerial philosophy in which it is believed that the long-run goal of profitability is achieved by providing customerneed and want satisfaction[44; 5, p. 35].

The literature review provided by Rhea and Shrocksuggests that care must be taken to incorporate multiplegoals in defining performance. In order to begin the taskof conceptualizing what the various dimensions oflogistics effectiveness might look like, we identified arepresentative set of answers to the question, “what islogistics performance?” The results of this exercise areshown in Figure 1. Logistics performance may be definedas the extent to which goals such as those suggested inFigure 1 are achieved.

Figure 1 incorporates various possible dimensions ofperformance in a single envelope to help highlight thenumerous interdependencies and conflicts between thegoals. For example, interdependency is likely to existbetween employee satisfaction, the quality of customerservice and profitability. A conflict would occur if a payrise for employees were postponed to achieve a short-term financial improvement – a move which may inhibitthe firm’s ability to attract and retain employees who arecapable of delivering quality customer service to thebenefit of long-run profits. Another example is that a firmmay find it advantageous in the short run to make moreextensive use of packaging in an environmentally-suboptimal way although it may face more stringent


Cost-efficiency

Profitability

Socialresponsibility

On-timedelivery

Sales growth

Job security andworking conditions

Customersatisfaction

Keepingpromises

Flexibility

"Fair" pricesfor inputs

Low lossand damage

Productavailability

Figure 1. What Is Logistics Performance?

Care must be takento incorporate multiple

goals…

regulation, increased paper burden and compliance costsand reduced profits in the long run.

Measuring Logistics PerformanceThe preceding discussion argues that performance ismulti-dimensional. No one measure will suffice forlogistics performance. Instead, the objective forresearchers and managers is to find a set of measureswhich collectively capture most, if not all, of theperformance dimensions thought to be important, overboth short- and long-term horizons. For example, in apresentation at the Tenth International LogisticsCongress in Toronto, in June 1993, Mr D. Eggleton ofRank-Xerox described the criteria on which hisperformance is evaluated as employee satisfaction,customer satisfaction, and the company’s rate of return.

Many dimensions of logistics performance lendthemselves well to hard performance measures. Arepresentative collection of these, along with keyadvantages and disadvantages, is shown in Table VII.Hard performance measures such as net income or orderfill rate are typically impersonal, accurate and easy andinexpensive to collect. Measures such as net income, andaccounting ratios such as return on investment (ROI) areuseful ways of capturing profitability, and will often beeasy and inexpensive to collect, particularly wherelogistics is treated as a profit centre. Profitability is aparticularly useful goal because it directly reflects thegoals of all of the organization’s internal constituentgroups to one extent or another, although it may not be agood indicator of the viability of the firm in the long run.

Cost accounting measures may also be useful,particularly in evaluating several dimensions ofefficiency. The data are often highly accurate and, inmany cases, available over long periods of time. However,these measures are not always comparable between oneorganization and another. Changes in accountingpractices may even inhibit valid comparisons within thesame organization over time.

There are some difficulties which are common to both rawfinancial measures and cost accounting data. Becausethey are often considered confidential, many firms arereluctant to release information to outsiders. Also, inmaking comparisons between organizations or timeperiods, variations in standards or accounting methodsare a frequent threat[45]. A particular problem to logisticsresearchers is that in many cases, the level of aggregationis so high that it is difficult to utilize for evaluating sub-functions of the firm.

The use of input-output ratios (also known asproductivity or performance indicators) is common inlogistics, and has received extensive treatments intextbooks and other literature[8, 46]. Many goals lendthemselves well to evaluation using these measures. Forexample, productivity may be measured utilizing ratiossuch as number of shipments per vehicle-mile, while aratio such as percentage orders delivered on time will behelpful in evaluating the quality of service rendered. Asthe concern of society over environmental issues grows,some firms may find it useful to calculate ratios such as“tons of packaging used/total tonnage shipped”. Again,the limited ability of the researcher to gain access to thesedata because of their confidential nature is a potentialdisadvantage. Also, variations in definitions and data

24 IJPD & LM 24,1

Measure type Advantages/disadvantages

Raw financial statistics (e.g. net income, Advantages: Often easy and inexpensive to collect, and is likely to be comparable gross sales) between organizations. Can capture several important dimensions of

performance, often with impressive accuracyCost statistics (e.g. transport cost,standard labour costs) Disadvantages: May not be comparable between one time interval and another.

Accounting methods may limit comparability between organizations. Level ofaggregation may be so large that it is difficult to assign responsibility. Firmsmay be unwilling to divulge information

Input/output measures or “performance Advantages: Can be used to evaluate goal attainment in many areas, particularly indicators” (e.g. number of shipments/ efficiency and effectivenessvehicle hour)

Quality measures (e.g. order cycle time) Disadvantages: Narrow focus upon individual performance indicators may easilycause faulty analysis or decision-making. Researchers may have trouble gettingdata. May be incomparable across organizations

Table VII. Alternative “Hard” Measures for Logistics Performance

collection procedures will often make it difficult tocompare performance indicators between oneorganization and another.

For service measures such as order cycle time or leadtime variability, the advantages and disadvantages areessentially the same as those of performance indicators.One limitation common to both is that there are manydimensions of performance which they cannot capture,particularly the extent to which customers are satisfied.

The difficulty in capturing customer satisfaction is theunderlying reason that hard measures should besupplemented with “soft”, perceptual ones. Althoughthere are several dimensions of logistics performancewhich hard measures cannot capture in a meaningfulway, customer satisfaction is perhaps the most critical. Aset of soft measures, collected using techniques such asthe mail survey, telephone interview, or similar methodare needed. Besides their usefulness in identifyingproblems, soft measures may also be called for whereavailable hard measures are not comparable between oneorganization and another because of differences inaccounting standards or similar problems. Thesemeasures are subject to the limitations inherent in anyself-report, such as consistency bias, and the socialdesirability problem[27,39]. The difficulties are especiallyserious for one-item measures[27].

Soft measures may also raise comparability problems.Suppose that a handful of manufacturers in the sameindustry report (on a scale of 1 to 5) how well theirlogistics function achieves on-time delivery goals. Theresponses may be difficult to interpret if (as will likely bethe case), there is variation in the competitive strategy orgoals of the firms. This may well influence what isreported by the respondent. As an example, consider twofirms with an actual on-time delivery level of 80 per cent.Each firm’s manager is asked to rate performance. IfFirm A’s goal is 100 per cent and Firm B’s goal is 80 percent, it is likely that Firm A’s manager will provide alower performance rating than Firm B’s. In othercontexts, low scores on “quality of service” may reflectthe presence of a cost-minimization strategy which arenot meaningfully comparable to those of a firm seeking todifferentiate itself by providing superior service. Soft

measures are associated with a host of other limitations,the most notable being self-report defects[27] or otherforms of bias[39].

An excellent way to improve the validity of soft measuresmay be found in Churchill’s paradigm for developingbetter measures of marketing constructs[47]. Essentially,the paradigm comprises a number of carefully-orderedsteps by which literature searches, surveys and analysisprocedures are linked together in a coherent and justifiedsequence. First, the domain of the construct is specified.Next, a sample of items is generated. Third, the measureis purified. Reliability and validity of the construct arethen assessed. The Churchill paradigm has beenemployed by a handful of logistics scholars[22], withpromising results.

The discussion presented here suggests that although theoptimal set of performance measures will depend on thepurpose of the research, it will often include a collection ofboth hard and soft measures. One important criterion toconsider in choosing the set is “representativeness”. Thatis, the set of measures should meaningfully capture thosedimensions of performance in which the researcher isinterested. The use of only one or two measures ofperformance is justified for the researcher whose studyaddresses only customer satisfaction or cost efficiency.However, findings from such studies should not becarelessly extrapolated to include the larger variable,logistics performance.

RecommendationsThis article argues that defining and measuringperformance in logistics is a difficult enterprise, for bothresearchers and managers. The literature review revealsthe nature and limitations of the various designs andmeasures which have been used thus far and, along withsubsequent sections, suggests that there are no “easy”ways to address the issues raised. In light of this review,we offer five recommendations which should helpimprove the quality of future research.

More Efforts to Develop Performance MeasuresIn the short run, we urge researchers to give moredetailed information about how they have defined andmeasured performance, and about the limitations of theirstudy and its findings. Inadequate reporting of theappropriate caveats serves only to mislead readers andfrustrate logistical excellence.

Special issues of logistics journals devoted tomeasurement and methodological issues would be mostuseful. Their leading role would be very likely to generateadded interest in logistics research methodology amongresearchers and students. A more coherent picture of thevarious dimensions of performance would make it much


Factor analysis willbe an especially

useful tool

easier to address subsequent questions such as, “how”should each dimension be measured?”, and “who shouldprovide data for each measurement?”

Increased attention to the development of valid measuresis warranted, particularly the soft ones which capturedimensions of performance which are otherwise left out.The encouraging results of the study by Gassenheimerand his colleagues, in which factor analysis is used,suggest that this will be an especially useful tool fordeveloping valid perceptual or self-report scales[11]. Asnoted earlier, expanded use of the paradigm suggested byChurchill[47] is very likely to enhance the validity ofstudies that are conducted, and also facilitate aggregationand comparison among them.

Encouragement of More Innovative Research DesignsStudies in which more than one constituency providesdata in evaluating performance should beencouraged[48]. Although “rate-your-own-company”studies have a legitimate place in research, they need tobe supplemented by data from other constituent groups,especially customers.

While we suspect that the mail survey will continue to bean important data-collection method, journal editorsshould encourage studies in which alternative methodsare used. Also, in the short run, journal reviewers andeditors should insist that submissions articulate andjustify how performance has been defined, and describethe measures that have been used. In particular,questionnaires should be reproduced verbatim in anappendix, or an address provided where interestedresearchers may obtain a copy. In this way, a body oftime-tested items can be built up in short order, meta-analyses are facilitated, and needless “re-invent thewheel” exercises are avoided.

Quantitative techniques such as data envelopmentanalysis and total factor productivity both representpotentially valuable ways to evaluate economicperformance, and are potentially important tools inlogistics research. However, these should not bepermitted to obscure the potential gains in knowledgethat may be facilitated by qualitative studies, mostnotably the “case study” approach. One particular form of

case study that has proven invaluable for many firms isthe “logistics performance audit” in which a teamsystematically evaluates a firm’s logistics function.

Development of Contingency Models of LogisticsPerformanceResearchers might do well to explore contingency modelsof logistics performance[49,50]. To date, their utilizationhas been sparse, despite their intuitive appeal. Ratherthan the “one-best-way” paradigm which is common in somany discussions of logistics research, a contingencymodel would be based on the supposition that the fitbetween logistics organization and strategy and theorganization’s environment, product line, productiontechnology and size will influence performance outcomevariables. The development and testing of such a model might well stimulate research on the question of theprimacy of various performance dimensions. Forexample, the firm which has adopted a cost-minimizationstrategy may be better served by focusing on costdimensions of performance than the firm which haschosen to differentiate itself on quality of service. Whilethis assertion may have intuitive appeal, it has little in theway of empirical support thus far.

Recognition of the Implications of Supply ChainManagementThe shift to supply chain management has twoimplications for logistics performance. First, themeasurement of performance must recognize theparticular role of an organization in a supply chain.Comparable stages in different channels may beassociated with different functions. Second, considerationshould be given to assessing the performance of thesupply chain, not just that of individual participants.

More Bridge-building between Theory and Practice IsNeededResearchers can and should play a role in makingpractitioners more familiar with the importance, natureand limitations of research. Practitioners, on the otherhand, can offer needed insights into the types of logisticsissues that merit greater study and thought, and moregenerally, what our overall “focus” should be. Takingadvantage of whatever opportunities we receive toengage in dialogue will help ensure that the research weconduct fulfils pragmatic purposes in the long run.

References1. Chow, G. and Henriksson, L. E., A Critique of Survey

Research in Logistics, Working Paper 93-TRA-009,University of British Columbia, Faculty of Commerceand Business Administration, 1993.

2. Hall, R., Organizations: Structures, Processes andOutcomes, Prentice-Hall, New York and London, 1991,p. 267.

26 IJPD & LM 24,1

Quantitative techniques arepotentially important in

logistics research

3. Armitage, H.M., “The Use of Management AccountingTechniques to Improve Productivity Analysis inDistribution Operations”, International Journal ofPhysical Distribution & Materials Management, Vol. 14No. 1, 1984, pp. 41-51.

4. Mentzer, J.T. and Konrad, B.P., “AnEfficiency/Effectiveness Approach to LogisticsPerformance Analysis”, Journal of Business Logistics,Vol. 12 No. 1, 1991, pp. 33-61.

5. Rhea, M.J. and Shrock, D.L., “Measuring the Effectivenessof Physical Distribution Customer Service Programs”,Journal of Business Logistics, Vol. 8 No. 1, 1987, pp. 31-45.

6. Byrne, P.M. and Markham, W.J., Improving Quality andProductivity in the Logistics Process, Council of LogisticsManagement, Oak Brook, IL, 1991.

7. La Londe, B.J., Cooper, M. C. and Noordewier, T.D.,Customer Service: A Management Perspective, Council ofLogistics Management, Oak Brook, IL, 1988.

8. Nevem Working Group, Performance Indicators inLogistics, IFS Publications, Bedford, 1989.

9. Read, W.F. and Miller, M.S., “The State of Quality inLogistics”, International Journal of Physical Distribution& Logistics Management, Vol. 21 No. 6, 1990, pp. 32-47.

10. Harrington, T.C., Lambert, D.M. and Christopher, M., “AMethodology for Measuring Vendor Performance”,Journal of Business Logistics , Vol. 12 No. 1, 1991,pp. 83-104.

11. Gassenheimer, J.B., Sterling, J.U. and Robicheaux, R.A.,“Long-term Channel Member Relationships”,International Journal of Physical Distribution & LogisticsManagement, Vol. 19 No. 12, 1989, pp. 15-28.

12. Cooper, J.C., Browne, M. and Peters, M., “LogisticsPerformance in Europe: The Challenge of 1992”,International Journal of Logistics Management, Vol. 1No. 1, 1990, pp. 28-34.

13. Clarke, R.L., “The Measurement of Physical DistributionProductivity: South Carolina, a Case in Point”,Transportation Journal, Vol. 31 No. 1, Fall 1991, pp. 14-21.

14. Yavas, U., Luqmani, M. and Quraeshi, Z.A., “PurchasingEfficacy in an Arabian Gulf Country”, InternationalJournal of Physical Distribution & Logistics Management,Vol. 19 No. 4, 1989, pp. 20-5.

15. Bowersox, D.J., Daugherty, P.L., Dröge, C.L., Rogers, D.S.and Wardlow, D.L., Leading-edge Logistics: CompetitivePositioning for the 1990s, Council of LogisticsManagement, Oak Brook, IL, 1989.

16. Chow, G., review of Leading Edge Logistics: CompetitivePositioning for the 1990s. Logistics and TransportationReview, Vol. 27 No. 4, December 1991, pp. 384-7.

17. Daugherty, P.J., Stank, T.P. and Rogers, D.S., “The Impactof Formalization on Warehousing Firms”, InternationalJournal of Logistics Management, Vol. 3 No. 2, 1992,pp. 49-61.

18. Cameron, K.S., “Effectiveness as Paradox: Consensus andConflict in Conceptions of Organizational Effectiveness”,Management Science, Vol. 32 No. 5, May 1986, pp. 539-57.

19. Daugherty, P.J., Sabath, R.E. and Rogers, D.S.,“Competitive Advantage through CustomerResponsiveness”, Logistics and Transportation Review,Vol. 28 No. 3, September 1992, pp. 257-71.

20. Dröge, C. and Germain, R., “The Impact of theCentralized Structuring of Logistics Activities on Span ofControl, Formalization and Performance”, Journal ofthe Academy of Marketing Science, Vol. 17 No. 1, 1989,pp. 83-9.

21. Germain, R., “Output Standardisation and LogisticalStrategy, Structure and Performance”, InternationalJournal of Physical Distribution & Logistics Management,Vol. 19 No. 1, 1989, pp. 21-9.

22. Fawcett, S.E. and Closs, D.J., “Coordinated GlobalManufacturing, the Logistics/Manufacturing Interaction,and Firm Performance”, Journal of Business Logistics,Vol. 14 No. 1, 1993, pp. 1-25.

23. Fawcett, S.E., “The Status and Impact of Logistics Issuesin the Success of Co-production via Maquiladoras”,International Journal of Logistics Management, Vol. 2 No.2, 1991, pp. 30-41.

24. Perry, J.H., “Firm Behavior and Operating Performancein Just-in-Time Logistics Channels”, Journal of BusinessLogistics, Vol. 9 No. 1, 1988, pp. 19-33.

25. Rhea, M.J. and Shrock, D.L., “Measuring DistributionEffectiveness with Key Informant Reports”, Logistics andTransportation Review, Vol. 23 No. 3, September 1987,pp. 295-306.

26. Rhea, M.J. and Shrock, D.L., “Physical DistributionImplementation Effectiveness: The CustomerPerspective”, Transportation Journal, Vol. 27 No. 1, Fall1987, pp. 36-42.

27. Podsakoff, P. and Organ, D.W., “Self-reports inOrganizational Research: Problems and Prospects”,Journal of Management, Vol. 12 No. 4, 1986, pp. 531-44.

28. Fawcett, S.E., “Logistics and Manufacturing Issues inMaquiladora Operations”, International Journal ofPhysical Distribution & Logistics Management, Vol. 20No. 4, 1990, pp. 13-21.

29. Fawcett, S.E. and Vellenga, D.B., “TransportationCharacteristics and Performance in MaquiladoraOperations”, Transportation Journal, Vol. 31 No. 4,Summer 1992, pp. 5-16.

30. Marr, N.E., “Management Sophistication and ServicePerformance”, International Journal of PhysicalDistribution & Logistics Management, Vol. 21 No. 4, 1991,pp. 32-41.

31. Clarke, R.L. and Gourdin, K.N., “Measuring the Efficiencyof the Logistics Process”, Journal of Business Logistics,Vol. 12 No. 2, Fall 1991, pp. 17-33.

32. Kleinsorge, I.K., Schary, P.B. and Tanner, R., “EvaluatingLogistics Decisions”, International Journal of PhysicalDistribution & Materials Management, Vol. 19 No. 12,1989, pp. 2-14.

33. Kleinsorge, I.K., Schary, P.B. and Tanner, R., “TheShipper-Carrier Partnership: A New Tool forPerformance Evaluation”, Journal of Business Logistics,Vol. 12 No. 2, 1991, pp. 35-57.

34. Hensher, D.A. and Waters, W.G., II, Using Total FactorProductivity and Data Envelopment Analysis forPerformance Comparisons among GovernmentEnterprises: Concepts and Issues (unpublishedmanuscript), Institute of Transport Studies, University ofSydney, Australia, 1993.


35. Lewin, A.Y. and Minton, J.W., “DeterminingOrganizational Effectiveness: Another Look, and anAgenda for Research”, Management Science, Vol. 32No. 5, May 1986, pp. 514-38.

36. Charnes, A., Cooper, W.W. and Rhodes, E., “MeasuringEfficiency of Decision Making Units”, European Journalof Operational Research, Vol. 2 No. 6, 1978, pp. 429-44.

37. Diewert, W.E. and Smith, A.M., “ProductivityMeasurement for a Distribution Firm” (unpublishedmanuscript), University of British Columbia, Departmentof Economics, 1992.

38. Gomes, R. and Mentzer, J.T., “The Influence of Just-in-Time Systems on Distribution Channel Performance inthe Presence of Environmental Uncertainty”,Transportation Journal, Vol. 30 No. 4, Summer 1991,pp. 36-48.

39. Armstrong, J.S., Long Range Forecasting (2nd ed.), Wiley,New York and Chichester, 1985.

40. Lambert, D.M. and Harrington, T.C., “MeasuringNonresponse Bias in Customer Service Mail Surveys”,Journal of Business Logistics, Vol. 11 No. 2, 1990, pp. 5-25.

41. Hunter, J.E., Schmidt, F.L. and Jackson, G.B., Meta-Analysis: Cumulating Research Findings across Studies,Sage, New York, NY, 1982.

42. Gleason, J.M. and Barnum, D.T., “Toward Valid Measuresof Public Sector Productivity: Performance Measures inUrban Transit”, Management Science, Vol. 28 No. 4, April1986, pp. 379-86.

43. Sink, D.S., Tuttle, T.C. and DeVries, S.J., “ProductivityMeasurement and Evaluation: What Is Available?”,

National Productivity Review, Vol. 4 No. 3, 1984, pp. 265-387.

44. Lambert, D.M. and Zemke, R., “The Customer ServiceComponent of the Marketing Mix”, Proceedings,Twentieth Annual Conference, National Council ofPhysical Distribution Management, 1982, pp. 1-24.

45. Dess, G.G. and Robinson, R.B., Jr, “MeasuringOrganizational Performance in the Absence of ObjectiveMeasures: The Case of the Privately Held Firm andConglomerate Business Unit”, Strategic ManagementJournal, Vol. 5, 1984, pp. 265-73.

46. Glaskowski, N.A., Hudson, D.R. and Ivie, R.M., BusinessLogistics (3rd ed.), Dryden, New York, NY, 1992.

47. Churchill, G.A., Jr, “A Paradigm for Developing BetterMeasures of Marketing Constructs”, Journal ofMarketing Research, Vol. 16, February 1979, pp. 64-73.

48. Jobson, J.D. and Schneck, R., “Constituent Views ofOrganizational Effectiveness: Evidence from PoliceOrganizations”, Academy of Management Journal,Vol. 25 No. 1, February 1982, pp. 25-46.

49. Persson, G., “Organisation Design Strategies forBusiness Logistics”, International Journal of PhysicalDistribution & Materials Management, Vol. 8 No. 6, 1978,pp. 287-97.

50. Pfohl, H.C. and Zöllner, W., “Organisation for Logistics:The Contingency Approach”, International Journal ofPhysical Distribution & Materials Management, Vol. 17No. 1, 1987, pp. 3-16.

28 IJPD & LM 24,1

Garland Chow is an Associate Professor, Trevor D. Heaver a Professor and Lennart E. Henriksson a Research Associate,all in the Faculty of Commerce and Business Administration, University of British Columbia, Vancouver, Canada.

documentp1

Documents