IJOPM2211

1288

International Journal of Operations ampProduction ManagementVol 22 No 11 2002 pp 1288-1310 MCB UP Limited 0144-3577DOI 10110801443570210450329

The success and failure ofperformance measurement

initiativesPerceptions of participating managers

Mike Bourne and Andy NeelyCentre for Business Performance Cranfield School of Management

Cranfield UK and

Ken Platts and John MillsCentre for Strategy and Performance Manufacturing Engineering

University of Cambridge Cambridge UK

Keywords Performance measurement Management Implementation

Abstract This paper investigates the success and failure of performance measurement systemdesign interventions in ten companies In each case the senior management team was facilitatedthrough a management process to redesign their performance measurement systems Analysis ofthe initial pilot case studies suggested three important differences between companies thatproceeded to implement the measures agreed during the process and those that did not Postintervention semi-structured interviews with the directors and managers directly involved revealedtwo main perceived drivers of implementation and four perceived factors that block implementationThe results are of specific interest for performance measurement system implementation but havewider implications for our view of management commitment in change management

IntroductionWith the balanced scorecard being cited by Harvard Business Review as one ofthe most important management tools of the last 75 years performancemeasurement is currently attracting a great deal of interest among bothindustrialists and academics alike However although there are numerousbalanced performance measurement frameworks (Keegan et al 1989 Lynchand Cross 1991 Fitzgerald et al 1991 Kaplan and Norton 1992 Neely et al2002) and management processes for the design of performance measurementsystems (Bitton 1990 Dixon et al 1991 Kaplan and Norton 1993 1996 Neelyet al 1996 Bititci et al 1998 Krause and Mertins 1999) there has been lessresearch into the success and failure of performance measurement initiatives

From industrial conferences (eg Business Intelligence 2000) one may wellcome to the conclusion that there are few problems with implementing newperformance measurement systems There are certainly many success stories(eg Kaplan and Norton 2000 the Mobile case) but there is now a growingliterature addressing the difficulties of implementation and it is claimed by somethat 70 per cent of performance measurement initiatives fail (McCunn 1998)

T h e c u r re n t i s s u e a n d fu ll t e x t a r c h iv e o f th is jo u r n a l is a v a i la b le a t

httpwwwem eraldinsightcom0144-3577htm

Research for this paper was conducted during the research project ndash Manufacturing Strategyand Performance Measurement ndash which was sponsored by the CDP section of EPSRC undergrant numbers GRH21470 and GRK53086

Performancemeasurement

initiatives

1289

Much of the existing literature is based on practitioner reflections and therehave been few research studies of performance measurement implementation(Lewy and Du Mee 1998 Bourne et al 1999 Hudson et al 2001 excepted) Butdrawing from this literature and categorising them using Pettigrew et alrsquos(1989) organisational context development process and measurement contentthe main reasons for success and failure are currently believed to be

(1) Contextual issues

The need for a highly developed information system (Bierbusse andSiesfeld 1997)

Time and expense required (Bierbusse and Siesfeld 1997 McCunn1998)

Lack of leadership and resistance to change (Hacker and Brotherton1998 Meekings 1995)

(2) Processual issues

Vision and strategy were not actionable (Kaplan and Norton 1996)as there were difficulties in evaluating the relative importance ofmeasures and the problems of identifying true ` driversrsquorsquo (Bierbusseand Siesfeld 1997 Schneiderman 1999)

Strategy was not linked to resource allocation (Kaplan and Norton1996 Meekings 1995)

Goals were negotiated rather than based on stakeholderrequirements (Schneiderman 1999)

State of the art improvement methods were not used (Schneiderman1999)

Striving for perfection undermined success (McCunn 1998Schneiderman 1999)

(3) Content issues

Strategy was not linked to department team and individual goals(Kaplan and Norton 1996 Bierbusse and Siesfeld 1997Schneiderman 1999)

Large number of measures diluted the overall impact (Bierbusse andSiesfeld 1997)

Metrics were too poorly defined (Schneiderman 1999)

The need to quantify results in areas that are more qualitative innature (Bierbusse and Siesfeld 1997)

If one reviews this list the majority of the items are process and measurementcontent issues the very issues the performance measurement design processesreferenced above are specifically developed to address This leaves the classicchange management issues of leadership and resistance to change with the

IJOPM2211

1290

other contextual factors of time and expense and IT systems Given that muchis based on practionersrsquo reflections further research is required

The change management literature provides general advice about howchange should be implemented (eg Pugh 1978 Beer et al 1990 Duck 1993Eccles 1994) and why change fails (eg Kotter 1995) but there is a dearth ofresearch based studies of performance measurement system implementationsAs Frizellersquos (1991) study of studies showed there are company generalattributes project general attributes and project specific attributes Thecompany and project general attributes (such as top management commitmenta perceived need for change and involving the end users) are well knownattributes promoted in the change management literature However the projectspecific attributes for performance measurement are not

Therefore this paper investigates the major factors that impact the successand failure of the implementation of a performance measurement systemfollowing action research interventions in ten manufacturing companies Allthe interventions were undertaken using the same management process so that` process related factorsrsquorsquo could be eliminated from the investigation Althoughthere was extensive case study data collected during six of the interventionsthe results of the case and cross-case analysis raised more questions than theyanswered Therefore post intervention interviews were conducted with the 25directors and managers who were directly involved in the performancemeasurement projects The interviews focused on what the participatingdirectors and managers perceived were the main reasons for success the mainreasons for failure and the difficulties encountered and overcome

An analysis of the interviews revealed that the directors and managersperceived that there were six main factors that influenced success and failureThese split into two main drivers of success and four barriers The barriers toimplementation also split into two categories those that were insurmountableand those that nearly all companies met but some overcame and others did not

The paper will therefore make a contribution by giving us a betterunderstanding of process success clarifying and modifying our understandingof specific performance measurement project success factors and presentingone of the basic tenets of the change management literature in a new light Thepaper will conclude by highlighting the insights and shortcomings of thesefindings and identifying the need for further research

BackgroundAll the companies involved in this research were medium sized (50-500employees) manufacturing businesses either privately-owned companies orsubsidiaries of larger groups The senior management team of each business(the managing director or general manager and his direct reports see Table I)were facilitated through the same tried and tested performance measurementsystem design process (Neely et al 1996) Each application took between fiveand six workshops to complete The intended outcome was an agreed top levelbalanced scorecard for the business and this was achieved in eight out of theten applications In two applications the management team failed to complete

Performancemeasurement

initiatives

1291

Table ISummary of initial

cases

Com

pan

yan

dgiv

ennam

eA

Con

trol

sL

tdB

Mac

hin

eShop

CC

ompon

ents

Plc

DIn

stru

men

tsL

tdE

Pum

ps

Ltd

FSea

ls

Ow

ner

ship

Subsi

dia

ryof

UK

Plc

Subsi

dia

ryof

larg

eU

Sco

rpor

atio

n

Subsi

dia

ryof

larg

eJa

pan

ese

corp

orat

ion

Ven

ture

capit

albac

ked

Pri

vat

ely

owned

Subsi

dia

ryof

pri

vat

egro

up

Busi

nes

sC

ontr

olIn

stru

men

tsG

roup

centr

alE

uro

pea

nm

achin

esh

op

Com

pon

ents

man

ufa

cture

ran

dsu

pplier

Man

ufa

cture

rsof

leak

det

ecti

oneq

uip

men

t

Man

ufa

cture

rsof

pum

ps

and

turb

ines

Man

ufa

cture

rsof

seal

san

dco

mpre

ssio

npac

kin

gs

Sen

ior

team

par

tici

pat

ing

inpro

cess

MD

pro

duct

ion

dir

ecto

rco

mm

erci

aldir

ecto

ran

dte

chnic

aldir

ecto

rpar

t1

only

Gro

up

man

ufa

cturi

ng

man

ager

si

tegen

eral

man

ager

and

six

loca

lm

anag

ers

Man

agin

gdir

ecto

rtw

odir

ecto

rsan

dfo

ur

man

ager

s

Man

agin

gdir

ecto

ran

dfo

ur

man

ager

s

Man

agin

gdir

ecto

ran

dfo

ur

dir

ecto

rs

Sit

edir

ecto

rfo

ur

BU

gen

eral

man

ager

san

d11

man

ager

s

Pro

gres

sP

art

1G

roups

esta

blish

edY

esY

esY

esY

esY

esY

esP

art

2O

bje

ctiv

esag

reed

Yes

Yes

Yes

Yes

Yes

Yes

Par

t3

Mea

sure

sdes

igned

No

Yes

Yes

Yes

Yes

Yes

Par

t4

Mea

sure

sag

reed

No

Yes

Yes

Yes

Yes

Yes

Par

t5

Mea

sure

sim

ple

men

ted

No

No

No

Yes

Yes

Yes

IJOPM2211

1292

the process The study was deliberately limited to the implementation of thetop level measures (between 7 and 17 across the cases) and excludedinvestigations of the further cascading of supporting measures (although therewas evidence of this occurring in all successful companies)

Prior to the intervention all the companies had been using financialperformance measures as the basis of regular business level managementreviews although some non-financial performance measures were in use in themain functions (eg sales manufacturing) After the interventions the eightcompanies completing the process populated all four of Kaplan and Nortonrsquos(1992) scorecard perspectives None of the scorecards was unduly biased to anyone perspective although the innovation and learning perspective wasconsistently the least measured

The workshops were deliberately planned at three to four week intervals sothat the development of the scorecard became part of every day business andnot just a `weekend awayrsquorsquo exercise The teams completed most of thedevelopment during the workshops but the design of the measures (Neely et al1997) was conducted outside the main workshops by members of the seniormanagement team and their direct reports The measures so developed werepresented back to the senior team as a whole and reviewed at the followingworkshop The whole design process took between 15 and 26 weeks with thesubsequent implementation in successful companies (from the end of theworkshops to measures being displayed around the business) a further nine to13 months (Bourne et al 2000)

The main commitment the businesses made to the project was managementtime initially in attending the workshops and developing the measures butlatterly in coordinating the implementation work of others Allimplementations involved in-house IT personnel and although differentapproaches were used to extract the data all used standard spread sheettechnology to record and present the final measures No external IT resourceswere used in any of the interventions

Success in this research was defined as occurring when the managementteams used the majority of the measures in the management of their businessThe evidence to confirm this came from attending at least two board meetingsreviewing company reporting documentation and observing the measuresbeing displayed around the business (in departments and in the canteen orother central meeting places) However none of the companies who failed toprogress their implementation denied the fact so there was no dispute betweenthe researchers and company personnel over whether the measures wereimplemented or not

In the next section the pilot cases are described with the interviews in thefollowing section

The pilot casesEarlier research (Bourne et al 1999) investigated the success and failure ofperformance measurement interventions through action research case studiesconducted in six manufacturing companies (hereafter referred to as the pilot cases)

Performancemeasurement

initiatives

1293

During these interventions the senior management team of the case studycompanies were facilitated through a performance measurement design processusing a common tried and tested methodology (Neely et al 1996) Data werecollected before during and after the interventions using a detailed case studyprotocol (Bourne et al 1997) based on the prescriptions of Yin (1994) and Pettigrewet alrsquos (1989) framework for research strategic change Table I summarises the casecompanies involved and the progress towards implementation each achieved

The detailed case and cross-case analysis revealed three factors in thepattern of the data that differentiated the companies that successfullyimplemented the performance measures from the companies that did not Thesewere

(1) Purpose At the start of the project the senior manager or director ofunsuccessful companies tended to express his purpose for undertakingthe project in terms of improving their companiesrsquo performancemeasures and performance measurement The senior director ofsuccessful companies tended to express their purpose for undertakingthe project in terms of managing the business better

(2) Structure All the successful companies were privately owned All theunsuccessful companies were part of larger publicly quoted groups

(3) Culture All the successful companies had cultures that could beconsidered to be paternalistic The culture of the unsuccessfulcompanies was not described this way

These factors are discussed in turn

(1) Probably the most compelling of the three factors identified above is` purposersquorsquo It could be argued that if companies clearly saw performancemeasurement as a technique for improving the management of thebusiness and moving the business forward they would be more likely togo on and implement the measures Companies that did not have thisclarity of vision would not go on to implement the measures

(2) The fact that privately-owned companies implemented the measures andsubsidiaries of larger groups did not is an observation not an explanationAn explanation of how the ` structurersquorsquo impacted implementation of themeasures requires an understanding of the interactions between theparent company and its subsidiary In particular the interactions whichnegatively influenced the implementation of the performance measuresneed to be identified Although there are some indications of how parentcompany initiatives had a negative impact on the performancemeasurement implementation there is insufficient evidence from thesecases to make a clear statement of how this worked in practice

(3) Finally although all the successful companies had cultures which couldbe considered to be paternalistic this was not measured scientificallyThe attribution of ` paternalisticrsquorsquo to the culture of the companies wasmade from direct observation of the strategy formulation process The

IJOPM2211

1294

attribution then appeared in the cross-case analysis It can be argued thathaving a paternalistic culture could be beneficial for the implementationof performance measures One would expect the fear of the consequencesof measurement to be less reducing the resistance to the implementation

However considering the substantial time and effort invested in conductingthese six case studies the results were disappointing It was therefore decidedto follow up the pilot case studies with a series of semi-structured interviews totry to obtain some greater insights into why the interventions succeeded incertain circumstances and did not in others

The interview researchInterviews in the six pilot case companies were undertaken as this allowed theresults from the case studies to be analysed in parallel with the perceptions ofthe managers and directors who had directly participated in the processworkshops However there was an opportunity to interview managers anddirectors of four other companies who had also undertaken the process in the12 months preceding the pilot cases These included three companies in whichthe process was facilitated by one of their own employees and one company inwhich the process had been facilitated by the one of the paperrsquos authors as partof his training in using the process These additional four companies were self-selecting as being the only other group of companies to whom the researcherhad access and who had recently attempted to implement a performancemeasurement system using the Neely et al (1996) process

Given the small population of potential intervieweersquos and the dearth of ourknowledge on performance measurement implementation problems fromevaluating the methodological options available (Mintzberg 1973) it wasdecided to adopt a grounded approach (Glaser and Straus 1967) rather than usea formal questionnaire Therefore a semi-structured interview protocol wasdesigned (and pre-tested) with the intention of gaining uninhibited responses toopen ended questions concerning the reasons why implementation proceededor not However there was also a concern that the interviewee may focus onone single factor and ignore other important factors To avoid this problem andto provide some degree of consistency across interviews a semi-structuredapproach was adopted with the resulting interview format being as follows

each interview started with a short series of open ended questions

the responses to these open ended questions were probed to ensure thatthe interviewer understood the points being made and to gather specificexamples

the open ended questions were followed by questions focused on aprompt list of possible factors

the responses to the prompt list were probed to ensure understanding

The prompt list was developed from the original research design and sofocused on Pettigrew et alrsquos (1989) dimensions of organisational contextperformance measurement content and the process of designing and

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1290

other contextual factors of time and expense and IT systems Given that muchis based on practionersrsquo reflections further research is required

The change management literature provides general advice about howchange should be implemented (eg Pugh 1978 Beer et al 1990 Duck 1993Eccles 1994) and why change fails (eg Kotter 1995) but there is a dearth ofresearch based studies of performance measurement system implementationsAs Frizellersquos (1991) study of studies showed there are company generalattributes project general attributes and project specific attributes Thecompany and project general attributes (such as top management commitmenta perceived need for change and involving the end users) are well knownattributes promoted in the change management literature However the projectspecific attributes for performance measurement are not

Therefore this paper investigates the major factors that impact the successand failure of the implementation of a performance measurement systemfollowing action research interventions in ten manufacturing companies Allthe interventions were undertaken using the same management process so that` process related factorsrsquorsquo could be eliminated from the investigation Althoughthere was extensive case study data collected during six of the interventionsthe results of the case and cross-case analysis raised more questions than theyanswered Therefore post intervention interviews were conducted with the 25directors and managers who were directly involved in the performancemeasurement projects The interviews focused on what the participatingdirectors and managers perceived were the main reasons for success the mainreasons for failure and the difficulties encountered and overcome

An analysis of the interviews revealed that the directors and managersperceived that there were six main factors that influenced success and failureThese split into two main drivers of success and four barriers The barriers toimplementation also split into two categories those that were insurmountableand those that nearly all companies met but some overcame and others did not

The paper will therefore make a contribution by giving us a betterunderstanding of process success clarifying and modifying our understandingof specific performance measurement project success factors and presentingone of the basic tenets of the change management literature in a new light Thepaper will conclude by highlighting the insights and shortcomings of thesefindings and identifying the need for further research

BackgroundAll the companies involved in this research were medium sized (50-500employees) manufacturing businesses either privately-owned companies orsubsidiaries of larger groups The senior management team of each business(the managing director or general manager and his direct reports see Table I)were facilitated through the same tried and tested performance measurementsystem design process (Neely et al 1996) Each application took between fiveand six workshops to complete The intended outcome was an agreed top levelbalanced scorecard for the business and this was achieved in eight out of theten applications In two applications the management team failed to complete

Performancemeasurement

initiatives

1291

Table ISummary of initial

cases

Com

pan

yan

dgiv

ennam

eA

Con

trol

sL

tdB

Mac

hin

eShop

CC

ompon

ents

Plc

DIn

stru

men

tsL

tdE

Pum

ps

Ltd

FSea

ls

Ow

ner

ship

Subsi

dia

ryof

UK

Plc

Subsi

dia

ryof

larg

eU

Sco

rpor

atio

n

Subsi

dia

ryof

larg

eJa

pan

ese

corp

orat

ion

Ven

ture

capit

albac

ked

Pri

vat

ely

owned

Subsi

dia

ryof

pri

vat

egro

up

Busi

nes

sC

ontr

olIn

stru

men

tsG

roup

centr

alE

uro

pea

nm

achin

esh

op

Com

pon

ents

man

ufa

cture

ran

dsu

pplier

Man

ufa

cture

rsof

leak

det

ecti

oneq

uip

men

t

Man

ufa

cture

rsof

pum

ps

and

turb

ines

Man

ufa

cture

rsof

seal

san

dco

mpre

ssio

npac

kin

gs

Sen

ior

team

par

tici

pat

ing

inpro

cess

MD

pro

duct

ion

dir

ecto

rco

mm

erci

aldir

ecto

ran

dte

chnic

aldir

ecto

rpar

t1

only

Gro

up

man

ufa

cturi

ng

man

ager

si

tegen

eral

man

ager

and

six

loca

lm

anag

ers

Man

agin

gdir

ecto

rtw

odir

ecto

rsan

dfo

ur

man

ager

s

Man

agin

gdir

ecto

ran

dfo

ur

man

ager

s

Man

agin

gdir

ecto

ran

dfo

ur

dir

ecto

rs

Sit

edir

ecto

rfo

ur

BU

gen

eral

man

ager

san

d11

man

ager

s

Pro

gres

sP

art

1G

roups

esta

blish

edY

esY

esY

esY

esY

esY

esP

art

2O

bje

ctiv

esag

reed

Yes

Yes

Yes

Yes

Yes

Yes

Par

t3

Mea

sure

sdes

igned

No

Yes

Yes

Yes

Yes

Yes

Par

t4

Mea

sure

sag

reed

No

Yes

Yes

Yes

Yes

Yes

Par

t5

Mea

sure

sim

ple

men

ted

No

No

No

Yes

Yes

Yes

IJOPM2211

1292

the process The study was deliberately limited to the implementation of thetop level measures (between 7 and 17 across the cases) and excludedinvestigations of the further cascading of supporting measures (although therewas evidence of this occurring in all successful companies)

Prior to the intervention all the companies had been using financialperformance measures as the basis of regular business level managementreviews although some non-financial performance measures were in use in themain functions (eg sales manufacturing) After the interventions the eightcompanies completing the process populated all four of Kaplan and Nortonrsquos(1992) scorecard perspectives None of the scorecards was unduly biased to anyone perspective although the innovation and learning perspective wasconsistently the least measured

The workshops were deliberately planned at three to four week intervals sothat the development of the scorecard became part of every day business andnot just a `weekend awayrsquorsquo exercise The teams completed most of thedevelopment during the workshops but the design of the measures (Neely et al1997) was conducted outside the main workshops by members of the seniormanagement team and their direct reports The measures so developed werepresented back to the senior team as a whole and reviewed at the followingworkshop The whole design process took between 15 and 26 weeks with thesubsequent implementation in successful companies (from the end of theworkshops to measures being displayed around the business) a further nine to13 months (Bourne et al 2000)

The main commitment the businesses made to the project was managementtime initially in attending the workshops and developing the measures butlatterly in coordinating the implementation work of others Allimplementations involved in-house IT personnel and although differentapproaches were used to extract the data all used standard spread sheettechnology to record and present the final measures No external IT resourceswere used in any of the interventions

Success in this research was defined as occurring when the managementteams used the majority of the measures in the management of their businessThe evidence to confirm this came from attending at least two board meetingsreviewing company reporting documentation and observing the measuresbeing displayed around the business (in departments and in the canteen orother central meeting places) However none of the companies who failed toprogress their implementation denied the fact so there was no dispute betweenthe researchers and company personnel over whether the measures wereimplemented or not

In the next section the pilot cases are described with the interviews in thefollowing section

The pilot casesEarlier research (Bourne et al 1999) investigated the success and failure ofperformance measurement interventions through action research case studiesconducted in six manufacturing companies (hereafter referred to as the pilot cases)

Performancemeasurement

initiatives

1293

During these interventions the senior management team of the case studycompanies were facilitated through a performance measurement design processusing a common tried and tested methodology (Neely et al 1996) Data werecollected before during and after the interventions using a detailed case studyprotocol (Bourne et al 1997) based on the prescriptions of Yin (1994) and Pettigrewet alrsquos (1989) framework for research strategic change Table I summarises the casecompanies involved and the progress towards implementation each achieved

The detailed case and cross-case analysis revealed three factors in thepattern of the data that differentiated the companies that successfullyimplemented the performance measures from the companies that did not Thesewere

(1) Purpose At the start of the project the senior manager or director ofunsuccessful companies tended to express his purpose for undertakingthe project in terms of improving their companiesrsquo performancemeasures and performance measurement The senior director ofsuccessful companies tended to express their purpose for undertakingthe project in terms of managing the business better

(2) Structure All the successful companies were privately owned All theunsuccessful companies were part of larger publicly quoted groups

(3) Culture All the successful companies had cultures that could beconsidered to be paternalistic The culture of the unsuccessfulcompanies was not described this way

These factors are discussed in turn

(1) Probably the most compelling of the three factors identified above is` purposersquorsquo It could be argued that if companies clearly saw performancemeasurement as a technique for improving the management of thebusiness and moving the business forward they would be more likely togo on and implement the measures Companies that did not have thisclarity of vision would not go on to implement the measures

(2) The fact that privately-owned companies implemented the measures andsubsidiaries of larger groups did not is an observation not an explanationAn explanation of how the ` structurersquorsquo impacted implementation of themeasures requires an understanding of the interactions between theparent company and its subsidiary In particular the interactions whichnegatively influenced the implementation of the performance measuresneed to be identified Although there are some indications of how parentcompany initiatives had a negative impact on the performancemeasurement implementation there is insufficient evidence from thesecases to make a clear statement of how this worked in practice

(3) Finally although all the successful companies had cultures which couldbe considered to be paternalistic this was not measured scientificallyThe attribution of ` paternalisticrsquorsquo to the culture of the companies wasmade from direct observation of the strategy formulation process The

IJOPM2211

1294

attribution then appeared in the cross-case analysis It can be argued thathaving a paternalistic culture could be beneficial for the implementationof performance measures One would expect the fear of the consequencesof measurement to be less reducing the resistance to the implementation

However considering the substantial time and effort invested in conductingthese six case studies the results were disappointing It was therefore decidedto follow up the pilot case studies with a series of semi-structured interviews totry to obtain some greater insights into why the interventions succeeded incertain circumstances and did not in others

The interview researchInterviews in the six pilot case companies were undertaken as this allowed theresults from the case studies to be analysed in parallel with the perceptions ofthe managers and directors who had directly participated in the processworkshops However there was an opportunity to interview managers anddirectors of four other companies who had also undertaken the process in the12 months preceding the pilot cases These included three companies in whichthe process was facilitated by one of their own employees and one company inwhich the process had been facilitated by the one of the paperrsquos authors as partof his training in using the process These additional four companies were self-selecting as being the only other group of companies to whom the researcherhad access and who had recently attempted to implement a performancemeasurement system using the Neely et al (1996) process

Given the small population of potential intervieweersquos and the dearth of ourknowledge on performance measurement implementation problems fromevaluating the methodological options available (Mintzberg 1973) it wasdecided to adopt a grounded approach (Glaser and Straus 1967) rather than usea formal questionnaire Therefore a semi-structured interview protocol wasdesigned (and pre-tested) with the intention of gaining uninhibited responses toopen ended questions concerning the reasons why implementation proceededor not However there was also a concern that the interviewee may focus onone single factor and ignore other important factors To avoid this problem andto provide some degree of consistency across interviews a semi-structuredapproach was adopted with the resulting interview format being as follows

each interview started with a short series of open ended questions

the responses to these open ended questions were probed to ensure thatthe interviewer understood the points being made and to gather specificexamples

the open ended questions were followed by questions focused on aprompt list of possible factors

the responses to the prompt list were probed to ensure understanding

The prompt list was developed from the original research design and sofocused on Pettigrew et alrsquos (1989) dimensions of organisational contextperformance measurement content and the process of designing and

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1291

Table ISummary of initial

cases

Com

pan

yan

dgiv

ennam

eA

Con

trol

sL

tdB

Mac

hin

eShop

CC

ompon

ents

Plc

DIn

stru

men

tsL

tdE

Pum

ps

Ltd

FSea

ls

Ow

ner

ship

Subsi

dia

ryof

UK

Plc

Subsi

dia

ryof

larg

eU

Sco

rpor

atio

n

Subsi

dia

ryof

larg

eJa

pan

ese

corp

orat

ion

Ven

ture

capit

albac

ked

Pri

vat

ely

owned

Subsi

dia

ryof

pri

vat

egro

up

Busi

nes

sC

ontr

olIn

stru

men

tsG

roup

centr

alE

uro

pea

nm

achin

esh

op

Com

pon

ents

man

ufa

cture

ran

dsu

pplier

Man

ufa

cture

rsof

leak

det

ecti

oneq

uip

men

t

Man

ufa

cture

rsof

pum

ps

and

turb

ines

Man

ufa

cture

rsof

seal

san

dco

mpre

ssio

npac

kin

gs

Sen

ior

team

par

tici

pat

ing

inpro

cess

MD

pro

duct

ion

dir

ecto

rco

mm

erci

aldir

ecto

ran

dte

chnic

aldir

ecto

rpar

t1

only

Gro

up

man

ufa

cturi

ng

man

ager

si

tegen

eral

man

ager

and

six

loca

lm

anag

ers

Man

agin

gdir

ecto

rtw

odir

ecto

rsan

dfo

ur

man

ager

s

Man

agin

gdir

ecto

ran

dfo

ur

man

ager

s

Man

agin

gdir

ecto

ran

dfo

ur

dir

ecto

rs

Sit

edir

ecto

rfo

ur

BU

gen

eral

man

ager

san

d11

man

ager

s

Pro

gres

sP

art

1G

roups

esta

blish

edY

esY

esY

esY

esY

esY

esP

art

2O

bje

ctiv

esag

reed

Yes

Yes

Yes

Yes

Yes

Yes

Par

t3

Mea

sure

sdes

igned

No

Yes

Yes

Yes

Yes

Yes

Par

t4

Mea

sure

sag

reed

No

Yes

Yes

Yes

Yes

Yes

Par

t5

Mea

sure

sim

ple

men

ted

No

No

No

Yes

Yes

Yes

IJOPM2211

1292

the process The study was deliberately limited to the implementation of thetop level measures (between 7 and 17 across the cases) and excludedinvestigations of the further cascading of supporting measures (although therewas evidence of this occurring in all successful companies)

Prior to the intervention all the companies had been using financialperformance measures as the basis of regular business level managementreviews although some non-financial performance measures were in use in themain functions (eg sales manufacturing) After the interventions the eightcompanies completing the process populated all four of Kaplan and Nortonrsquos(1992) scorecard perspectives None of the scorecards was unduly biased to anyone perspective although the innovation and learning perspective wasconsistently the least measured

The workshops were deliberately planned at three to four week intervals sothat the development of the scorecard became part of every day business andnot just a `weekend awayrsquorsquo exercise The teams completed most of thedevelopment during the workshops but the design of the measures (Neely et al1997) was conducted outside the main workshops by members of the seniormanagement team and their direct reports The measures so developed werepresented back to the senior team as a whole and reviewed at the followingworkshop The whole design process took between 15 and 26 weeks with thesubsequent implementation in successful companies (from the end of theworkshops to measures being displayed around the business) a further nine to13 months (Bourne et al 2000)

The main commitment the businesses made to the project was managementtime initially in attending the workshops and developing the measures butlatterly in coordinating the implementation work of others Allimplementations involved in-house IT personnel and although differentapproaches were used to extract the data all used standard spread sheettechnology to record and present the final measures No external IT resourceswere used in any of the interventions

Success in this research was defined as occurring when the managementteams used the majority of the measures in the management of their businessThe evidence to confirm this came from attending at least two board meetingsreviewing company reporting documentation and observing the measuresbeing displayed around the business (in departments and in the canteen orother central meeting places) However none of the companies who failed toprogress their implementation denied the fact so there was no dispute betweenthe researchers and company personnel over whether the measures wereimplemented or not

In the next section the pilot cases are described with the interviews in thefollowing section

The pilot casesEarlier research (Bourne et al 1999) investigated the success and failure ofperformance measurement interventions through action research case studiesconducted in six manufacturing companies (hereafter referred to as the pilot cases)

Performancemeasurement

initiatives

1293

During these interventions the senior management team of the case studycompanies were facilitated through a performance measurement design processusing a common tried and tested methodology (Neely et al 1996) Data werecollected before during and after the interventions using a detailed case studyprotocol (Bourne et al 1997) based on the prescriptions of Yin (1994) and Pettigrewet alrsquos (1989) framework for research strategic change Table I summarises the casecompanies involved and the progress towards implementation each achieved

The detailed case and cross-case analysis revealed three factors in thepattern of the data that differentiated the companies that successfullyimplemented the performance measures from the companies that did not Thesewere

(1) Purpose At the start of the project the senior manager or director ofunsuccessful companies tended to express his purpose for undertakingthe project in terms of improving their companiesrsquo performancemeasures and performance measurement The senior director ofsuccessful companies tended to express their purpose for undertakingthe project in terms of managing the business better

(2) Structure All the successful companies were privately owned All theunsuccessful companies were part of larger publicly quoted groups

(3) Culture All the successful companies had cultures that could beconsidered to be paternalistic The culture of the unsuccessfulcompanies was not described this way

These factors are discussed in turn

(1) Probably the most compelling of the three factors identified above is` purposersquorsquo It could be argued that if companies clearly saw performancemeasurement as a technique for improving the management of thebusiness and moving the business forward they would be more likely togo on and implement the measures Companies that did not have thisclarity of vision would not go on to implement the measures

(2) The fact that privately-owned companies implemented the measures andsubsidiaries of larger groups did not is an observation not an explanationAn explanation of how the ` structurersquorsquo impacted implementation of themeasures requires an understanding of the interactions between theparent company and its subsidiary In particular the interactions whichnegatively influenced the implementation of the performance measuresneed to be identified Although there are some indications of how parentcompany initiatives had a negative impact on the performancemeasurement implementation there is insufficient evidence from thesecases to make a clear statement of how this worked in practice

(3) Finally although all the successful companies had cultures which couldbe considered to be paternalistic this was not measured scientificallyThe attribution of ` paternalisticrsquorsquo to the culture of the companies wasmade from direct observation of the strategy formulation process The

IJOPM2211

1294

attribution then appeared in the cross-case analysis It can be argued thathaving a paternalistic culture could be beneficial for the implementationof performance measures One would expect the fear of the consequencesof measurement to be less reducing the resistance to the implementation

However considering the substantial time and effort invested in conductingthese six case studies the results were disappointing It was therefore decidedto follow up the pilot case studies with a series of semi-structured interviews totry to obtain some greater insights into why the interventions succeeded incertain circumstances and did not in others

The interview researchInterviews in the six pilot case companies were undertaken as this allowed theresults from the case studies to be analysed in parallel with the perceptions ofthe managers and directors who had directly participated in the processworkshops However there was an opportunity to interview managers anddirectors of four other companies who had also undertaken the process in the12 months preceding the pilot cases These included three companies in whichthe process was facilitated by one of their own employees and one company inwhich the process had been facilitated by the one of the paperrsquos authors as partof his training in using the process These additional four companies were self-selecting as being the only other group of companies to whom the researcherhad access and who had recently attempted to implement a performancemeasurement system using the Neely et al (1996) process

Given the small population of potential intervieweersquos and the dearth of ourknowledge on performance measurement implementation problems fromevaluating the methodological options available (Mintzberg 1973) it wasdecided to adopt a grounded approach (Glaser and Straus 1967) rather than usea formal questionnaire Therefore a semi-structured interview protocol wasdesigned (and pre-tested) with the intention of gaining uninhibited responses toopen ended questions concerning the reasons why implementation proceededor not However there was also a concern that the interviewee may focus onone single factor and ignore other important factors To avoid this problem andto provide some degree of consistency across interviews a semi-structuredapproach was adopted with the resulting interview format being as follows

each interview started with a short series of open ended questions

the responses to these open ended questions were probed to ensure thatthe interviewer understood the points being made and to gather specificexamples

the open ended questions were followed by questions focused on aprompt list of possible factors

the responses to the prompt list were probed to ensure understanding

The prompt list was developed from the original research design and sofocused on Pettigrew et alrsquos (1989) dimensions of organisational contextperformance measurement content and the process of designing and

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1292

the process The study was deliberately limited to the implementation of thetop level measures (between 7 and 17 across the cases) and excludedinvestigations of the further cascading of supporting measures (although therewas evidence of this occurring in all successful companies)

Prior to the intervention all the companies had been using financialperformance measures as the basis of regular business level managementreviews although some non-financial performance measures were in use in themain functions (eg sales manufacturing) After the interventions the eightcompanies completing the process populated all four of Kaplan and Nortonrsquos(1992) scorecard perspectives None of the scorecards was unduly biased to anyone perspective although the innovation and learning perspective wasconsistently the least measured

The workshops were deliberately planned at three to four week intervals sothat the development of the scorecard became part of every day business andnot just a `weekend awayrsquorsquo exercise The teams completed most of thedevelopment during the workshops but the design of the measures (Neely et al1997) was conducted outside the main workshops by members of the seniormanagement team and their direct reports The measures so developed werepresented back to the senior team as a whole and reviewed at the followingworkshop The whole design process took between 15 and 26 weeks with thesubsequent implementation in successful companies (from the end of theworkshops to measures being displayed around the business) a further nine to13 months (Bourne et al 2000)

The main commitment the businesses made to the project was managementtime initially in attending the workshops and developing the measures butlatterly in coordinating the implementation work of others Allimplementations involved in-house IT personnel and although differentapproaches were used to extract the data all used standard spread sheettechnology to record and present the final measures No external IT resourceswere used in any of the interventions

Success in this research was defined as occurring when the managementteams used the majority of the measures in the management of their businessThe evidence to confirm this came from attending at least two board meetingsreviewing company reporting documentation and observing the measuresbeing displayed around the business (in departments and in the canteen orother central meeting places) However none of the companies who failed toprogress their implementation denied the fact so there was no dispute betweenthe researchers and company personnel over whether the measures wereimplemented or not

In the next section the pilot cases are described with the interviews in thefollowing section

The pilot casesEarlier research (Bourne et al 1999) investigated the success and failure ofperformance measurement interventions through action research case studiesconducted in six manufacturing companies (hereafter referred to as the pilot cases)

Performancemeasurement

initiatives

1293

During these interventions the senior management team of the case studycompanies were facilitated through a performance measurement design processusing a common tried and tested methodology (Neely et al 1996) Data werecollected before during and after the interventions using a detailed case studyprotocol (Bourne et al 1997) based on the prescriptions of Yin (1994) and Pettigrewet alrsquos (1989) framework for research strategic change Table I summarises the casecompanies involved and the progress towards implementation each achieved

The detailed case and cross-case analysis revealed three factors in thepattern of the data that differentiated the companies that successfullyimplemented the performance measures from the companies that did not Thesewere

(1) Purpose At the start of the project the senior manager or director ofunsuccessful companies tended to express his purpose for undertakingthe project in terms of improving their companiesrsquo performancemeasures and performance measurement The senior director ofsuccessful companies tended to express their purpose for undertakingthe project in terms of managing the business better

(2) Structure All the successful companies were privately owned All theunsuccessful companies were part of larger publicly quoted groups

(3) Culture All the successful companies had cultures that could beconsidered to be paternalistic The culture of the unsuccessfulcompanies was not described this way

These factors are discussed in turn

(1) Probably the most compelling of the three factors identified above is` purposersquorsquo It could be argued that if companies clearly saw performancemeasurement as a technique for improving the management of thebusiness and moving the business forward they would be more likely togo on and implement the measures Companies that did not have thisclarity of vision would not go on to implement the measures

(2) The fact that privately-owned companies implemented the measures andsubsidiaries of larger groups did not is an observation not an explanationAn explanation of how the ` structurersquorsquo impacted implementation of themeasures requires an understanding of the interactions between theparent company and its subsidiary In particular the interactions whichnegatively influenced the implementation of the performance measuresneed to be identified Although there are some indications of how parentcompany initiatives had a negative impact on the performancemeasurement implementation there is insufficient evidence from thesecases to make a clear statement of how this worked in practice

(3) Finally although all the successful companies had cultures which couldbe considered to be paternalistic this was not measured scientificallyThe attribution of ` paternalisticrsquorsquo to the culture of the companies wasmade from direct observation of the strategy formulation process The

IJOPM2211

1294

attribution then appeared in the cross-case analysis It can be argued thathaving a paternalistic culture could be beneficial for the implementationof performance measures One would expect the fear of the consequencesof measurement to be less reducing the resistance to the implementation

However considering the substantial time and effort invested in conductingthese six case studies the results were disappointing It was therefore decidedto follow up the pilot case studies with a series of semi-structured interviews totry to obtain some greater insights into why the interventions succeeded incertain circumstances and did not in others

The interview researchInterviews in the six pilot case companies were undertaken as this allowed theresults from the case studies to be analysed in parallel with the perceptions ofthe managers and directors who had directly participated in the processworkshops However there was an opportunity to interview managers anddirectors of four other companies who had also undertaken the process in the12 months preceding the pilot cases These included three companies in whichthe process was facilitated by one of their own employees and one company inwhich the process had been facilitated by the one of the paperrsquos authors as partof his training in using the process These additional four companies were self-selecting as being the only other group of companies to whom the researcherhad access and who had recently attempted to implement a performancemeasurement system using the Neely et al (1996) process

Given the small population of potential intervieweersquos and the dearth of ourknowledge on performance measurement implementation problems fromevaluating the methodological options available (Mintzberg 1973) it wasdecided to adopt a grounded approach (Glaser and Straus 1967) rather than usea formal questionnaire Therefore a semi-structured interview protocol wasdesigned (and pre-tested) with the intention of gaining uninhibited responses toopen ended questions concerning the reasons why implementation proceededor not However there was also a concern that the interviewee may focus onone single factor and ignore other important factors To avoid this problem andto provide some degree of consistency across interviews a semi-structuredapproach was adopted with the resulting interview format being as follows

each interview started with a short series of open ended questions

the responses to these open ended questions were probed to ensure thatthe interviewer understood the points being made and to gather specificexamples

the open ended questions were followed by questions focused on aprompt list of possible factors

the responses to the prompt list were probed to ensure understanding

The prompt list was developed from the original research design and sofocused on Pettigrew et alrsquos (1989) dimensions of organisational contextperformance measurement content and the process of designing and

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1293

During these interventions the senior management team of the case studycompanies were facilitated through a performance measurement design processusing a common tried and tested methodology (Neely et al 1996) Data werecollected before during and after the interventions using a detailed case studyprotocol (Bourne et al 1997) based on the prescriptions of Yin (1994) and Pettigrewet alrsquos (1989) framework for research strategic change Table I summarises the casecompanies involved and the progress towards implementation each achieved

The detailed case and cross-case analysis revealed three factors in thepattern of the data that differentiated the companies that successfullyimplemented the performance measures from the companies that did not Thesewere

(1) Purpose At the start of the project the senior manager or director ofunsuccessful companies tended to express his purpose for undertakingthe project in terms of improving their companiesrsquo performancemeasures and performance measurement The senior director ofsuccessful companies tended to express their purpose for undertakingthe project in terms of managing the business better

(2) Structure All the successful companies were privately owned All theunsuccessful companies were part of larger publicly quoted groups

(3) Culture All the successful companies had cultures that could beconsidered to be paternalistic The culture of the unsuccessfulcompanies was not described this way

These factors are discussed in turn

(1) Probably the most compelling of the three factors identified above is` purposersquorsquo It could be argued that if companies clearly saw performancemeasurement as a technique for improving the management of thebusiness and moving the business forward they would be more likely togo on and implement the measures Companies that did not have thisclarity of vision would not go on to implement the measures

(2) The fact that privately-owned companies implemented the measures andsubsidiaries of larger groups did not is an observation not an explanationAn explanation of how the ` structurersquorsquo impacted implementation of themeasures requires an understanding of the interactions between theparent company and its subsidiary In particular the interactions whichnegatively influenced the implementation of the performance measuresneed to be identified Although there are some indications of how parentcompany initiatives had a negative impact on the performancemeasurement implementation there is insufficient evidence from thesecases to make a clear statement of how this worked in practice

(3) Finally although all the successful companies had cultures which couldbe considered to be paternalistic this was not measured scientificallyThe attribution of ` paternalisticrsquorsquo to the culture of the companies wasmade from direct observation of the strategy formulation process The

IJOPM2211

1294

attribution then appeared in the cross-case analysis It can be argued thathaving a paternalistic culture could be beneficial for the implementationof performance measures One would expect the fear of the consequencesof measurement to be less reducing the resistance to the implementation

However considering the substantial time and effort invested in conductingthese six case studies the results were disappointing It was therefore decidedto follow up the pilot case studies with a series of semi-structured interviews totry to obtain some greater insights into why the interventions succeeded incertain circumstances and did not in others

The interview researchInterviews in the six pilot case companies were undertaken as this allowed theresults from the case studies to be analysed in parallel with the perceptions ofthe managers and directors who had directly participated in the processworkshops However there was an opportunity to interview managers anddirectors of four other companies who had also undertaken the process in the12 months preceding the pilot cases These included three companies in whichthe process was facilitated by one of their own employees and one company inwhich the process had been facilitated by the one of the paperrsquos authors as partof his training in using the process These additional four companies were self-selecting as being the only other group of companies to whom the researcherhad access and who had recently attempted to implement a performancemeasurement system using the Neely et al (1996) process

Given the small population of potential intervieweersquos and the dearth of ourknowledge on performance measurement implementation problems fromevaluating the methodological options available (Mintzberg 1973) it wasdecided to adopt a grounded approach (Glaser and Straus 1967) rather than usea formal questionnaire Therefore a semi-structured interview protocol wasdesigned (and pre-tested) with the intention of gaining uninhibited responses toopen ended questions concerning the reasons why implementation proceededor not However there was also a concern that the interviewee may focus onone single factor and ignore other important factors To avoid this problem andto provide some degree of consistency across interviews a semi-structuredapproach was adopted with the resulting interview format being as follows

each interview started with a short series of open ended questions

the responses to these open ended questions were probed to ensure thatthe interviewer understood the points being made and to gather specificexamples

the open ended questions were followed by questions focused on aprompt list of possible factors

the responses to the prompt list were probed to ensure understanding

The prompt list was developed from the original research design and sofocused on Pettigrew et alrsquos (1989) dimensions of organisational contextperformance measurement content and the process of designing and

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1294

attribution then appeared in the cross-case analysis It can be argued thathaving a paternalistic culture could be beneficial for the implementationof performance measures One would expect the fear of the consequencesof measurement to be less reducing the resistance to the implementation

However considering the substantial time and effort invested in conductingthese six case studies the results were disappointing It was therefore decidedto follow up the pilot case studies with a series of semi-structured interviews totry to obtain some greater insights into why the interventions succeeded incertain circumstances and did not in others

The interview researchInterviews in the six pilot case companies were undertaken as this allowed theresults from the case studies to be analysed in parallel with the perceptions ofthe managers and directors who had directly participated in the processworkshops However there was an opportunity to interview managers anddirectors of four other companies who had also undertaken the process in the12 months preceding the pilot cases These included three companies in whichthe process was facilitated by one of their own employees and one company inwhich the process had been facilitated by the one of the paperrsquos authors as partof his training in using the process These additional four companies were self-selecting as being the only other group of companies to whom the researcherhad access and who had recently attempted to implement a performancemeasurement system using the Neely et al (1996) process

Given the small population of potential intervieweersquos and the dearth of ourknowledge on performance measurement implementation problems fromevaluating the methodological options available (Mintzberg 1973) it wasdecided to adopt a grounded approach (Glaser and Straus 1967) rather than usea formal questionnaire Therefore a semi-structured interview protocol wasdesigned (and pre-tested) with the intention of gaining uninhibited responses toopen ended questions concerning the reasons why implementation proceededor not However there was also a concern that the interviewee may focus onone single factor and ignore other important factors To avoid this problem andto provide some degree of consistency across interviews a semi-structuredapproach was adopted with the resulting interview format being as follows

each interview started with a short series of open ended questions

the responses to these open ended questions were probed to ensure thatthe interviewer understood the points being made and to gather specificexamples

the open ended questions were followed by questions focused on aprompt list of possible factors

the responses to the prompt list were probed to ensure understanding

The prompt list was developed from the original research design and sofocused on Pettigrew et alrsquos (1989) dimensions of organisational contextperformance measurement content and the process of designing and

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1295

developing the measures As can be seen from Table II these prompts weretailored to elicit the main reasons for continuing to implement the performancemeasures and the main reasons for not continuing Successful companies werealso asked to identify problems they had encountered but overcome

Data were collected during the interviews through the researcher makingdetailed notes as the interview was in progress These notes were then writtenup within 24 hours of the interview and then later used as the basis for analysisThe analysis involved coding the responses given and using this summariseddata to identify the main themes arising during the interviews

Because the interviews were in a semi-structured format the interviewertook advantage of the discretion this allowed to follow up the answers given tothe original questions In most of the interviews undertaken with thecompanies that did not proceed to implement their performance measuresdetailed probing of the responses given led to far greater insights into why theinitiatives failed than given in the initial bland responses These responseswere gained by asking oblique additional questions which reflected back whatthe interviewer had revealed but questioned the completeness of the initialresponse In all cases this elicited information of a more confidential nature andas a direct result the description of the four additional cases has had to beheavily disguised

The semi-structured interviews were conducted with directors andmanagers who had originally directly participated in the initial performancemeasurement design workshops These interviews were conducted betweennine and 18 months after the intervention so that it would be clear whether ornot the measures designed and agreed during the process were implemented ornot

The results of the interviews are presented in two groups

(1) The participating managers in the companies who successfullyimplemented their performance measures (Leak Detection Ltd PumpsLtd and Seals cases D E and F from the pilot cases above)

(2) The participating managers in the companies who did not implementtheir performance measures (Machine Shop Controls Ltd andComponents plc cases A B and C from the pilot cases above) and in fourother companies who had previously attempted the process (FoodConsumer Consumables Ltd Models Ltd and Specialist Engineeringcases G H I and J)

The successful companiesLeak Detection Ltd Pumps Ltd and Seals all progressed to implement theperformance measures designed and agreed during the performancemeasurement workshops The next section focuses on the reasons forcontinuing and the following section discusses the difficulties overcome

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1296

Table IISummary of promptedinterview responses

Rea

sons

for

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Rea

sons

for

not

conti

nuin

gfr

omlist

Fre

quen

cyof

resp

onse

Dif

ficu

ltie

sov

erco

me

from

list

Fre

quen

cyof

resp

onse

This

was

exac

tly

the

kin

dof

init

iati

ve

we

nee

ded

6T

his

kin

dof

init

iati

ve

was

nev

ergoi

ng

tow

ork

her

eD

oing

this

kin

dof

init

iati

ve

was

alw

ays

goi

ng

tobe

dif

ficu

ltin

this

com

pan

y

3

Peo

ple

mad

eti

me

for

the

pro

ject

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

2T

her

ew

asa

lack

ofm

anag

emen

tti

me

3

We

got

goo

dea

rly

resu

lts

4W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

1W

ehad

pro

ble

ms

imple

men

ting

the

mea

sure

sac

tion

pla

ns

we

des

igned

5

The

key

peo

ple

push

edth

epro

ject

alon

g5

Key

peo

ple

left

and

the

mom

entu

mw

aslo

stK

eypeo

ple

left

and

the

mom

entu

mw

aslo

stT

he

pro

ject

was

giv

enpri

orit

y4

The

pro

ject

was

over

taken

by

other

even

ts3

The

pro

ject

was

nea

rly

over

taken

by

other

even

ts2

Sen

ior

man

agem

ent

wer

eco

mm

itte

dto

doi

ng

this

8Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

ed3

Sen

ior

man

agem

ent

com

mit

men

tto

the

pro

ject

wan

edW

ehad

not

got

ever

yth

ing

out

ofth

epro

ject

by

then

5W

egot

what

we

wan

ted

from

the

pro

ject

and

stop

ped

1W

egot

what

we

init

ially

wan

ted

from

the

pro

ject

and

ther

ew

aspre

ssure

tost

opT

he

pro

ject

was

wel

lm

anag

edan

dco

ordin

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

2T

he

pro

ject

was

bad

lym

anag

edor

coor

din

ated

1

The

faci

lita

tion

bro

ught

out

the

key

issu

es7

The

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

faci

lita

tion

did

nrsquot

mee

tup

toou

rre

quir

emen

tsT

he

pro

cess

focu

sed

us

onkey

poi

nts

5T

he

pro

cess

was

too

cum

ber

som

e1

The

pro

cess

was

dif

ficu

ltto

use

1

Ther

ew

asen

thusi

asm

toco

nti

nue

afte

rth

efa

cilita

tor

left

1T

he

faci

lita

tor

left

and

the

pro

ject

pet

ered

out

The

faci

lita

tor

left

and

mom

entu

mw

aslo

stW

ew

ere

get

ting

alo

tou

tof

the

pro

cess

for

the

tim

ew

eto

ok6

We

did

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

1Som

edid

not

thin

kth

ere

sult

sw

ere

wor

thth

eef

fort

we

put

into

the

pro

ject

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1297

The successful companiesrsquo reasons for continuingAnalysis of the semi-structured interview results revealed five main themes asto why these companies progressed to implement the performance measuresThese five themes (in order of frequency of citation) were

(1) The benefits of performance measurement (oded as ` benefitsrsquorsquo)

(2) Continued top management commitment (coded as ` TMCrsquorsquo)

(3) Time and effort required (coded as ` worth effortrsquorsquo)

(4) As a consequence of the activities of the internal or external facilitator(coded as ` facilitatorrsquorsquo)

(5) The juxtaposition of the performance measurement intervention withother projects (coded as ` other projectsrsquorsquo)

These themes are now addressed in turnBenefits When asked why they had continued every interviewee

commented on the benefits they perceived arising from performancemeasurement For example

They are working if we had not measured we wouldnrsquot have seen the improvement in orderquality and on-time delivery The actions taken were a result of having the measures(Managing Director Leak Detection Ltd)

Firstly they [the measures] focused on issues which were important to the company andsecondly they forced us to look at the measures and in particular the changes when they werenot happening and being reflected in the measures (Managing Director Pumps Ltd)

Without it we wouldnrsquot be where we are and wersquove moved a hell of a long way (GeneralManager A Seals Ltd)

From these responses it appears that the managers involved believed thatperformance measurement was ` a good thingrsquorsquo per se and that it was deliveringbusiness results For example

Because we believe in it I believe in this more than a lot of other things (Works DirectorPumps Ltd)

There is a belief now in the management team [and later] itrsquos bringing home the bacon(PersonnelManager Seals)

` Benefitsrsquorsquo was by far the most cited reason for continuing being cited moretimes than all the other reasons summed together

TMC Top management commitment (TMC) to the project was cited in allthe cases as a reason for continuing Given the attention top managementcommitment receives in the change management literature (Kotter 1996Frizelle 1991) this result is not unexpected However it is interesting that it iscited far less frequently than ` benefitsrsquorsquo

Worth effort The time and effort required to design and implement themeasures was raised in two companies (Pumps Ltd and Seals) Time and effortare usually reasons for not continuing (see difficulties overcome below) butwere raised here as the interviewees believed the results were worth the timeand effort required For example

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1298

Wersquore not just doing this for your benefit Mike we wouldnrsquot be doing this if we didnrsquot think itwas worth it (Sales Director Pumps Ltd)

[We continued] partly due to the process which was tortuous but got buy in (Site DirectorSeals)

Further performance measurement had continued while other importantprojects had not continued because time and effort was not available

People are exceptionally busy The PampL [divisional profit and loss account project] has fallenby the wayside because of other pressures (General Manager B Seals)

Minor themes The role of the facilitator was cited in Leak Detection Ltd andSeals as were the influence of other projects in Seals but because of theirinfrequency of citation they have been included as minor themes

The successful companiesrsquo difficulties overcomeIn the second part of the semi-structured interviews interviewees were asked todiscuss some of the problems they faced when designing and implementing theperformance measurement systems and in particular the difficulties they hadovercome Analysis of the semi-structured interview results revealed four mainthemes which were cited across all three case companies These four themes (inorder of citation) were

(1) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(2) The time and effort required (coded as ` effortrsquorsquo)

(3) Difficulties concerned with developing appropriate measures (coded as`measuringrsquorsquo)

(4) The personal consequences of performance measurement (coded as` consequencesrsquorsquo)

In addition interviewees in Seals also commented on the impact of the culturein different parts of the organisation (coded as ` culturersquorsquo) and the politicalproblems of dealing with head office as a supplier (coded as ` politicsrsquorsquo)

IT Data access was a recurring problem in two companies (Leak DetectionLtd and Pumps Ltd) Both had relatively new computer systems Thecomments captured reflect difficulties in data accuracy data access and thetime and effort required for programming the data extraction reports requiredMany of the problems at Leak Detection Ltd were overcome by manualcollection of the data but transaction volumes were too high for this to be donesuccessfully at Pumps Ltd Therefore at Pumps Ltd the implementation of themeasures was totally dependent on the programming being completed

Seals operated a much older computer system run on a main frame computerat their head office in the South of England However they had the advantageof being able to directly access the data they wanted and download it onto localPC-based spread sheets using a package called Power Play IT was therefore asignificantly smaller problem for Seals and was even cited as a reason forcontinuing by the site director

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1299

Effort Besides the time and effort required to access data from the ITsystems interviewees also commented on the time and effort required forperformance measurement Comments were made in general terms

Finding the time there is never quite enough (Operations Manager Seals Ltd)

Time is a commodity which is incredibly precious especially for something that is new andhas not proved itself (General Manager A Seals Ltd)

Also they were made in terms of setting up the system

Initially it took time to set the data up I had to dig out a yearrsquos worth of back data a yearrsquosworth of warranty returns for example I had to find half an hour here and half an hour thereto do it Now it takes no time at all (Quality Manager Leak Detection Ltd)

In addition they were made in terms of using the implemented measures

Now it takes time for the management meetings because I have to research the reasons for agood or bad performance and explain them (Works Director Pumps Ltd)

Measuring The difficulty of measuring was raised during interviews across allthree cases

There is a problem with re-quotes when they go out of validity without this it is not asensible measure The conversion rate is key but we need to know the accuracy Have we lostthe order or re-quoted (Sales Manager Leak Detection Ltd)

had a great deal of difficulty with the measure of increasing the number of orders forspecial products (Sales Director Pumps Ltd)

the [problem of the] rigorousness of what gets included in the subjective measures (SiteDirector Seals)

These comments appear to illustrate that the managers have understood theimportance of the definitions of the individual measures and how they werestruggling to overcome these difficulties through implementing meaningfulmeasures

Consequences The personal consequences of being measured wereoccasionally raised during the interviews and are included as a theme as theywere cited across all three cases For example consequences were cited by thesite director at Seals

Itrsquos perceived as a big stick but that is their perception it should be a good way ofmanaging

They were seen as one of the reasons behind the slow implementation of themeasures in Pumps Ltd

there was the problem a reluctance to implement the measures (Managing DirectorPumps Ltd)

Much more severe reactions to the personal consequences of the implementationof performance measurement systems are described in later interviews withmanagers of companies that did not progress to implementation The type ofpersonal consequence found in these cases is probably best illustrated by LeakDetection Ltdrsquos sales manager in his comment

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1300

Presenting it [the performance measure] to people when you have a problem What annoysme is the operations manager is saying why donrsquot you do this and that I donrsquot want theirinput what do they know about sales I will ask if I want it I sort of resent it I do resent it

Minor themes Within one case Seals three minor themes were identifiedChanging the culture was seen as one of the main benefits of implementingperformance measurement (Thompson 1997) The differences between thecultures of the business units and service departments was cited as a reason forthe different rates of progress with business units implementing the measuresmuch more rapidly than the service functions This could be interpreted asresistance to performance measurement because of the personal consequencesof doing so but it was never described as such

` Politicsrsquorsquo also appears in this case The head office site was a major supplierof raw materials to Seals and believed to be one of their poorest performingsuppliers Two of the general managers wished to raise the issue with headoffice as result of the performance measurement project but the site directorprevented them from doing so

Summary From the initial analysis of the results of the semi-structured postprocess interviews in the successful companies five reasons for continuing toimplement the performance measures were identified and seven difficultiesovercome In the next section the results from the unsuccessful companies arepresented and discussed

The unsuccessful companiesAs previously described the pilot case companies Controls Ltd Machine Shopand Components plc all failed to implement the performance measuresdesigned and agreed during the performance measurement workshops Inaddition access was available to four other companies who had previouslyattempted the process (Food Consumer Consumables Ltd Models Ltd andSpecial Engineering) All four of these companies had also failed to implementthe performance measures

Analysis of the semi-structured interview results revealed seven mainthemes as to why these companies did not implement the performancemeasures These seven themes (in order of frequency of citation) were

(1) Time and effort required (coded as ` effortrsquorsquo)

(2) The personal consequences of implementing the performance measures(coded as ` consequencesrsquorsquo)

(3) The perceived lack of benefit from proceeding with performancemeasurement (coded as ` benefitsrsquorsquo)

(4) Difficulties with data access and the information technology systems(coded as ` ITrsquorsquo)

(5) Continued top management commitment (coded as `TMCrsquorsquo)

(6) The impact of parent company activities and initiatives (coded as` parent companyrsquorsquo)

(7) Problems with applying the process (coded as ` processrsquorsquo)

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1301

These are now addressed in turnEffort As found with the successful pilot company cases finding the time

and effort was a problem and it was the most frequently cited reason for notcontinuing in these seven cases For example

We are an under-resourced low volume company and there was permanently a lack of timeand resource for doing things like this (Commercial Manager Special Engineering)

Finding the people and the time to do it is the real problem (Director Models Ltd)

But also finding the time and effort became linked to the perceived benefits ofcontinuing with performance measurement For example

we werenrsquot sure what we were going to get out of it time became a premium and we lostcommitment (Operations Director Controls Ltd)

There is a lack of confidence in the performance measures if we go through the time andeffort of producing the performance measures will it actually do anything different at the endof the day (Director Models Ltd)

Consequences Here the personal consequences are raised in two differentsituations

First there is the situation where the individual who has the authority toimplement the performance measures takes a deliberate decision not to do so ashe believes it will portray adverse performance of the business for which he hasresponsibility For example

The ones [performance measures] we had got are the ones the corporate wanted us to haveand for which we get the stars (UK Manufacturing Manager Machine Shop)

It is not just the problem of change itrsquos exposing yourself All the worms are under the stonesand the performance measurement system lifts the stones off (Director Models Ltd)

Second there is the situation where those being measured were in fear ofperformance measurement (in the same sense that Deming 1986 writes aboutthe fear and is similarly raised in the context of performance measurement byMeekings 1995)

It is just a question of using the measures we are not mature enough yet to talk about solvingthe problems everybody sees the measures as a personal attack on themselves We are atwo excuse company the first excuse is the figures are bollocks the second excuse is shootthe messenger (Commercial Manager Special Engineering)

maybe you ought to run the next project on how you stop the bully especially when thebully is the boss (Internal Facilitator Food)

In both instances the consequences for the individual of performancemeasurement appeared to be a major factor in the measures not beingimplemented

Benefits The benefits from performance measurement were cited as a reasonfor continuing by every interviewee from the successful case companies In thisgroup of unsuccessful companies lack of perceived benefit was cited threetimes (Controls Ltd Machine Shop and Models Ltd)

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1302

IT As with the successful companies access to data was cited by threeinterviewees who reached the stage of agreeing their performance measures(Components Plc Machine Shop and Models Ltd)

TMC Top management commitment was seen to wane in a number of ways

as there was a lack of commitment to implement the action plan (SpecialEngineering)

as benefits became unsure and time a premium (Controls Ltd)

as everyone had their own agenda (Components Plc)

Parent company The influence of the parent company on the implementation ofthe performance measurement system was very strong in four cases First atComponents and Machine Shop the parent company influenced what wasreported

For Ray if thatrsquos his key performance measure [market share] then they [Japanese parentcompany] will measure him on that Very heavily driven by the Japanese measures ndash theydrive us (Operations Manager Components Plc)

the ones we had got are the ones the corporate wanted us to have (UK ManufacturingManger Machine Shop)

Second at Consumer Consumables Ltd and Components Plc changes instrategy were decided at the level of the parent company and then imposed onthe subsidiary These changes in strategy by the parent company are changeswhich go to the heart of the rationale behind undertaking the performancemeasurement process and therefore it is probably not surprising that thesecompanies did not succeed in implementing their measures

Process On two occasions the application of the process was considered afactor In the first instance (Food) this occurred when the process was appliedin a research organisation and the facilitator found adapting the processdifficult In the second instance (Controls Ltd) the operations directorconsidered that the process was inappropriate for the managing director

The prompted responsesThe result from the prompted responses (see Table II) highlighted theimportance of senior management commitment to the project and the influenceof other events already identified from the analysis of the open questions butthey also provide one further insight

The responses from the successful companies to the ` reasons for continuingrsquorsquolist and from the unsuccessful companies to the ` reasons for not continuingrsquorsquo listwere strikingly different On the former every reason for continuing was citedat least once On the latter with the exception of ` senior managementcommitment to the project wanedrsquorsquo and ` the project was overtaken by othereventsrsquorsquo most other categories were not cited as reasons for the implementationnot proceeding

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1303

Interpretation of the interview resultsStarting with the prompted responses the difference between the reasons forcontinuing list and reasons for not continuing is interpreted in two ways Firstit provides strong evidence that the application of the process was not a majorfactor influencing success and failure as project management facilitation andthe process were not frequently cited reasons for failure Second the responsessupported the unprompted interview results As the prompt list was added toensure that interviewees considered contextual process and content factorsthis gives some comfort that no major factors were overlooked

With regards to the unprompted responses Table III shows the codedresults from both the successful and unsuccessful companies Table III hasbeen deliberately drawn to superimpose the reasons for continuing cited by thesuccessful companies with the responses from the unsuccessful companieswhere these reasons were lacking Table III is interpreted below

Reviewing Table III there appear to be two types of factors identified herethose factors that compel progress and those factors that block progressTaking the factors in turn and beginning with the factors that compel progress

Top management commitment to the project and the perceived benefitsthe project brought are cited as reasons for continuing by respondents inall the successful cases In comparison either lack of perceived benefit orsenior management commitment waning were cited as reasons for notcontinuing in six other case companies This suggests that these twofactors are immensely important for success

Worth the effort is interpreted as being related to the benefits and effortinvolved and therefore should be seen under those two headings It isincluded here as it suggests that management make a trade-off betweeneffort and benefit

Facilitation relates to the comments about the support provided by both theinternal and external facilitator It can be interpreted as relating either to anexample of action which demonstrated top management commitment or tothe need for the project to be properly managed As these codes alreadyexist facilitation is not considered sufficiently important to be a separatefactor and so is subsumed into TMC and process

Other projects was raised only once and therefore seen as an incidentalcontextual factor

This suggests that the factors compelling progress can be reduced to twoimportant factors the benefit from undertaking the project and topmanagement commitment Some of the comments made suggest that the twoare linked but this needs further investigation

Turning to the blocking factors

The effort required was cited as a reason for terminating the project inall but one of the unsuccessful companies but cited as a difficultyovercome in all the successful companies This suggests that the effortrequired can be overcome under the right circumstances

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1304

Table IIISummary ofunprompted interviewresponses

Succ

essf

ul

com

pan

ies

Unsu

cces

sful

com

pan

ies

Init

ial

pilot

case

sIn

itia

lpilot

case

sP

revio

us

cohor

tca

ses

DE

FA

BC

GH

IJ

Rea

sons

for

continuin

gB

enef

its

Yes

Yes

Yes

Lac

kof

ben

efit

sL

ack

ofben

efit

sL

ack

ofben

efit

sT

MC

(top

man

agem

ent

com

mit

men

t)Y

esY

esY

esL

ack

ofT

MC

Lac

kof

TM

CL

ack

ofT

MC

Wor

thef

fort

Yes

Yes

Fac

ilit

ator

Yes

Yes

Oth

erpro

ject

sY

es

Dif

ficu

ltie

sov

erco

me

Eff

ort

Yes

Yes

Yes

No

No

No

No

No

No

ITY

esY

esN

oN

oN

oM

easu

ring

Yes

Yes

Yes

Con

sequen

ces

Yes

Yes

Yes

No

No

No

No

Pro

cess

No

No

Par

ent

com

pan

yN

oN

oN

o

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1305

IT had a similar pattern of responses to effort and is interpreted in thesame manner

Measurement problems were cited only by the companies who hadreached the stage of implementation These companies were strugglingto develop meaningful measures and the interpretation of the commentsmade is that they were trying to ensure that the results of implementingspecific measures were worth the effort of measuring This suggeststhat companies reaching this stage will find measuring a difficulty butnot be blocked from continuing to implement the measures

Consequences of measuring were raised in both the successful andunsuccessful responses

Process was raised in two companies as a reason for not continuingWhen combined with the prompted response this suggests that asatisfactory application of the process (the point of entry procedureproject management and participation (Platts 1994)) is a necessary butinsufficient factor in determining the successful implementation of theperformance measures

The disruption from other parent company initiatives was cited as areason for not continuing in three cases and was unobserved in thesuccessful cases This suggests that the impact of parent companyinterventions requires further investigation

In summary there appear to be four main blocking factors to implementationof the measures

(1) the effort required

(2) the ease of data accessibility through the IT systems

(3) the consequences of measurement

(4) being overtaken by new parent company initiatives

The problems of effort and IT systems have been demonstrably overcome inthe successful cases and therefore are not considered fatal to projects if theright conditions exist (for example if there is sufficient senior managementcommitment) However the consequences of measurement and parentcompany initiatives require further investigation

DiscussionThe conclusions from the six pilot research cases were that there wereindications of three factors which differentiated between the successful andunsuccessful companies These were

(1) purpose

(2) structure

(3) culture

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1306

These factors could be related to three issues raised during the post processinterviews ndash top management commitment parent company intervention andconsequences of measurement

(1) The fact that the senior directors of successful companies tended toexpress their purpose for undertaking the project in terms of managingthe business better may well be an indication that at the outset there wasa greater level of top management commitment as ` managing thebusiness betterrsquorsquo can be considered a higher goal than simply ` improvingthe measurement systemrsquorsquo

(2) The reason for structure being identified as a factor in the pilot caseswas not understood However the interview results which suggestedthat parent company interventions interrupted implementation providean explanation as to why structure appeared as an important factor

(3) At the end of the pilot cases it was argued that a paternalistic culturemight well be beneficial for the implementation of performancemeasures as this would reduce the fear of measurement and thereforethe resistance to implementation The fact that the consequence ofmeasurement was raised as a major factor during the interviewsstrongly supports this argument

Therefore the semi-structured interviews provide some explanation of theresults obtained from the cross-case analysis and the results of the cross-caseanalysis provide evidence of the impact of three of the factors found from ananalysis of the interviews

In addition to the situation where the findings from case studies andinterviews coincided the semi-structured interviews highlighted three othermain factors that influenced the success or failure of implementation These arenow discussed in turn

(1) The perceived benefit of performance measurement should have beenidentified during the post project interviews in the pilot cases althoughthese interviews focused more on objective rather than subjectivefactors that influenced implementation This may explain why thebenefits were missed but this could also have resulted from the fact thatthe benefits did not occur until much later ndash when the measures hadbeen fully implemented The later explanation is the more plausible andthis suggests that the post project interviews were conducted too earlyto capture the perceived benefits of the project

(2) The IT hurdle was observed in the pilot cases but not consistentlyenough to be cited as a major difference in the cross-case analysis Theinterviews raised the importance of this factor

(3) The effort required largely went unobserved From the nature of theinterventions undertaken it was mainly unobservable and as it was nota factor identified in the academic frameworks used to create the casestudy protocol data were not collected on the effort required

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1307

The use of multiple methodologies in this research highlights some of theadditional insights that can be gained In particular it suggests that the benefitsfrom implementing a performance measurement system occur very late in theprocess and highlights the ` effortrsquorsquo required a factor not previously identifiedas important

ConclusionsThere are many studies of implementation but it is exceptionally rare for astudy to research the implementation of the same artefact in multipleorganisations using a common and moderated process approach Although thisapproach is open to criticism it does control for variables that are notcontrolled in other studies and so provides different insights

This paper makes four contributions as follows

(1) It qualifies Plattsrsquo (1994) findings on process success

(2) It demonstrates that by using a structured performance measurementdesign process approach many of the implementation problemshighlighted in the literature can be overcome

(3) It identifies the major performance measurement project specific factorsinfluencing implementation

(4) It suggests that top management commitment changes during theproject a factor implicit in but ignored by much of the changemanagement literature

These are now discussed in turnFirst Platts (1994) in his research into the application of management

processes for the development of manufacturing strategy identified four factorsassociated with the application of the process which appeared to be significantThese were point of entry (how the introduction and launch was handled)participation (who was involved) project management and procedure (the toolsused in the process itself)

This research supports his findings that these factors are necessary butsuggests that in themselves they may not be sufficient for successfulimplementation of performance measurement systems Other non-processfactors are also important

Second many of the factors causing problems for implementationhighlighted in the introduction to this paper (eg strategy and vision notactionable and measures poorly defined) could be attributed to poor designprocess The fact that none of these issues was raised as an impediment toimplementation by this research suggests that a well designed and executedprocess can overcome these implementation issues

Third given that a suitable management process is correctly applied thefindings from this research proposes that there are four important performancemeasurement project specific implementation factors These are

(1) The effort required for implementation

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1308

(2) The ease of data accessibility through the IT systems

(3) The consequences of performance measurement

(4) The project being overtaken by parent company initiatives

The results also show that problems with the IT systems and the effortrequired can be overcome This suggests that these two blockers are simplyhurdles to implementation rather than factors which completely stop theproject

This work confirms three of the issues thought to be important frompractionersrsquo reflections but qualifies our understanding by differentiatingbetween hurdles and substantial blockers The impact of parent companyinitiatives is a significant factor not previously recognised in the literature

Fourth TMC is widely taken as a major factor influencing the success andfailure of project implementations (Frizelle 1991) In this study the interviewresults identified top management commitment and the benefits fromperformance measurement as being perceived as the two main factors whichdrove implementation But it must be remembered that the interviews were allconducted some time after the process was completed Many of the managerswere therefore reflecting on the perceived benefits of the performance measuresafter they were implemented This raises the question `What was supportingthe performance measurement initiative through to implementation before thebenefits occurredrsquorsquo

The results of this research suggest that management commitment could beinfluenced by the change in balance between the expected benefits of theintervention and the expected effort required to implement the performancemeasures Therefore management commitment may not be static but changeover time as the intervention unfolds Careful reading of the changemanagement literature alludes to this occurring (eg creating a guidingcoalition (Beer et al 1990 Goss et al 1993 Kotter 1996) and establishing theneed for change (Pugh 1978)) but even the processual approaches (Lewin 1947Dawson 1996 Burnes 1997) do not consider how TMC can be influenced overa projectrsquos life If management commitment is dynamic and is a dependant notan independent variable then change management should be looked at in anew light

We would go further and suggest that the insights gained in this studywould not have been raised by a ` scientificallyrsquorsquo designed questionnaireadministered to a representative sample of managers The semi-structuredapproach uncovered the ` non-standardrsquorsquo responses as a direct result of probingand using knowledge from the original case studies The conclusion that topmanagement commitment is dynamic is not explicitly addressed in the changemanagement literature and this may be the result of researchers restrictingtheir methodologies

This is now the subject of further research using longitudinal case studiesand data collection methods designed to track changes in TMC and the factorsthat influence these changes

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

Performancemeasurement

initiatives

1309

References

Beer M Eisenstat RA and Spector B (1990) Harvard Business Review NovemberDecemberpp 158-66

Bierbusse P and Siesfeld T (1997) `Measures that matterrsquorsquo Journal of Strategic PerformanceMeasurement Vol 1 No 2 pp 6-11

Bititci U Carrie A and Turner T (1998) `Diagnosing the integrity of your performancemeasurement systemrsquorsquo Control April pp 9-13

Bitton M (1990) `MeAcircthode de conception et drsquoimplantation de systeAacutemes de measure deperformances pour organisations industriellesrsquorsquo theAacutese drsquo automatique UniversiteAcirc deBordeaux I Bordeaux

Bourne MCS Mills JF Bicheno J Hamblin DJ Wilcox M Neely AD and Platts KW(1999) ` Performance measurement system design testing a process approach inmanufacturing companiesrsquorsquo International Journal of Business Performance MeasurementVol 1 No 2 pp 154-70

Bourne MCS Mills JF Richards AH Neely AD Platts KW Gregory MJ Barnes DLBicheno JR Gill R Hamblin DJ Rowbotham F and Wilcox M (1997) `Manufacturingstrategy and performance measurement research designrsquorsquo Manufacturing Systems Groupworking paper University of Cambridge Cambridge

Bourne MCS Mills JF Wilcox M Neely AD and Platts KW (2000) `Designingimplementing and updating performance measurement systemsrsquorsquo International Journal ofProduction amp Operations Management Vol 20 No 7 pp 754-71

Burnes B (1997) ` Organizational choice and organizational changersquorsquo Management DecisionVol 35 No 10 pp 753-9

Business Intelligence (2000) Business Intelligence Conference chaired by Kaplan RS andNorton DP 3-5 May London

Dawson P (1996) ` Beyond conventional change models a processual perspectiversquorsquo Asia PacificJournal of Human Resources Vol 34 No 2 pp 57-71

Deming WE (1986) Out of the Crisis Quality Productivity and Competitive Position CambridgeUniversity Press Cambridge

Dixon JR Nanni AJ and Vollmann TE (1991) `An instrument for investigating the matchbetween manufacturing strategy and performance measuresrsquorsquo working paper BostonUniversity Boston MA

Duck JP (1993) `Managing change the art of balancingrsquorsquo Harvard Business Review NovemberDecember

Eccles A (1994) Succeeding with Change Implementing Action-driven Strategies McGraw-HillLondon

Fitzgerald L Johnston R Brignall TJ Silvestro R and Voss C (1991) PerformanceMeasurement in Service Businesses The Chartered Institute of Management AccountantsLondon

Frizelle GDFM (1991) ` Deriving a methodology for implementing CAPM systemsrsquorsquoInternational Journal of Production Management Vol 11 No 7 pp 6-26

Glaser BG and Straus AL (1967) The Discovery of Grounded Theory Strategies forQualitative Research Aldine Chicago IL

Goss T Pascale R and Athos A (1993) ` The reinvention roller coaster risking the present fora powerful futurersquorsquo Harvard Business Review NovemberDecember

Hacker ME and Brotherton PA (1998) ` Designing and installing effective performancemeasurement systemsrsquorsquo IIIE Solutions Vol 30 No 8 pp 18-23

Hudson M Stuart PA and Bourne MCS (2001) ` Theory and practice in SME performancemeasurement systemsrsquorsquo International Journal of Operations amp Production ManagementVol 21 No 8 pp 1095-115

IJOPM2211

1310

Kaplan RS and Norton DP (1992) `The balanced scorecard ndash measures that drive performancersquorsquoHarvard Business Review JanuaryFebruary pp 71-9

Kaplan RS and Norton DP (1993) ` Putting the balanced scorecard to workrsquorsquo Harvard BusinessReview SeptemberOctober pp 134-47

Kaplan RS and Norton DP (1996) ` Using the balanced scorecard as a strategic managementsystemrsquorsquo Harvard Business Review JanuaryFebruary pp 75-85

Kaplan RS and Norton DP (2000) The Strategy Focused Organization How BalancedScorecard Companies Thrive in the New Business Environment Harvard Business SchoolPress Boston MA

Keegan DP Eiler RG and Jones CR (1989) `Are your performance measures obsoletersquorsquoManagement Accounting June pp 45-50

Kotter JP (1995) ` Leading change why transformation efforts failrsquorsquo Harvard Business ReviewVol 73 No 2 pp 59-67

Kotter JP (1996) Leading Change Harvard Business School Press Boston MA

Krause O and Mertins K (1999) ` Performance managementrsquorsquo in Mertins K Krause O andSchallock (Eds) Global Production Management Proceedings of the IFIP WG57International Conference on Advances in Production Management Systems September

Lewin K (1947) ` Frontiers in group dynamicsrsquorsquo Human Relations Vol 1 No 1 pp 5-47

Lewy and Du Mee (1998) ` The ten commandments of balanced scorecard implementationrsquorsquoManagement Control and Accounting April

Lynch RL and Cross KF (1991) Measure up ndash The Essential Guide to Measuring BusinessPerformance Mandarin London

McCunn P (1998) `The balanced scorecard the eleventh commandmentrsquorsquo ManagementAccounting December pp 34-6

Meekings A (1995) ` Unlocking the potential of performance measurement a guide to practicalimplementationrsquorsquo Public Money amp Management October-December pp 1-8

Mintzberg H (1973) The Nature of Managerial Work Harper amp Row New York NY

Neely AD Adams C and Kennerley M (2002) The Performance Prism The Scorecard forMeasuring and Managing Business Success FT Prentice-Hall London

Neely AD Mills JF Gregory MJ Richards AH Platts KW and Bourne MCS (1996)Getting the Measure of Your Business Findlay London

Neely AD Richards AH Mills JF Platts KW and Bourne MCS (1997) `Designingperformance measures a structured approachrsquorsquo International Journal of Operations ampProduction Management Vol 17 No 11 pp 1131-52

Pettigrew A Whipp R and Rosenfield R (1989) ` Competitiveness and the management ofstrategic change processesrsquorsquo in Francis A and Tharakan PKM (Eds) TheCompetitiveness of European Industry Country Policies and Company StrategiesRoutledge London

Platts KW (1994) ` Characteristics of methodologies for manufacturing strategy formulationrsquorsquoComputer Integrated Manufacturing Systems Vol 7 No 2 pp 93-9

Pugh D (1978) `Understanding and managing organisational changersquorsquo London Business SchoolJournal Vol 3 No 2 pp 29-34

Schneiderman A (1999) `Why balanced scorecards failrsquorsquo Journal of Strategic PerformanceMeasurement special edition pp 6-11

Thompson M (1997) ` Getting the measure of your business the experience of James WalkerrsquorsquoPresentation to the local branch of the Institute of Management 12 NovemberWhitehaven

Yin RK (1994) Case Study Research Design and Methods 2nd ed Sage PublicationsThousand Oaks CA