sustaining healthcare excellence through performance measurement

This article was downloaded by: [Northeastern University]On: 23 November 2014, At: 15:57Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Total Quality Management & Business ExcellencePublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/ctqm20

Sustaining healthcare excellence throughperformance measurementGopal Kanji a & Patrícia Moura e Sá ba Kanji Quality Culture , Sheffield Science & Technology Park, Sheffield, UKb Faculdade de Economia , Universidade de Coimbra , Coimbra, PortugalPublished online: 25 Aug 2010.

To cite this article: Gopal Kanji & Patrícia Moura e Sá (2003) Sustaining healthcare excellence through performancemeasurement, Total Quality Management & Business Excellence, 14:3, 269-289, DOI: 10.1080/1478336032000046607

To link to this article: http://dx.doi.org/10.1080/1478336032000046607

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/ctqm20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/1478336032000046607

http://dx.doi.org/10.1080/1478336032000046607

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

TOTAL QUALITY MANAGEMENT, VOL. 14, NO. 3, 2003, 269–289

Sustaining healthcare excellence throughperformance measurement

G K. K1 & P M S2

1Kanji Quality Culture, Sheffield Science & Technology Park, Sheffield UK & 2Faculdade deEconomia da Universidade de Coimbra, Coimbra, Portugal

The healthcare sector is one of the fastest growing areas of the economy of most developedcountries. Governments (and taxpayers) invest in it increasingly larger amounts of money, eitherdirectly or indirectly, and expect in return high quality services. The reality, however, is oftendifferent: long waiting times, inefficiency, low productivity, stressed medical staff and less thansatisfied patients. Over the last decade, total quality management (TQM) has emerged as onepotential solution to improve the efficiency and effectiveness of healthcare provision and, ultimately,to lead to healthy communities. Despite the enthusiasm raised by the potential benefits of TQMimplementation, the fact is that many initiatives have not delivered the promised results. Reasons forfailure are to be essentially found in the insufficient support of health professionals, the lack ofleadership commitment and the tendency to look at TQM in isolation rather than putting it at thecore of the institution’s strategy. The system of performance measurement we present in this paper isexpected to overcome some of these deficiencies, contributing to sustaining the TQM efforts and, thus,to achieving organizational excellence. The main potential of the system comes from the explicit andactive involvement of all stakeholders and from the holistic and integrated approach it forces.

Introduction

Healthcare is responsible for a considerable proportion of public expenses. Even in caseswhere the private sector is strong, public investment is still very significant. Therefore, usingpublic money effectively and efficiently is of particular importance. However, the need forproviding good value for money must be complemented with (and sometimes balancedagainst) equity concerns and the need to ensure all citizens have access to quality healthcare.

Whether citizens and users pay directly or indirectly (thorough their insurance plans ortaxes) for the services they receive, they have the right and the responsibility to demand highstandards of healthcare provision.

Information on performance is essential to support most decisions. Within the principleof choice, service users are entitled to know the standards of service they should expect. AlsoGPs and other healthcare purchasers and suppliers need information on a variety of indicators.Additionally, to make informed funding and resource allocation decisions, Governmentsmust assess how well the different healthcare providers are doing in relation to a series ofcriteria they feel are important in order to assure public interest.

Correspondence: G.K. Kanji, Kanji Quality Culture, Sheffield Science & Technology Park, Copper Building,Unit 13, Arundel Street, Sheffield S1 2NS, UK. E-mail: [email protected]

ISSN 1478-3363 print/ISSN 1478-3371 online/03/030269-21 © 2003 Taylor & Francis LtdDOI: 10.1080/1478336032000046607

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014

270 G. K. KANJI & P. M. E. SA

Clearly, the precise characteristics of individual performance measurement systems willvary from country to country and from institution to institution. Yet, it is the authors’ beliefthat the system should be based on a set of common performance drivers that correspond toa core of TQM principles and concepts, which are adequately covered in Kanji’s PyramidModel (Kanji, 1998). Moreover, the system should reflect the different views and prioritiesassigned by the various healthcare stakeholders. Consequently, the proposed performancemeasurement system captures and uses the feedback provided by the groups that have amajor interest in the healthcare system. With this purpose, in addition to the internalassessment made by the health institution’s leaders and staff members, a number of BusinessScorecards (Kanji & Sa, 2002) are developed to measure the organization’s performanceaccording to the external stakeholders’ views.

This paper characterizes a typical healthcare delivery system and describes the pressuresfor change felt by many healthcare institutions as well as the benefits of TQM in addressingsome of the new challenges. Next, the reasons for TQM failure are reviewed and the role ofa performance system in sustaining successful TQM efforts is analysed. An innovativeperformance measurement system is then presented that can enhance organizational excel-lence. Finally, some indications are given of how a healthcare organization can implementthe system to measure (and improve) its level of performance. It is expected that, bycombining and integrating the assessment of the various stakeholders and by providing aholistic view of the organization, the system can guide a healthcare institution in its quest fororganizational excellence.

The healthcare system: pressures for change and TQM implementation

The healthcare system is composed of a complex net of entities, activities and processes—atthe core of which, inevitably, are the clinical processes—and involves a wide range ofparticipants (Stahr et al., 2000), with each of these aspects bringing to the system a differentset of needs, priorities and evaluation criteria.

Figure 1 summarizes the main (potential) stakeholders of a typical healthcareorganization.

These stakeholders interact with each other at different stages of the healthcare deliveryprocess.

As with any other organization, the typical healthcare institution embraces a variety ofresources (human, material and knowledge), which are used in a series of processes thatultimately aim to improve the medical condition of the patient and contribute to healthiercommunities (Rivers & Boe, 1999).

As depicted in Fig. 2, the hospital is usually at the centre of the healthcare deliverysystem. At the top, establishing general guidelines and setting some constraints and prioritiesis the Healthcare Board, whose actions are likely to be influenced by overall funding andsocial policy issues. Within each hospital (regarded as the key healthcare provider unit), theManagement Group (Board) develops its own Business Plan, defining the goals for theinstitution as a whole and affecting the resources for each division/project/activity. In theirturn, each Clinical Division normally defines its individual procedures and more specificpolicies. As expected, clinical divisions are the focus of healthcare provision. It is here thatinterpersonal interactions between medical staff and service users take place and that technicalskills and knowledge are used to the benefit of the patient. However, for the appropriatefunctioning of the clinical divisions, the role of the Supporting Directorates (comprisingmanagement and support processes) is essential.

On the inputs side, we typically have the patients (with unmet needs related to their

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014

SUSTAINING HEALTHCARE EXCELLENCE 271

Figure 1. Key stakeholders of a typical healthcare organization.

Figure 2. Description of a typical healthcare delivery system.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


medical conditions), upon which the hospital processes act hopefully to meet or improvetheir state of health. Indirectly, especially through preventive and educative actions, thehospital produces healthy communities. Additionally, the system is the recipient of otherstakeholders’ influences, such as GPs, Health Authorities, pharmaceutical suppliers anduniversities (see also Fig. 1).

The overall performance of the system is periodically assessed and reviewed. Convention-ally, the assessment involves patient satisfaction surveys, staff questionnaires and the computa-tion of clinical effect and cycle time indicators. However, as argued later, there is littleintegration between these performance evaluation components.

Obviously, the particular characteristics of each healthcare system naturally differ fromcountry to country and would be impossible to describe in this paper. Therefore, theresponses developed by each system to cope with the new environmental pressures andchallenges will differ.

In any case, some trends are consistently identified in the literature (Kunst & Lemmink,2000; Lim & Tang, 2000; Lin & Clausing, 1995; Rivers & Boe, 1999; Short & Rahim, 1995;Theodorakiogleu & Tsiotras, 2000):

Ω escalating costs and financial constraints;Ω population senescence;Ω medical breakthroughs;Ω rising standards of living and education and high expectations of consumers;Ω change from a seller-oriented to a buyer-oriented culture;Ω increasing competition and alternative mechanisms of healthcare delivery;Ω governmental regulations and monitoring by public and private groups;Ω increased information availability;Ω difficulty in recruiting and retaining good healthcare professionals.

The pressures listed above made health providers increasingly aware of the need to becomemore customer (patient)-oriented, control costs, improve resources efficiency and effec-tiveness, and publicly show the results of the improvement efforts.

In the pursuit of this new orientation, TQM undoubtedly raised considerable interest,since—through its focus on the customer and its philosophy of continuous improvement—itpromises to increase productivity, satisfaction, profitability and long-term survival. It wasquickly regarded as a way to improve managerial practices, securing the best value for moneyand delivering the best possible service to patients (Debrah, 1994).

Although successful stories of TQM implementation in the health sector are reported inthe literature, often the TQM efforts did not pay-off. Different reasons explain the failures(see Table 1 for a summary). First, the introduction of TQM in most hospitals faces anadverse culture, which is characteristic of large professional bureaucracies (sometimes furthercomplicated by public sector features that leave managers limited room to manage in areassuch as human resource policies and performance-based rewards). Second, too often theprocess of implementation was not properly thought through. There was the naive assumptionthat people would automatically adhere to the new philosophy and the results would appearimmediately. In the face of a lack of immediate tangible results, disbelief and distrustsuddenly surfaced. However, perhaps the major cause of failure was a lack of consistency,alignment and integration. Without leadership commitment and a sense of direction, TQMinitiatives had a minimal impact. At best, they resulted in the alteration of formal structuresand written procedures, but did not really challenge internal practices, especially those relatedto core activities and processes.

The difficulties probably begin with finding a common and complete definition of quality

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Table 1. Obstacles for TQM implementation in healthcare organizations.(Adinolfi, 2002; Lim& Tang, 2000; Short & Rahim, 1995; Theodorakiogleu & Tsiotras, 2000; Zaboda et al.,

1998)

Ω Lack of senior management commitmentΩ Fuzzy missionsΩ Short-term orientation of administratorsΩ Poor communication of the strategies and goals of the organizationΩ Tensions and misunderstanding between management and medical professionalsΩ Resistance by physicians, nurses and other medical staff

Ω Organizational structure strongly departmentalized, hierarchical and authoritativeΩ Strong inter-functional barriersΩ Professional autonomy and occupational subculturesΩ Compensation systemΩ Healthcare organizations are inward-lookingΩ Failure of healthcare providers in understanding the voice of the customerΩ Lack of appropriate budgetary control

in the health sector. As Zabada et al. (1998) note, ‘the existence of many participants withdifferent or even opposing interests . . . makes it difficult to define healthcare quality’. Thecomplex nature of healthcare practice (including the direct involvement of the patient(customer) in many processes) and ethical considerations add to the difficulty.

One of the most well-known attempts to define quality in healthcare was made byDonabedian (1980, cited in Aggarwal & Zairi, 1997) who suggested that it comprises the‘goodness’ of three interrelated components: (a) technical care, (b) interpersonal relation-ships, and (c) amenities of care.

In the UK, the NHS provides the following definition of quality:

The appropriateness and effectiveness of the clinical care delivered to patients andthe manner to which it is delivered.

A usual distinction is made between technical and functional quality (Lam, 1997). Accordingto this classification, technical quality refers to the accuracy of the medical diagnosis andprocedures and the conformance to professional specifications, whereas functional qualityrelates to the way the healthcare service is delivered to the patients (i.e. the interpersonalaspect of care).

As emphasized earlier, quality in both the clinical and support processes (administrative,personnel, etc) is required for an adequate service provision.

Performance measurement and TQM

The benefits of performance measurement for any organization are not unknown (seeTable 2).

Good management requires reliable and timely information on facts. In spite of the

Table 2. Roles of a performance measurement system

Ω Check progress towards the established goalsΩ Provide accountability mechanismsΩ Support future resource allocation decisionsΩ Communicate goals and priorities and act as a motivation toolΩ Drive improvement

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


unquestionable truth of this statement, there is a prevalent tendency to rely on intuition andopinions and to assume that the organization is ‘doing the right things right’ without anysupport from facts.

Performance measurement provides hospitals with ‘hard evidence about existing prac-tices, values, beliefs, and assumptions’ (Lim et al., 1999) and ‘enables hospitals to develop asystematic means of identifying shortfalls and enhance their future performance’ (Lim &Tang, 2000).

Looking at the purposes of performance measurement its importance in sustaining TQMbecomes apparent. The system of performance measurement, if properly designed andimplemented, will focus organizational efforts onto a common purpose by directing everyone’sattention into a set of key goals and objectives. Furthermore, it will constitute the basis toencourage the appropriate behaviours, assess individual and team performance and rewardsignificant contributions towards quality. Finally, by measuring performance gaps andhighlighting problems, the system will pinpoint areas where improvement is necessary andurgent, while providing a methodology to evaluate progress towards organizational excellence.

However, to fulfil these roles successfully, a good system of performance measurementshould exhibit the following characteristics (Kanji & Sa, 2001):

Ω measure performance from a multi- and interrelated perspective,Ω be linked to the organization’s values and strategy,Ω be based on the critical success factors or performance drivers,Ω be valid, reliable and easy to use,Ω enable comparisons to be made and progress to be monitored,Ω be linked to the rewards’ system and encourage the appropriate behaviours,Ω highlight improvement opportunities and suggest some improvement strategies.

The system proposed by the authors and described next is expected to constitute an importantadvancement towards a holistic, comprehensive and relevant approach to performancemeasurement.

Kanji’s Business Excellence Measurement System (KBEMS)

Kanji’s Business Excellence Measurement System (KBEMS) is grounded on Critical SuccessFactors (CSFs) which, by definition, are ’the limited number of areas in which results, ifsatisfactory, will ensure successful performance for the organization’ (Rockart, 1979, cited inLeidecker & Bruno, 1984). Therefore, the CSFs correspond to the drivers of performance.These CSFs are embedded in Kanji’s Pyramid Model (Kanji, 1998), which is the conceptualmodel of Business Excellence adopted. Based on its general principles and core concepts,two structural models were developed: Kanji’s Business Excellence Model (KBEM) (Kanji,1998) and Kanji’s Business Scorecard (KBS) (Kanji & Sa, 2002). The first (KBEM) isdedicated to the measurement of performance from the internal stakeholders’ perspective,whereas the later (KBS) assesses performance from the external stakeholders’ point of view.Internal and external scores are finally combined to calculate the final organizationalperformance excellence index (OPI).

The final OPI—which gives an aggregate measure of the excellence of the organizationin managing all the CSFs—is simply the average between the scores of PerformanceExcellence A and Performance Excellence B, i.e.

OPIóAòB

2î10

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Figure 3. Performance excellence measurement process.

Since, potentially, the KBS is applied to various stakeholders, the organization is likely tohave n different scores for Part B (for customers, suppliers, Government, etc). In this case,the B score to be included in the formula above is a mean of the scores obtained, i.e.

Bó;

n

ió1Bi

n

Figure 3 summarizes the process described above.Thus, the system of performance measurement proposed (see Fig. 6 later) comprises

two parts—part A and part B—respectively directed towards the assessment of internal andexternal stakeholders.

Part A closely follows Kanji’s Business Excellence Model (Kanji, 1998), either in its fullor condensed version (as presented in Fig. 4).

As depicted in Fig. 4, Leadership is the prime aspect of Part A, meaning that leaders

Figure 4. Kanji’s Business Excellence Model (KBEM)—condensed model (based on Kanji, 1998).

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Figure 5. Kanji Business Scorecard (Kanji & Sa, 2002).

Figure 6. Kanji’s Business Excellence Measurement System (KBEMS) (Kanji & Sa, 2001).

are the most important driving force for quality improvement and Business Excellence. Theirattitude must promote four principles: delight the customer, management by fact, people-based management and continuous improvement. Each principle operates in one essentialcore concept, namely Customer Focus, Process Improvement, People Performance andContinuous Improvement Culture, respectively.

Part B constitutes a Business Scorecard (Kanji & Sa, 2002) (see Fig. 5) to be customizedaccording to the various external stakeholders—customers, suppliers, government, etc—under consideration.

In Part B, Organizational Values are the prime aspect of Process Excellence, Organiza-tional Learning and Stakeholders’ Delight. Effective management of these critical successfactors will lead to a high Performance Excellence Index in Part B.

As emphasized earlier, the same critical success factors (or performance drivers) underlieboth parts of KBEMS (Fig. 6). Only the level of detail and focus differs to reflect thepriorities, needs and expertise of different stakeholders.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


It should be noted that Parts A and B must be applied simultaneously, since they forma single and complementary view of organizational performance.

Each of the criteria that comprise the system is described in detail elsewhere (see Kanji,2002a). Therefore, our focus in the next section is on the way KBEMS can be applied in thehealthcare sector.

Application of KBEMS in a healthcare organization: guidelines to a systematicimplementation

The application of KBEMS in the health sector is currently taking its first steps. A pioneeringexample is reported by Adinolfi (2002). Most of the work conducted until now refers to theinterpretation of the core TQM principles and concepts (i.e. the key performance drivers) inhealthcare. Besides providing a few guidelines on how to interpret some of the KBEMScriteria, our aim is to explain briefly the steps involved in implementing KBEMS in ahealthcare institution.

The general approach to performance measurement embedded in KBEMS is depictedin Fig. 7.

For illustration purposes, we will describe each of these steps with reference to HospitalX, which is a public hospital located in England.

Step 1. Adopt KBEMS and consider who your stakeholders are

As shown in Fig. 7, the starting point is to adopt KBEMS measurement models (i.e. KBEMand KBS) which, as explained previously, include the set of criteria against which theorganization’s performance will be measured (see Fig. 6).

These criteria should reflect the Critical Success Factors (CSFs). Some CSFs might beindustry- and organization-specific; however, from previous applications in different con-texts—education (Kanji & Tambi, 2002), supply chain management (Kanji & Wong, 2002a),retail banking (Kanji & Wong, 2002b) and local government (Sa, 2002)—there is widesupport for the universality of the drivers proposed. In any case, if necessary, KBEMS isflexible enough to accommodate minimal changes in the CSFs to fit better the particularcontext of the healthcare institution under consideration.

Since, in the assessment exercise, feedback is collected from different groups, theorganization must first consider who its most important stakeholders are. A list of potentialstakeholders is provided in Fig. 1.

In the case of Hospital X, as a public healthcare provider, four main stakeholders wereidentified:

Ω physicians, nurses and other staff members;Ω patients;Ω health authorities; andΩ suppliers.

Step 2. Design the assessment questionnaires

Since each performance criterion corresponds to a concept that cannot be directly measured,it must be translated into a set of indicators that are then converted into items of aquestionnaire. Basically, each question addresses a management practice the organization isexpected to have in place if it is adopting the principles and concepts underlying KBEMS.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Figure 7. Performance measurement approach (Kanji, 2002b).

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Separate questionnaires need to be created for Parts A and B of KBEMS.Accordingly, internal stakeholders (i.e. medical professionals, administrators and other

staff ) will make the organization’s assessment in each of the areas that form KBEM (PartA), which in the condensed version of the model (see Fig. 6) are:

Ω leadership;Ω delight the customer;Ω management by fact;Ω people-based management;Ω continuous improvement;Ω customer focus;Ω process improvement;Ω people performance;Ω continuous improvement culture; andΩ performance excellence A.

Respondents indicate the extent to which the organization practices each suggested aspecton a ten-point scale. An extract of the internal questionnaire developed for hospital X ispresented in Fig. 8.

Leadership

The extent to which:

Figure 8. Extract of the internal stakeholders’ questionnaire for Hospital X.

Additionally, questionnaires are designed for each external stakeholder category (i.e. patients,GPs, health authorities and the local community) according to the criteria that comprise PartB of KBEMS (see Fig. 6):

Ω organizational values;Ω process excellence;Ω organizational learning;Ω delight the stakeholders; andΩ performance excellence B.

Examples are given next of the questions used by Hospital X to measure external stakeholdersperceptions (see Figs 9–11).

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Delight the Stakeholders


Figure 9. Extract of the patients’ questionnaire for Hospital X.

Organizational Learning


Figure 10. Extract of the health authorities’ questionnaire for Hospital X.

Scorecard Excellence


Figure 11. Extract of the suppliers’ questionnaire for Hospital X.

Step 3. Data collection

The questionnaires prepared in the previous step are administered to the various stakeholders.It is important to pre-test the questionnaires and ensure that there are no wording problems

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Table 3. Spreadsheet portion for raw data introduction

Area 3. Raw Questionnaire Data—Part A

Leadership Delight the customer

Q11 Q12 Q13 Q14 Q15 Q21 Q22 Q31

9 7 8 9 9 9 5 66 5 7 3 5 7 4 49 6 7 7 8 6 5 59 8 8 7 8 8 6 58 5 6 4 5

and the most relevant issues are properly addressed. Furthermore, the administration processmust be carefully planned to obtain a good response rate and a truthful and reliable pictureof the organization’s performance. Stakeholders need to understand the importance of theassessment exercise, have confidence that it will produce results and feel comfortableexpressing their genuine opinions.

In order to generate most reliable and significant results, about 250 questionnairesshould ideally be collected from each stakeholder, although it is possible to run the assessmentprocess with less than that number.

Step 4. Prepare data for analysis

At this stage, data collected in the previous step are introduced in a spreadsheet. This is astraightforward procedure aimed at preparing the information for analysis according to therequirements of K&W management software (see next section).

If the organization is using this package, it will be provided with a pre-designed Excelspreadsheet divided into different areas. Some of these areas are dedicated to the introductionof the raw data collected from the various organizational stakeholders (see Table 3).

Step 5. Data analysis, K&W software and key performance results

Data analysis should start as soon as possible in order to enhance the timeliness andusefulness of the measurements in supporting decision-making and improvement.

Stakeholders’ responses to the questionnaires are analysed according to the principles ofstructural equation modelling, using the K&W software package (Kanji, 2001).

Structural equation modelling is particularly suitable for analysing situations where thereis interdependence between the models’ dimensions, such as happens with the performanceaspects that form KBEMS. Thus, the various relationships are estimated simultaneouslyaccording to the whole data set and the final parameters and scores associated with KBEMSwill reflect how well the organization is managing the totality of the CSFs.

K&W software uses the Partial Least Squares (PLS) method to estimate simultaneouslythe weights of the constructs (i.e. the performance criteria) that form KBEMS. It calculatesthese weights in a way that maximizes the goodness of fit of the models and, thus, thesystem’s ability to explain performance excellence as the ultimate endogenous variable.

The dedicated software provides a variety of outputs. Essentially, information is givenon the scores obtained in each criterion as well as the performance excellence scores for PartsA and B. By combining these elements, the final performance excellence index is also

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


calculated. Furthermore, the structural coefficients, which indicate the strength of relation-ships among the performance criteria, are equally provided, along with other information onthe quality of the measurement scales and the overall explanatory power of the model.

Data collected from 200 patients attending Hospital X in the first six months of 2002were introduced and analysed with the K&W software. Next, examples of the main outputsare presented.

Scale quality indicators

If the indexes produced by the program are to be meaningful and truthful, the indicatorsused to measure each criterion should exhibit some characteristics in terms of validity andreliability (Nunnally, 1967; Flynn & Saladin, 2001).

The software provides two main indicators of the quality of the scales: Cronbach alphacoefficients (see Table 4) and outer coefficients (see Table 5).

Cronbach alpha gives a good indication of the quality of the measurement scalesemployed to assess each criterion of the performance system. As a general reference, 0.7 is agood value, demonstrating that the scale is reliable. However, especially when the question-

Table 4. Reliability scores (Cronbach-alpha) for Part B—patients

Alphas

Process Delight PerformanceOrg. Values Excellence Org. Learning Stakeholders Excellence B

0.754 0.821 0.878 0.724 0.911

Table 5. Outer coefficients for Part B—patients

Outer Coefficients

Delight PerformanceOrg. Values Process Excellence Org. Learning Stakeholders Excellence B

Q111 0.433111 0 0 0 0Q112 0.102842 0 0 0 0Q113 0.528478 0 0 0 0Q121 0 0.191697 0 0 0Q122 0 0.260787 0 0 0Q123 0 0.315147 0 0 0Q124 0 0.402996 0 0 0Q131 0 0 0.261328 0 0Q132 0 0 0.410696 0 0Q133 0 0 0.172161 0 0Q141 0 0 0 0.478667 0Q142 0 0 0 0.274778 0Q143 0 0 0 0.046378 0Q144 0 0 0 0.321695 0Q151 0 0 0 0 0.226636Q152 0 0 0 0 0.239579Q153 0 0 0 0 0.202262Q154 0 0 0 0 0.201337

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


naires have been used for only a short period of time, smaller values are acceptable. In anycase, alpha-values bellow 0.6 should cause some concern and may require changes in themeasurement instrument, such as removing or adding a new question.

As observed in Table 4, in the Hospital X example, all the scales have a good reliability.Sometimes the scenario is not so promising. The quality of the scales with lower Cronbach-alphas may be improved by removing a very small number of questions that, rather thancontributing to explaining the associated criterion, may be causing problems. Outer coeffi-cients give important indications in this regard.

The outer coefficients in Table 5 show how each question is loaded into the respectivedimension (criterion). It reveals that some variables contribute more than others to therespective construct. Question 143 seems to have a relatively weak association with Delightthe Stakeholders. In this particular case, if we recall the items that form the scale (see Fig.7) the low outer coefficient essentially indicates that the hospital does not handle patients’complaints effectively. Formal procedures to collect suggestions and complaints may be inplace, but patients feel that the information is not in fact used to improve the servicesprovided.

Association between the performance criteria

The performance drivers are associated with a final measure of excellence either directly orindirectly through a series of paths. The inner coefficients (see Table 6) show the strength ofthese causality links. They represent the marginal impact that each criterion has on perfor-mance excellence.

The relationship between the various criteria is expected to be positive, emphasizing thereinforcing nature of the TQM principles and core concepts and the importance of imple-menting them simultaneously. Clearly, some relationships are likely to be stronger than othersbut any negative values must be looked at with suspicion.

Table 6 results can easily be depicted in the more familiar form of a path diagram (seeFig. 12).

For Hospital X, data collected from its patients show that organizational values are(prime factor) in fact strongly associated with the other critical dimensions. Additionally, allcoefficients are positive and significant, strongly supporting the holistic and synergic natureof the model. However, at the operational level, Hospital X is not properly managing the linkbetween process excellence and performance. A strong effort should be put in improving thedisclosure of key process indicators to make the association with performance excellencemore visible from the patients’ point of view.

Table 6. Inner coefficients for Part B—patients

Inner coefficients

Process Delight PerformanceOrg. Values Excellence Org. Learning Stakeholders Excellence B

Org. values 0 0 0 0 0Process Excellence 0.83 0 0 0 0Org. Learning 0.75 0 0 0 0Delight Stakeholders 0.67 0 0 0 0Performance Exc. B 0 0.18 0.37 0.45 0

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Figure 12. Path diagram with the structural coefficients for Part B—patients.

Latent variable (criterion) scores

Up to 100 points can be obtained in each criterion. The final Performance Excellence Indices(for Parts A and B) will reflect the simultaneous effect of all the relationships estimated inthe model.

Figure 13 shows the scores obtained in the Patients’ Scorecard (Part B1 of KBEMS) forthe first semester of 2002.

Looking at Fig. 13, some observations can be made.

Ω The final performance excellence index for Part B is above 50%, but, accordingto patients’ evaluation, the hospital is not performing equally well in the variousdimensions.

Ω The fact that the score of the prime factor of the model i.e. organizational values, islow (51%) is particularly worrying, suggesting that further efforts are required incommunicating to the patients what the hospital stands for, demonstrating that ethical

Figure 13. Latent variable scores for the criteria in Part B—patients.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


principles are clearly respected and that strategies and policies are aligned to thevalues.

Ω Process excellence (43%) is also pushing down the performance excellence index. Itindicates that although patients recognize that healthcare is provided with technicalmastery and professionalism, they feel that services do not work as smoothly asnecessary, with long waiting times and some cancellations. There is also a sense thatdifferent departments do not collaborate sufficiently to the benefit of the patient.

Ω These areas are, to some extent, compensated for by the hospital’s visible capacity tolearn. Patients feel that the organization has a culture of continuous improvement andmeets the ever-changing demands of healthcare.

Ω Owing to a strong patient orientation, the hospital achieves relatively good scores inStakeholders’ Delight (58%). Patients feel that their clinical needs are taken intoaccount and that proper information is given regarding their state of health andnecessary treatments. Yet patients are still suspicious regarding the effectiveness ofsuggestions and complaining schemes.

Identical diagrams were produced for the other stakeholders of Hospital X, i.e. healthauthorities (Part B2), suppliers (Part B3) and internal stakeholders (Part A).

Figure 14 shows the results achieved by Hospital X in Part A of KBEMS.As depicted in Fig. 14, the internal stakeholders assessment is quite positive, leading to

a performance excellence index for Part A of 69%. It is possible to point out the main areasof strength and the principal weaknesses.

Ω On the positive side, the hospital is developing an outward-focus, making everyoneaware of the need to satisfy its internal and external customers. Care is provided withcompassion, courtesy and regard for the patient. Interaction between staff membersand service users is encouraged and there are methods in place to determine currentand future needs and expectations.

Figure 14. Latent variable scores for the criteria in Part A—internal stakeholders.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Table 7. Performance excellence indexes

InternalStakeholders External Stakeholders

Final PerformancePart A Patients—B1 Health Authorities—B2 Suppliers—B3 Excellence Index

69.6 53.2 65.3 47.8 62.5

Ω In addition, there is a continuous improvement culture with the hospital willing tolearn from mistakes and to avoid their repetition in the future. Benchmarking is widelypractised and changes are made in response to patient needs and clinical breakthroughs.

Ω On the other hand, management by fact is not yet fully implemented. Despite theamount of information collected, there is still a tendency to make decisions based onintuition and management assumptions. Key processes are often not identified andcontrolled. As a result, there is not enough coordination between interdependent tasks.

Ω Staff members do not seem very pleased with the human resource managementpolicies. Not enough feedback is provided to staff on their performance and, as oftenhappens in the public sector, the system of rewards is not tied to the quality strategyand objectives.

Ω In order to improve in these areas, the leadership role is essential. It is the leaders’responsibility to align the various functions and policies providing the hospital with asense of purpose and a clear direction.

Internal and external measurements are then combined to calculate the final performanceexcellence index.

For the illustrative example, the scores obtained for the various stakeholders are shownin Table 7.

The final performance excellence index was calculated according to the formula presentedearlier, i.e.

PEIóPEAòPEB

2

where

PEAó69.6 PEBó53.2ò65.3ò47.8

3ó55.4

Therefore PEIó62.5The same information is displayed in Fig. 15, where the discrepancy between internal

and external stakeholders’ evaluations is apparent.As shown, the external stakeholders’ evaluation is somehow less optimistic than the

internal assessment. Patients and suppliers in particular point out to some problems thatmay have been underestimated internally.

Overall, the hospital needs to listen more to external stakeholders’ opinions, finding newforms of communication and demonstrating that their feedback is taken into account instrategy formulation and implementation. There is a need to ‘walk the talk’ more, so thattrust and belief in what has been communicated improves.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Figure 15. Performance excellence final indexes.

Step 6. Design improvement strategies and keep the momentum

By knowing how each critical success factor impacts on an aggregate measure of performance,it is possible to analyse where improvement efforts are more likely to have a greater impact.That is precisely the purpose of the last stage of the performance measurement approach.

The Business Excellence Seeker’s approach (Kanji, 2001) allows the forces of excellenceto be identified and indicates how they can be used to drive improvement. Briefly, it involvesthe use of an optimization algorithm for determining which indices to increase and by howmuch to achieve a given target level of excellence. Since organizations usually face someconstraints, the algorithm allows the introduction of upper limits of improvement for one (ormore) critical success factors.

Table 8 illustrates the application of the Excellence Seeker’s approach to delineatestrategies to improve performance excellence from the patients’ perspective.

In this case we set the upper limit of improvement at 75 points. The areas to be targetedare prioritized based on their contribution to performance excellence, given by the respectiveinner coefficients.

The required performance levels for the critical success factors associated with a PEB ofrespectively 60 and 70% are shown in Table 8. A considerable improvement can be obtainedonly by increasing the score of the Delight the Stakeholders dimension, through measuressuch as providing them with better and more reliable information and carefully listen to theirneeds. In order to achieve an index of 70, the hospital also has to improve the way itcommunicates what it stands for and develop consistent strategies. By getting all factors totheir upper limit, it can potentially achieve a PEB index of 78 points.

To keep the momentum, the approach should produce tangible results. This requiresmaking good use of the measurements generated by the system and demonstrating to allstakeholders the progress achieved.

Table 8. Performance Excellence Seeker’s simulation example—patients

Original Target PerformanceIndex Excellence Index Performance

Critical success factor andperformance excellence Upper limit PEB1ó53 60 70 78

Organizational Values 75 51 51 72 75*Process Excellence 75 43 43 43 75*Organizational Learning 75 66 66 75* 75*Delight the Stakeholders 75 58 74 75* 75*

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Finally, note the need to apply and review the whole process on a regular basis, so thatthe organization can identify improvement opportunities, monitor progress over time andanticipate environmental changes.

Conclusions

Rather than a piece by piece/fragmented view of performance, as traditionally happens,Kanji’s Business Excellence Measurement System (KBEMS) achieves a high degree ofintegration and comprehensiveness. Moreover, it makes use of a rigorous methodology thatpermits all the system’s parameters to be estimated simultaneously. Thus, KBEM is holisticand inclusive.

Ω The whole system is driven by leadership and organizational values. Leadership hasbeen consistently regarded as the key element in sustaining TQM and OE. Inhealthcare, its role is also primarily important in introducing changes, creating aculture of improvement, enhancing an outwards-looking attitude and driving out fearsand internal barriers. Moreover, the existence of shared values and meanings providesa focus for the system and a starting point to coordinate the efforts of all stakeholders.Without any doubt, it contributes to the establishment of a cooperative environmentbetween managers and clinicians, thus removing one of the most powerful barriers toimprovement in the health sector.

Ω Internal and external stakeholders are actively involved in the assessment process, thusgiving a more embracing and realistic view of performance. Support for a stakeholders’approach to performance measurement is widely found in the literature ( Jackson,1995; Kaplan, 2001). However, in healthcare there has been some opposition to avery strong customer/user involvement in the system (Zoboda et al., 1998). Legiti-mately, this opposition calls attention to the impossibility (and undesirability) ofconsidering the patient as the sole judge of quality in the health sector. Clearly, thereare aspects of healthcare provision (the technical ones in particular) which the patientis often unable to evaluate fully (Lam, 1997). Moreover, frequently the needs andexpectations of the individual service user must be balanced against the needs of thesociety/community, so that equity, equality, fairness and the wide public interest aretaken into consideration. This recognition cannot, however, underestimate the criticalimportance of customer satisfaction and patient feedback.

Ω KBEMS supports a balanced view of performance measurement. There is a concernto focus the customer assessment on those aspects that patients are especially preparedto judge and where their feedback is invaluable. At the same time, the opinions ofother relevant external stakeholders (the Government, taxpayers, suppliers, the widecommunity, etc) are captured in their respective Scorecards and ultimately integratedwith the internal assessment to obtain an aggregated measure of performance.

Ω By pointing out areas where stakeholders’ evaluations are very divergent, the systemhighlights communication problems and calls attention to the need for informationsharing and cooperation amongst the various stakeholders.

The performance assessment process can take different forms. Although the methodologysuggested is essentially quantitative and questionnaires are the key pieces in measuring theorganization’s performance, it is also possible (and recommended in certain stages) to useother self-assessment techniques, such as interviews and focus groups. This may giveadditional support to the performance measurements obtained, while helping to ensure thatthe questionnaires adequately cover the most relevant aspects and are properly interpreted.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014


Within the system fundamentals, each hospital is encouraged to develop an approachthat better fits its needs and resources in order to find its own path to OrganizationalExcellence.

References

A, P. (2002) Total quality management in public health care: a study of Italian and Irish hospitals,Proceedings of the Seventh World Congress for Total Quality Management—Volume 2, Verona, Italy.

A, A. & Z, M. (1997) The role of total quality management in enabling a primary heath-careorientation, Total Quality Management, 10(8), 347–359.

D, Y. (1994) Evolution and implementation of a quality improvement programme: a case study of twohealthcare organizations, Total Quality Management, 5(3), pp. 11–25.

F, B. & S, B. (2001) Further evidence on the validity of the theoretical models underlying theBaldrige criteria, Journal of Operations Management, 19, pp. 617–652.

J, P. (1995) Reflections on performance measurement in public service organizations. In: P.M. J

(Ed.), Measures for Success in the Public Sector (Public Finance Foundation) pp. 1–18.K, G.K. (1998) Measurement of business excellence, Total Quality Management, 9(7), pp. 633–643.K, G.K. (2001) Forces of excellence in Kanji’s Business Excellence Model, Total Quality Management,

12(2), pp. 259–272.K, G.K. (2002a) Performance measurement systems, Total Quality Management, 13(5), pp. 715–728.K, G.K. (2002b) Business excellence: make it happen, Proceedings of the Seventh World Congress for Total

Quality Management—Volume 1, Verona, Italy.K, G.K. & S, P.M. (2001) Kanji-certification of performance measurement. www.Gopal-Kanji.com.K, G.K. & S, P.M. (2002) Kanji’s Business Scorecard, Total Quality Management, 13(1), pp. 13–27.K, G.K. & T, A. (2002) Business Excellence in Higher Education (UK, Kingsham Press).K, G.K. & W, A. (2002a) Supply Chain Management Excellence (UK, Kingsham Press).K, G.K. & W, W. (2002b) Business Excellence and Banking (UK, Wisdom House).K, R. (2001) Strategic performance measurement and management in nonprofit organizations, Nonprofit

Management & Leadership, 11(3), pp. 353–370.K, P. & L, J. (2000) Quality management and business performance in hospitals: a search for

success parameters, Total Quality Management, 11(8), pp. 1123–1133.L, S. (1997) SERVQUAL: a tool for measuring patients’ opinions of hospital service quality in Hong Kong,

Total Quality Management, 8(4), pp. 145–152.L, J. & B, A. (1984) Identifying and using critical success factors, Long Range Planning, 17(1),

pp. 23–32.L, P. & T, N. (2000) The development of a model for total quality healthcare, Managing Service Quality,

10(2), pp. 103–111.L, P., T, N. & J, P. (1999) An innovative framework for health care performance measurement,

Managing Service Quality, 9(6), pp. 423–433.L, B. & C, J. (1995) Total quality management in healthcare: a survey of current practices, Total

Quality Management, 6(1), pp. 69–78.N, J. (1967) Psychometric Theory (McGraw-Hill).R, P. & B, S. (1999) Aligning information systems for effective total quality management implementa-

tion in healthcare organisations, Total Quality Management, 10(2), pp. 281–289.S, P.M. (2002) Organisational excellence in the public sector: with special reference to the Portuguese local

government. PhD Thesis, Sheffield Hallam University.S, P. & R, M. (1995) Total quality management in hospitals, Total Quality Management, 6(3),

pp. 255–263.S, H., B, B. & S, M. (2000) The Excellence Model in the Health Sector: Sharing Good Practice

(Kingsham Press).T, Y. & T, G. (2000) The need for the introduction of quality management into

Greek health care, Total Quality Management, 11(8), pp. 1153–1165.Z, C., R, P. & M, G. (1998) Obstacles to the application of total quality management in

health-care organizations, Total Quality Management, 9(1), pp. 57–66.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

15:

57 2

3 N

ovem

ber

2014

sustaining healthcare excellence through performance measurement

Documents