defining the conditions of a contractors’ past performance ... · defining the conditions of a...

CIB2007-102

Defining The Conditions of a Contractors’ Past Performance Measurement System for

Dutch Construction Clients

Rob P. Geraedts, Jelle Koolwijk

ABSTRACT In 2004 three Dutch ministries stated, in the mission statement ‘Perspective for the Building and Construction Industry’, a great need existed to reform the Dutch building and construction industry. Based on an international study by the national institute PSIBouw it was concluded that the prequalification of contractors based on their past performance could contribute to this mission. To define the conditions of an instrument by which the performances of contractors on project level can be measured, a first study (PaPer Phase I) was conducted. The first conceptual instrument (framework) has been developed based on three parts: a literature survey, an inventory of comparable Dutch and foreign instruments, and a study on the legal conditions of the instrument. This paper describes the conceptual instrument and the underlying principles, choices and difficulties. Keywords Prequalification of contractors, Past Performance, Benchmarking, Process Quality 1. INTRODUCTION Based on an international survey on reform programs in the building and construction industry, it was concluded that the registration and prequalification of contractors based on their (past) performance (PaPer) by Dutch (public) clients, could encourage the Dutch building and construction industry in raising standards such as quality, delivery and

814 CIB World Building Congress 2007

2 CIB World Building Congress 2007 other non-price factors (Ang, 2004). This will contribute to the aims of the mission statement ‘Perspective for the Building and Construction Industry’ which was presented by the three Dutch Ministries of Trade and Industry, Traffic and Civil works and Housing, Spatial Planning and the Environment in November 2003. Delft University of Technology was asked to research the conditions of an instrument by which Dutch public clients can register and pre-qualify contractors based on their past performance and to develop a first conceptual framework of the so called PaPer instrument. This paper describes the underlying principles, choices and difficulties of the conceptual framework.

2. METHODOLOGY

This first step in the development of the PaPer instrument has been guided by representatives of several public clients (General building agency, Roads department, etc). In four workshops spread over a period of four months the literature survey, analysis of comparable instruments and the legal aspects of the systems where reviewed and discussed. This resulted in decisions defining the conditions of the PaPer instrument that has been developed in phase II of the research project. Phase II will be discussed in another paper. The legal issues on this topic where researched by a third party, namely prof. dr. mr. Monica Chao-Duivis (Chao-Duivis et al, 2005). This paper will only refer to some of the conclusions made by Chao-Duivis, because of the specialist nature of this particular part of the project.

3. PERFORMANCE AS ONE OF THOSE “SUITCASE WORDS”

Lebas and Euske (2002) wrote: “The word performance is widely used in all fields of management. In the management control area, terms such as performance management (Euske et al, 1993) measurement, evaluation, or appraisal (e.g. Burns, 1992) are used. Despite the frequency of the use of the word, its precise meaning is rarely explicitly defined by authors even when the main focus of the article or the book is performance (e.g., Baird, 1986; Richard, 1989). Often performance is identified or equated with effectiveness and efficiency (e.g., Neely, et al, 1995; Corvellec, 1994)… In short, performance is one of those “suitecase words” (Bourguignon, 1995) in which everyone places the concepts that suit them, letting the context take care of the definition. Using context to clarify the meaning may help create a basis for understanding and discussion, yet it may engender ambiguous definitions. Ambiguity can be beneficial. Differing interpretations of the same reality may generate interactions that spawn new and creative outcomes.” This statement is the starting point of this paper. Designing a framework for a Past Performance measurement instrument which

CIB World Building Congress 2007 815

Paper, a contractors’ past performance measurement system 3

measures the delivered performance by contractors on a project level, needs to be defined in a very ambiguous way to prevent “the waste of scarce resources”. This is further complicated by the fact that this measured performance will be used in future tendering procedures by public clients, placing the instrument in a complicated juridical context.

4. PERFORMANCE AS ONE OF THOSE “SUITCASE WORDS”

Performance measurement in it self is a broad definition. This paragraph will give a short overview of the possible areas, within or outside organizations, where performance measurement can be applied. At the end of the paragraph the PaPer-instrument will be positioned in the performance measurement arena. With intra-organizational performance measurement we mean the measurement of input, processes and outcomes within ones own organization (Schmitz, et al, 2003). Measures used can range from strategic level, such as market share, annual growth, etc. to everyday operational measures, such as duration or cost of projects, capacity utilization, quality, etc (Schmitz, et al, 2003). Purported aims of using intra-organizational measures and related procedures include co-ordinate and manage organizational activities and inform decisions that are concerned with these activities (Simons, 1999). By doing this the efficiency of the organization can be measured, evaluated and where needed processes can be adjusted to raise efficiency. Another objective of performance measurement can be that of the client, which uses the output of the supplier for its own processes or to sell the product to the market. The so called inter-organizational performance measurement deals with the evaluation of performance outside one’s own organization, e.g. supplier quality, delivery performance, number of suppliers that are ISO 9000 certified, etc (Schmitz, et al, 2003). Suggested benefits from using these measures and related procedures include for example for the customer to be assist them to better manage the supply base and to inform decisions that are concerned with the supply base (i.e. supplier selection) or for the supplier to be able to adapt to customers expectations (Handfield and Nichols, 1999). Being able to adapt to customers expectations can lead to higher customer satisfaction, which are key drivers of financial performance of suppliers (Matzler, 2004). Numerous empirical studies confirm a positive relationship between customer satisfaction and profitability (e.g., Anderson, et al, 1994, Eklof, et al, 1999, Ittner, et al, 1998). Another use of inter-organizational performance measurement can be the measurement of efficiency between two or more organizations for comparison. By doing this, the organization can learn from each other, thus helping each other to raise efficiency together. This same principle is being used in England by various benchmarking clubs which are brought together under the umbrella of the Construction excellence program (Bakens, 2005).


4 CIB World Building Congress 2007 Because the PaPer instrument is being developed for the registration and prequalification of contractors by Dutch (public) construction clients, with the goal of being able to better manage the supply base, the PaPer instrument should be categorized as an inter-organizational performance measurement instrument. Knowing this, the PaPer instrument should measure the performance of a contractor on a project level, because it’s during a project where the organizations of a client and its contractor intertwine. This also has a direct influence on what the PaPer-instrument should measure. For instance a client is not interested in the profit a contractor makes as a company or even on a project level. The client is more interested in a timely delivery of its requested product, site safety (image), product quality.

5. CHARACTERISTICS OF PERFORMANCE MEASUREMENT ON A PROJECT LEVEL

The PaPer-instrument will be designed to measure the delivered performance by a contractor to the client on a project level. This paragraph will take a look at the difficulties this might impose on the instrument, and will propose a solution for them. One of the characteristics of construction projects is that the project risks (i.e. project variables) involved with construction can be different from project to project. These project variables can be of impact on the performance of a contractor on a project level. Project variables have been the subject of many studies all over the world. Chan and Kumaraswamy (2002) compared eleven different studies on project variables causing delays in construction projects from 1971 up till now. Chan (1998) in his dissertation defined four main variables that influence the performance of a contractor: project scope, project complexity, project environment and management attributes. What complicates the measurement of performance in this context is that not all of the project variables are under control of the contractor. For instance project variables as client related factors, unknown geological conditions, political risks, design variations can all have their effect on final outcome i.e. the performance of a contractor. Without the contractor having any control over these variables. Akinci (1998) underlines this statement. What complicates this even further is the fact that the party responsible for a certain risk can be different from project to project. For example mistakes in the design can be the responsibility of the client in one project, while on another project this is the responsibility of the contractor (for instance in a design and build contract).

6. THE DIMENSIONS OF PERFORMANCE ON A PROJECT LEVEL



Yasamis et al (2002) defines the quality of a construction project in two main parts, namely the produced facility (or product) and the contracting service (or process) by which the product has been delivered. The contracting service is explained as the transformation process of available resources into the produced facility. The Project Management Institute (PMI) uses a similar definition for quality: “Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs” (Ulmer, 2002). Process quality can not be defined in one single definition. Process quality should be described in multiple dimensions. Evans and Lindsay (1996) give the following dimensions for process quality: time, timing, completeness, courtesy, consistency, accessibility, accuracy, and quickness of reaction. Both Yasamis and the PMI define the quality of a project as a combination between the process quality and product quality. Measuring the performance of a contractor on a project level should therefore measure product aspects as well as process aspect. In PaPer another method of approach has been chosen. PaPer will be designed to measure only the process side of the project, because of the simple fact that every project has to be delivered according to the specifications set by the client. A contractor can not be held responsible for the performance of the building as a facility based on these specifications. A contractor can only be held responsible for not conforming to these specifications.

7. TYPE OF PERFORMANCE INDICATORS

Before defining the performance indicators of PaPer, it is important to realize what type of performance indicators can be identified. According to Stricoff (2000) performance indicators can broadly be classified as leading or lagging indicators. This paragraph will give an overview of the different types of performance indicators that can be identified, giving directions for the development of the PaPer instrument. Leading, lead, or prospective indicators is a performance driver, i.e. a measure which drives the performance of the outcome measure (Ahren, 2005). These lead measures are proactive and preventive in nature (Anderson & McAdam, 2004). These measures help anticipate and impact future desired results (Manoochehri, 1999). They are needed to drive performance throughout the organization (Clark and Taylor, 1998) and they furnish information on incremental steps towards larger goals (Anderson & McAdam, 2004). In other words the leading indicators deal with issues that will eventually impact on the financial performance, but significantly, provide the information before the issues have had time to have any effect (Kagioglou et al, 2000)


6 CIB World Building Congress 2007 The outcome measure itself is simply the lagging, lag, or retrospective indicator. Typical lagging indicators are different financial (return on investment, sales per employee, profit per unit production) measurements. According to Sangers (1998) financial measures which are useful – but they tend to measure the past – and they tend to measure the easily measurable. Lagging indicators report on results in decisions made in the past and therefore is of little use in improving current performance. Organizations that rely on financial measures alone can identify their past performance but not what contributed to achieve that performance (Kagioglou et al, 2000). They are a result of management performance, not the cause of it (Drucker, 1993). In economical contexts, often one additional indicator is used. This is the so called coincident indicator (Akintoye et al, 1996). This indicator changes its value at the same time as the specific object, target, or event of interest is changing. They are beneficial to use when the measure of interest is more or less impossible to measure (Ahren, 2005). An example of typical coincident indicators, are those who indicates changes in the overall economy. Traditionally, benchmarking has occurred at the output stage, which is more downstream, based on the measurement of lag benchmarks of organizational performance. Increasingly, benchmarking is occurring at the input, process stage, which is otherwise known as the upstream elements of the organization whereby lead benchmarks of performance are readily identified. Therefore, it is clearly evident that benchmarking must evolve from being backward looking static measures to more forward looking dynamic ratios (Sarkis, 2001). The same must be said for the PaPer-instrument. Measuring the output of a construction project may give directions for improvement in the future, but being able to identify the key performance drivers in a project will have a much greater impact on the final performance.

8. WHAT TO MEASURE?

A performance measurement system can measure almost everything you want it to measure, from financial performance till the level of education of personnel. It’s the question however, how to focus the instrument on those attributes of performance that it will contribute to the main objectives: (1) stimulating contractors to perform better in construction projects on time, costs and quality (2) in such a way that the satisfaction of clients will rise. This paragraph will first go in on main concept that was chosen to focus the PaPer instrument. Secondly this paragraph will present the main attributes chosen for further development. The concept of the Importance Performance Analyses (IPA) is chosen to be the leading concept in the development of the PaPer instrument. IPA, originally introduced by Martilla and James (1977), yields insights into which product or service attributes a firm should focus on to achieve customer satisfaction (Matzler, 2004). The IPA consists out of a two



dimensioned matrix, where importance is depicted along the y-axis and performance (satisfaction) along the x-axis. The means of performance and importance divide the following matrix into four quadrants (see figure 1). Low satisfaction on highly important attributes demand immediate attention (Quadrant 2). To enhance overall satisfaction, a firm should concentrate on these attributes (Matzler, 2004). This same concept is used to define those attributes PaPer should measure. By focussing PaPer on the same quadrant (2), contractors will be pressured in giving more attention to these attributes. Contractors however should always bear in mind the minimum level of attention needed for the whole project, in order not to loose the satisfaction of a client on less important issues.

Quadrant 4

Low importance

High performance

“Possible Overkill”

Quadrant 1

High importance

High performance

“Keep up the good work”

Quadrant 3

Low importance

Low performance

“Low Priority”

Quadrant 2

High importance

Low peformance

“Concentrate Here”

High

Low High

Performance(satisfaction

Importance

Figure 1 Importantce-performance analyses (Martilla and James, 1977)

8.2 Attributes with a low satisfaction and a high Importance rating

Soetanto, et al (2000) conducted a study on the perception of clients and architects about the performance of contractors in the construction sector. The goal of the study was to supply contractors with information where they could best improve their performances in order to reach higher satisfaction among their clients. The same principle of the importance-performance analyses (Martilla, et al, 1977) was used. The outcome of the study is summarized in table 1. This research by Soetanto gives a first insight in what clients find most important in what contractors should improve in order to raise client satisfaction. Further insight can be given looking at similar performance measurement systems in other countries.


8 CIB World Building Congress 2007 Table 1 Highest priority contractor performance criteria (most needed improvements according

to clients and architects (Soetanto, et al, 2000)) Performance criteria contractor Ranking clients Ranking architects Completion of defects 1 1 Adherence to schedule (time performance) 2 8 Management and co-ordination of subcontractors and suppliers

3 2

Adherence to budget (cost performance) 4 Quality of hand-over document (Organisation & Management Manual, Health and Safety)

5

Ability to plan and programme properly 6 3 Manpower management (quantity and quality of craftsmen)

7

Proactive attitude towards scheme 8 Avoidance of claims 9 4 Quality of construction and workmanship 10 5 Smoothness of operation and hand-over 6 Site supervision and control 7 Site organisation, tidiness and cleanliness 9 Preparation of shop drawings and as-built drawings 10 There is a wide variety of performance measurement and registration instruments operational in the building and construction industry. The following three categories of performance measurement instruments have been defined: • Clients measuring the performance of contractors on a project level, for

the use of prequalification of contractors (same goal as PaPer). • Benchmarking instruments (where companies compare performances) • Instruments for the pre-qualification of contractors which qualify

contractors based on standard criteria, and do not take in account some sort of measured performance.

Figure 2 Selection of most relevant instruments for the development of

the PaPer instrument (see appendix for abbreviations) The instruments can be divided in instruments that measure the process performance, the product performance or both. The following instruments have been identified and categorized (see figure 2).

P re Q u alificat ion / R egistrat ionbased on standard criteria

( fin an cial sou ndn ess , capacity , etc)

P re Q u alification / B id evalu at ionbased on th e m easu red

perform an ce

P erform an ce m easu rem en t w ithanothe r objective th an

pre - qu alification

T h ree categ ories o f instru m ents Pro du ct Pro cess Pro d uct & Process

APCC

CHI

CII -BM

CLU BS

COL

QONQU AS

K PI

NS WPQS

PQS - PM (& PQS )

R W S

VE H

Pro R ailQualib at

PQ - states -U S

NS W - PM CPRCCAS S

K lasBel

R g d



The instruments positioned in the upper right corner of figure 2 are defined as the most relevant for this study. The next table (2) shows the focus of the selected instruments. Together with the criteria out table 1 a clear client focus on certain aspects can be defined (see the last column, table 2).

Table 2 (Main) Indicators used by the different performance measurements instruments that

are used for prequalification purposes

9. CONCLUSIONS FOR FURTHER DEVELOPMENT

PQS-PM NSW-PM CPR CCASS RWS Rgd Possible main attributes for PaPer

Time Time mgt. Time mgt.

Timely performanc

e

Quality of planning and work

prep

Planning and

compliance to date of delivery

Time performance

Mgt. and suitability of personnel

Mgt of site

person-nel

Mgt. & Suitability of

site personnel

Industrial rel. and

subcontr.

Mgt. of subcontr.,

consult. and suppliers

Effectiveness of mgt.

Handl of subcontr

Handling of subcontractors

Function / quality

Standard of work

Stand-ard of Work

Standard of work

Standard of work /

compliance to contract

Contract doc.

Contract adm.

Contract adm.

contract deviations

Contract adm.

Contract adm. / deviations

Quality systems

Quality assu-rance

Quality control

Quality plan

Quality control

Health & Safety

health, safety and

rehabilitation mgt.

health and

safety

Compliance to labor &

safety standards

safety and

health plan

safety and health plan

Compliance to Health and

Safety standards

Environ-ment

Environmental mgt

Environment

Training and

indigenous employmen

t

Workforce and skills

development

Training, workforce

development

Co-operation relationships

quality of relations

Organization and

communic.

Quality of relationship,

commnunication

Variations / claims

Variations and claims

Variations and claims


10 CIB World Building Congress 2007 The objective of this research project was to define the conditions of a past performance measurement instrument for contractors. Clients want to use this measured performance in future tendering procedures, helping them in identifying high performing contractors. The following conditions are defined out of literature: I. The PaPer instrument should be categorized as an inter-organizational

performance measurement instrument. It deals with the evaluation of performance outside ones own organization, e.g. supplier quality, delivery performance etc. This means that the PaPer instrument should measure the performance of a contractor on a project level because this is where the organizations of a client and its contractor intertwine.

II. Project variables and the organization responsible for these variables can be different from project to project. Some variables, which influence the performance of a contractor, are the responsibility of the contractor and some aren’t. This means the PaPer instrument should be able to take in account these changing circumstances.

III. Performance on a project level has two main parts: process and product performance. PaPer will be designed to measure the process performance, because a contractor cannot be held responsible for the performance of a building based on specifications set by client.

IV. PaPer should be designed not only to measure lagging indicators but also leading indicators, because leading indicators drive the performance of an organization.

V. To enhance overall satisfaction among clients, a performance measurement instrument should at least measure or stimulate those attributes where clients experience a low satisfaction on highly important attributes.

VI. Looking at other instruments and research done in the past, the instrument should measure the following attributes: time performance, quality control, standard of work, handling of subcontractors, suitability of site personnel, contract administration, environmental issues, training and workforce development, quality of relationship and communication, variations and claims/cost performance.

Now the main conditions for the Paper instrument are identified, the instrument itself can be developed in the next research phase 2. Together with participants from clients’ as well as the contractors’ side, the instrument can be designed to measure those attributes of performance that really matter. Next to the development of the instrument itself, the legal discussion on the application of the measured performance can be started.



10. REFERENCES

Ahren, T., (2005), A study of Maintenance Performance Indicators for the Swedish Railroad System, Lulea University of Technology. Akinci, B., & Fischer, M., 1998, “Factors Affecting Constractors’ Risk of Cost Overburden”, Journal of Management in Engineering, p.67-76. Akintoye, A., Bowen, P., & Hardcastle, C., 1998, Macro-economic leading indicators of construction contract price, Construction Management and Economics, Vol. 16, pp.159-175. Anderson, K., and McAdam, R., 2004, A critique of benchmarking and performance measurement, Lead or Lag?, Benchmarking: an international journal, Vol. 11, No. 5, pp. 465-483. Ang, K.I., Courtney, R.G., Spekkink, D. (2004), Inventory of international reforms in building and construction, and summaries country reports, PSIB, Gouda, The Netherlands. Baird, L., 1986, Managing Performance, (New York: John Wiley). Bourguignon, C. 1995, Peut-on definer la performance?, Reveu Francaise de comptabilite, nr.269, July-August. Burns, W.J. Jr. (ed.), 1992, Performance Measurment, Evaluation and Incentives, (Boston: Harvard Business School Press). Chao-Duivis, M.A.B, Koolwijk, J.S.J., & Geraedts, R.P., 2005, Prequalification of contractors based on past performance, Legal Feasibility (in Dutch), Delft University of Technology, The Netherlands. Corvellec, H., 1994, Performance: From one language into another or the mutations of a Notion, PaPer presented at the 17th EAA Congress, Venezia, Italy. Drucker, P.F., 1993, We need to measure, not count, Wall street journal, Apr 13, 1993 Sec: A p: 18. Eklof, J.A., Hackl, P., & Westlund, A., 1999, On measuring interactions between customer satisfaction and financial results. Total Quality Management, no. 10, pp.514-522. Evans, J.R. and Lindsay, W.M., 1996, The Management and control of quality, 3rd edition, (St. Paul, MN: West Publishing). Handfield, R.B. & Nichols, E.L., 1999, Introduction to supply chain management, (Upper Saddle River, NJ: Prentice Hall). Ittner, C.D., & Larcker, D.F., 1998, Are non financial measures leading indicators of financial performance? Journal of Accounting Research, no. 36, pp.1-35. Kagioglou, M., Cooper, R., and Aouad, G., 2000, Performance management in construction: a conceptual framework, Construction management and economics, Vol. 19, pp.85-95. Koolwijk, J.S.J., Geraedts, R.P., & Chao-Duivis, M.A.B., 2005, Prequalification of contractors based on past performance, a feasibility study (in Dutch), Delft University of Technology, The Netherlands. Lebals, M. & Euske, K., 2002, A conceptual and operation delineation of performance, 2002, Business Performance Measurement, (Cambridge University Press), pp. 65-79. Manoochehri, G., 1999, Overcoming obstacles to developing effective


12 CIB World Building Congress 2007 performance measures, Work study, Vol. 48, No. 6, pp.223-229. Martilla, J.A. and James, J.C., 1977, Importance-performance analyses, Journal of Marketing, vol 41 (1), pp. 77-79. Matzler, K., Bailom, F., Hinterhuber, H.H., Renzel, B. and Pichler, J., 2004, The asymmetric relationship between attribute-level performance and overall customer satisfaction, Industrial Marketing Management, Vol. 33, pp.271-277. Neely, A. (ed.), 2002, Business Performance Measurement, Theory and Practice, (Cambridge: Cambridge University Press). Richard, J., 1989, Audit des performances, (Paris: La Villegeurin). Sangers, M., 1998, Supporting the balanced scorecard, Work study, Vol.

47 (6), pp.197-200. Sarkis, J., 2001, Benchmarking for agility, Benchmarking: An international journal, Vol. 8, No. 2, pp. 88-107. Schmitz, J.& Platts, K.W., 2003, Roles of supplier performance measurement; Management Decision, vol. 48. nr. 8, p.711-721. Simons, R., 1999, Performance Measurement and control systems for implementing strategy, (Upper Saddle River, NJ: Prentice-Hall). Soetanto, R., Proverbs, D.G., & Holt, G.D., 2000, Achieving quality construction projects based on harmonious working relationships, International journal of quality & reliability management, vol. 18-5, pp. 528-548. Stricoff, R., 2000, Safety Performance Measurement, Professional Safety, vol. 45, pp. 36-39. Ulmer, K., 2002, Project Quality Management, Study Notes, PMBOK 2000 based, Version 6.0, In preparation for PMP Certification Exam, Project Management Institute. Yasamis, F., Arditi, D., & Mohammadi, J., 2002, Assessing contractor quality performance, Construction Management and Economics, Vol. 20, pp.211-223.


defining the conditions of a contractors’ past performance ... · defining the conditions of a...

Documents