the application of life cycle costing

8/18/2019 The Application of Life Cycle Costing

1/37

This article was downloaded by: [Ministerio De Defensa]On: 01 April 2015, At: 03:57Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Defence and Peace EconomicsPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/gdpe20

The Application of Life Cycle Costing

in Evaluating Military Investments: An

Empirical Study at an International

ScaleAndrés Navarro-Galera a , Rodrigo I. Ortúzar-Maturana b &

Francisco Muñoz-Leivaa

a Facultad de Ciencias Económicas y Empresariales , University of

Granada , Campus Cartuja, s/n 18071 Granada, Spainb Management of Programs, Research and Development , Chilean

Navy, ChilePublished online: 15 Oct 2010.

To cite this article: Andrés Navarro-Galera , Rodrigo I. Ortúzar-Maturana & Francisco Muñoz-Leiva (2011) The Application of Life Cycle Costing in Evaluating Military Investments: AnEmpirical Study at an International Scale, Defence and Peace Economics, 22:5, 509-543, DOI:10.1080/10242694.2010.508573

To link to this article: http://dx.doi.org/10.1080/10242694.2010.508573

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or

arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

http://dx.doi.org/10.1080/10242694.2010.508573http://www.tandfonline.com/action/showCitFormats?doi=10.1080/10242694.2010.508573http://www.tandfonline.com/loi/gdpe20


2/37

Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditionshttp://www.tandfonline.com/page/terms-and-conditions


3/37

Defence and Peace Economics, 2011

ISSN 1024-2694 print: ISSN 1476-8267 online © 2010 Taylor & Francis

THE APPLICATION OF LIFE CYCLE COSTING IN

EVALUATING MILITARY INVESTMENTS: AN

EMPIRICAL STUDY AT AN INTERNATIONAL SCALE

ANDRÉS NAVARRO-GALERAa,*, RODRIGO I. ORTÚZAR-MATURANAb

AND FRANCISCO MUÑOZ- LEIVAa

aFacultad de Ciencias Económicas y Empresariales, University of Granada, Campus Cartuja, s/n

18071 Granada, Spain;

b

Management of Programs, Research and Development, Chilean Navy, Chile

Taylor and FrancisGDPE_A_508573.sgm

(Received 6 October 2009; in final form 30 June 2010)

10.1080/10242694.2010.508573Defence and Peace Economics1024-2694 (print)/1476-8267 (online)Original Article2010Taylor & Francis00000000002010Dr. [email protected] life cycle costing (LCC) is internationally considered the best instrument for evaluating investments inmilitary equipment, its practical application remains insufficiently studied. This paper presents an internationalpanorama of systems used for evaluating military investments, identifying the procedures and instruments mostcommonly employed, and identifying their strengths and weaknesses. The LCC methodology is then systematicallyapplied, revealing opportunities for improvements. The research methodology is based on a questionnaire sent to 64countries, inquiring into their standard practice regarding decision taking, cost estimation, risk management andperformance indicators. The results obtained show that the main limitations facing these countries in employing LCCmethodology concern weaknesses in their organizational structures and in their technical regulations, with particular

respect to cost breakdown structures. Also significant are the scant number of cost estimation models available, theabsence of up-to-date, reliable databases and the limited use made of appropriate techniques for risk and uncertaintyestimation.

Keywords: Life cycle costing; Military investment; Project assessment

JEL Codes: H43, H56, M41

INTRODUCTION

Various studies have analysed the rapid recent growth in the costs of weapons systems (for

example, Coleman et al., 2000; Eskew, 2000; Birkler et al., 2001; Dameron et al., 2001; Sippleet al., 2004). All these authors, in agreement with the Government Accountability Office(2006), have observed that in the majority of weapons investment programmes, the real costis well above the budgeted amount, and thus greater effort is required to improve, on the onehand, the initial estimates and, on the other, the cost control systems used. In this respect,Melese et al. (2007) concluded that cost estimates serve a dual function: first, as an integralpart of the decision-making process to evaluate military purchases/investments, and second,as a baseline for future defence budgets.

*Corresponding author. E-mail: [email protected]

http://dx.doi.org/10.1080/10242694.2010.508573

Vol. 22(5), October, pp. 509–543


4/37

2 A. NAVARRO-GALERA ET AL.

According to ISO 15288, there are six life cycle phases of any investment in capital goods:concept, development, production, utilization, support and retirement. Nevertheless, the distri-bution of the total cost of a national defence investment project is concentrated in the utiliza-tion and support phases – see Clark et al. (1999), Martin and Evans (2000), Flowe (2002),Nelson (2003) and Gates and Greenberg, (2006), as well as the US Navy Cost Estimating

Handbook published by the Naval Sea Systems Command – NAVSEA (2005). Figure 1 showsthat in the USA the life cycle cost of national defence investment projects is concentrated inthe utilization and support phases (when disposal is included, these items account for 72% of the total). This finding has been corroborated by Calobrisi (2006). A similar cost distributionis to be found in other countries, such as Australia (Flyvbjerg et al., 2002) and New Zealand(Pugliese, 2007), and may even be more extreme, as is the case of Germany, where theMinistry of Defence has recognized that the cost of utilization and support amounts to 80–95%of the entire life cycle cost (Biermann, 2005).FIGURE 1 Percent distribution of the cost of a weapons programme.Source: Defense System Management College (1997)In this situation, with diverse alternatives available, the result of the decision-makingprocess to evaluate military investments may vary widely, depending on whether the evalua-tion is made exclusively on the basis of acquisition cost or whether life cycle costs are takeninto account. Under the latter, the estimator must estimate the costs derived from all the phasesof the project life, and not just those associated with the initial investment.

The suitability of life cycle cost analysis for the above purpose has been defended in variousseminal studies; Waak (2004) examined cases in which support costs could be reduced by upto 50% by using an integrated logistic support system in which life cycle costing (LCC) wouldhave a different impact depending on the type of system examined and the degree of sensitivitythat can be measured in the cost/effectiveness analysis. In parallel terms, Masiello (2002) heldthat with LCC analysis it is possible to identify the most significant cost generators and thusobtain the best combination of resources, while Ferrín and Plank (2002) claimed that LCCevaluation provided a long-term view, giving management a more accurate assessment of

acquisitions.

FIGURE 1 Percent distribution of the cost of a weapons programmeSource: Defense System Management College (1997)

510


5/37

LIFE CYCLE COSTING 3

The use of LCC for evaluating investments in weapons purchases is not a new topic; accord-ing to Blanchard and Fabricky (1991), the logic of LCC means that, for all the phases of theengineering design process, cost considerations should be explicitly incorporated, thusenabling the design decisions adopted to be aimed at selecting the best product or system. InOctober 2001, the ‘Cost Structure and Life Cycle Cost for Military Systems’ symposium was

held, coordinated by the SAS-028 Study, Analysis and Simulation Panel, a NATO workgroup. This symposium was organized to begin to address the question, and initiatives are stillbeing taken towards this end. Thus, the NATO work group AC/327 ‘Life Cycle ManagementGroup’ was created, to analyse the generation of life cycle processes for application in coop-eration and weapons-supply programmes. In addition, the SAS-028 work group published adocument in 2003, ‘Cost Structure and Life Cycle Costs for Military Systems’, whichacknowledged the absence of a standardized costs structure and recommended research intohow LCC is incorporated into national defence investment decision-taking processes. In paral-lel, the LCC technique has been adopted for the evaluation of military investments in countriesincluding the United Kingdom (Moore and Antill, 2001; Hartley and Parker, 2003), Australia(Clark et al., 1999), and the United States (Boudreau and Naegle, 2005).

In this context, several factors may be cited to corroborate the timeliness and interest instudying the question of LCC: first, although this technique is internationally considered oneof the best available for evaluating military investments, both theory and top military staff recognize that the LCC methodology remains insufficiently studied. Tysseland (2008) identi-fied the scant knowledge of LCC as being one of the most severe shortcomings affecting theeconomic evaluation of military investments. Similarly, NATO (2008, 2009) has recognizedthat no effective means currently exist for the application of this technique. Secondly,although according to ISO 15288, there are six phases in the life cycle of a capital investment,the distribution of the total cost of a national defence investments project is concentratedparticularly in those of utilization and support, as concluded by Clark et al. (1999), Martin and

Evans (2000), Flowe (2002), Nelson (2003), Gates and Greenberg (2006) and NAVSEA(2005). Thus, given the existence of diversity and alternative solutions, the application of LCCis of interest because it provides useful information for deciding upon projects that representthe lowest overall life cycle cost – not necessarily those with the lowest acquisition costs – thusenabling financial and efficiency savings in public spending on national defence projects.Thirdly, although the effectiveness and viability of LCC has been demonstrated in particularcases (Waak, 2004; Masiello, 2002; Ferrín and Plank, 2002), very few empirical studies havebeen aimed at analysing the generalized implementation of this technique at an internationallevel, and so it could be very useful to identify opportunities for improvement in the currentsystems applied for evaluating military investments in different countries.

This paper is intended to contribute to furthering our knowledge and application of the LCCmethodology to decision taking in the field of investments in military equipment, and for thispurpose we established a two-fold objective: first, to identify the procedures, models andinstruments that are most widely used at present in evaluating national defence investments.The idea underlying this initial goal is to examine whether the international trend in adminis-trative systems for assessing military investments, and the legislation and regulation in thisrespect, is towards the generalized introduction of the LCC methodology. Secondly, to iden-tify deficiencies in current practices of weapons systems evaluation, as regards a systematicapplication of the LCC method, in the firm conviction that a clear definition of the weaknessesof current evaluation models will enable us to define opportunities for improvement, to be

worked on in the future.Thus, the ultimate aim of this paper is to provide an international overview of evaluationsystems for military investments; through an analysis of their strengths and weaknesses, weseek to improve the generalized application of the LCC methodology for evaluation and

511


6/37


selection in this field. The rest of the paper is organized as follows: the next section discussesthe conceptual framework of the LCC technique, including its basic definitions, managementactivities, and the stages and classes of life cycle costs in military investments. We thenpresent the results of an empirical study based on a questionnaire sent to the DefenceMinistries of 64 countries. This consisted of 20 items concerning decision-taking procedures,

cost-estimation models and methods, the cost breakdown structure, risk and uncertaintymanagement, and performance indicators. The results obtained, which included a total of 29completed questionnaires, are analysed globally and by geographic areas (Africa, America,Asia and Oceania, Europe and the Middle East). In addition, a cluster analysis is carried out,from which we draw conclusions by countries grouped according to the homogeneity of thereplies received. Finally, the Conclusions section brings together the main findings obtainedfrom this empirical investigation.

Our findings show that the main limitations to the use of the LCC methodology concerndeficiencies in countries’ organizational structures and in their technical regulations, togetherwith the lack of cost estimation models and the insufficient use made of appropriate risk anduncertainty estimation techniques. These limitations, which are less acute in North Americaand Western Europe, enable us to identify opportunities for improvement via the systematicapplication of the LCC methodology, involving strengthening organizational structures, rais-ing technical standards, adopting measures of innovation in cost and risk estimation models,and creating and maintaining reliable databases. In general, countries in South America andCentral Europe present the most severe shortcomings with respect to the estimation of theLCC of military investments.

LIFE CYCLE COSTING AS A TECHNIQUE FOR EVALUATING NATIONAL

DEFENCE INVESTMENTS

The earliest references to the application of LCC techniques are to be found in the USA, duringthe Second World War, as a result of the activities of the Massachusetts Institute of Technology Radiation Laboratory (RadLab). LCC, as a formal technique, was first imple-mented in the USA during the 1960s, and in the UK in the 1970s (Woodward, 1997). In 1945,RadLab created and implemented different design processes for systems to facilitate the devel-opment of microwave radars for military use, and among these processes was the Life CycleCost (Christensen et al., 2005). Subsequently, in 1960, US Department of Defense officialscalculated that the utilization and support costs of weapons systems represented 75% of thetotal costs incurred during their life cycle (Gupta and Chow, 1985). According to Dhillon

(1989), between 1978 and 1981 there was an exponential increase in the number of studiesmade in this field, with the main area of application being that of the construction of buildingsand other infrastructure. Among the contributions made concerning the defence industry arethe studies by Blanchard (1979, 1995, 1999), who, together with Fabricky (Blanchard andFabricky, 1991), established that the logic of LCC demands that cost considerations be incor-porated when investment decisions are to be taken, so that the best product or system may beachieved.

Conceptual Premises of Life Cycle Costing

We agree with Kirkpatrick (2008) that it is currently important for any nation to make sure thatresearch into national defence issues contributes to ensuring that investment decisions aretaken on the basis of appropriate military and economic criteria, a process in which the LCCmethodology can help.

512


7/37


With respect to the conceptual premises of LCC, according to the definition made by the

US Department of Defense (2008), the life cycle corresponds to ‘all the stages that elapsefrom the moment an item is initially developed until the moment it is consumed by use orwithdrawn as surplus’. In 2007, the AAP-48 Procedure ‘NATO System Life Cycle Stages andProcesses’ was published; this document became the standard handbook for implementationof a structure oriented towards life cycle management and to providing a common methodol-ogy for implementing the principles and terminology of this new means of administering‘systems of interest’.

Under the systems engineering approach, Arnold (2005) cited three models proposed foridentifying life cycle stages. As shown in Table I, these correspond to: (1) InternationalOrganization for Standardization (2002): ISO/IEC 15288 Systems Engineering. System life

cycle processes; (2) US Department of Defense – DOD (2003a): Instruction 5000.2 Operationof the Defense Acquisition System’; and (3) United Kingdom Ministry of Defence – MoD(2005): The Acquisition Handbook – Sixth Edition.

On the basis of the above references, the following cost categories could be consideredwithin a LCC programme: (a) pre-acquisition costs, which are those associated with theConcept definition, and correspond to the disbursements made for internal activities of theorganization and/or external advisory services; (b) acquisition costs, that is all the costs asso-ciated with R&D, and those corresponding to the initial investment, i.e. the engineering,construction, testing, commissioning, transfer and integrated logistics support activitiesrequired to incorporate a system into an organization; (c) utilization costs, corresponding to

the variable costs derived from the system’s operation, in accordance with the degree of activity planned (for example, hours/year), or with the operational profile (for example,speed); (d) support costs, this item reflects the consumption of resources attributable topreventive and corrective maintenance, baseline infrastructure, spare parts and associated

TABLE I Life Cycle Stages of Weapons Systems

ISO/IEC United States DoD

United Kingdom

MoD Objectives Decision options

Concept Concept Refinement Concept – A statement of the militarycustomer’s requirement

– Analysis of the concept ofuse

– Propose viable solutions

– Execute the next phase.– Continue the present

phase.– Return to the previous

phase.Development Technology

Development

System Developmentand Demonstration

Assessment

Development

– Adjust system requirements– Describe the solutions

developed– Build the system– Verify and validate the

system

– Postpone the project– End the project.

Production Production andDeployment

Manufacturing – Mass production (or other)system

– Inspection and testingUtilization Operations and

Support (includedisposal)

In-Service Operation of the system to

satisfy users’ needs

Support Provide the sustained capacityof the system

Retirement Disposal Efficient, effective and safedisposal of the equipment

Source: Arnold (2005)

513


8/37


expenses (not considered in Acquisition), repairs, and modifications and/or foreseeableupgrades, to ensure the availability of the system and fulfilment of its function; (e) disposalcosts, those arising from the maintenance of the system from the moment it is consideredreserve material and, eventually, those corresponding to its disarming, removal, scrappingor incapacitation, with due consideration for environmental and safety requirements.

Kirkpatrick (2000) studied this question with regard to the United Kingdom and concludedthat the life cycle costs of a weapons system consist of procurement cost, direct in-servicecost and indirect in-service cost.

State of the Art in LCC

Some countries have long been making resolute efforts to incorporate the LCC methodol-ogy into the evaluation of their military equipment investment projects. Thus, with the aimof reducing the life cycle costs of defence equipment, the concept of Continuous Acquisi-tion and Life-Cycle Support was widely implemented in 1990 within NATO countries(Freeman, 1995; NATO CALS, 2000). Similarly, the United Kingdom, applying an acquisi-tions policy based on the criterion of ‘value for money’ (Moore and Antill, 2001), estab-lished its own approach to life cycle costing for defence programmes, introducing a broaderconcept known as Whole Life Cost (Hartley and Parker, 2003). In Australia, too, theeconomic evaluation of defence programmes was reoriented to take LCC into account(ANAO, 1998; Clark et al., 1999). A similar trend was followed in the USA; a majoradvance in this respect was made in 2000, when the weapons acquisition system wasrestructured and the concept of Total Ownership Cost introduced, based on LCC principlesand methods (Boudreau and Naegle, 2005). Notwithstanding these initiatives, Tysseland(2008) claimed there remained serious deficiencies in the process of the financial evaluationof military investments, as a result of the inherent uncertainty in weapons programmes, the

degree of information available, the attitudes of project leaders and the lack of knowledgeregarding LCC.

The LCC technique is considered among the best instruments available for evaluatingnational defence investment projects. However, both theoreticians and the principal mili-tary authorities in this field have recognized that the methodology is insufficiently under-stood. Accordingly, further studies are needed, at both conceptual and empirical levels, of techniques and procedures that may prove effective for estimating the costs associatedwith the life cycle phases of investments in weapons programmes. As acknowledged byNATO (2008, 2009) there is widespread awareness of the issue of LCC, but as yet, thereare no effective solutions – thus reflecting the complexity, importance and timeliness of

the question.Accordingly, a critical evaluation of the use of LCC could highlight areas in whichfurther study of the technique can usefully be made. Previous research has suggested thatone area to be investigated in the development of the LCC methodology is that of thedecision taking procedures established. Tysseland (2008) identified the lack of politicalresolution as one of the main hindrances in this respect, and observed that this couldhinder the issuance of standards and the creation of evaluation organizations. In parallel,in the USA, the Federal Acquisition Regulations state that a key factor to success lies inestablishing clear-cut responsibilities within the organization, as a basic element in proce-dural design. Furthermore, the RTO-SAS-54 workgroup has confirmed that the LCC

methodology is still insufficiently rooted in the organizational culture of NATO membercountries.Cost estimation and risk and uncertainty analysis methods have been highlighted as two

of the subjects in need of most investigation with respect to improving the implementation

514


9/37


of the LCC technique. The RTO-SAS-54 workgroup, taking as a reference the PhasedArmaments Programme System (PAPS) (NATO Standardisation Agency, 1992), examined12 countries and identified the following deficiencies that hinder the correct application of the LCC technique: very limited use of the life cycle phase terminology proposed in PAPS;the need to establish bases and assumptions for the development of cost models; the

absence of instruments for addressing risk and uncertainty; and the low level of implemen-tation of the generic cost breakdown structure. In the same line, Barringer and Weber(1996) identified the initial hypotheses and the reliability of estimation methods as beingthe main limitations of LCC. On the other hand, for Tysseland (2008), the measurement of uncertainty and the non-availability of information represent important obstacles. Therefore,in order to achieve a better understanding of the study and application of the LCC method-ology, it would be interesting to analyse whether current practice regarding the evaluationof military investments enables users to: (a) review, adjust and update estimation models;(b) standardise initial bases and general assumptions regarding the scenarios in which esti-mations are made; (c) utilize a normalized, codified cost breakdown structure; (d) systemat-ically evaluate risk.

EMPIRICAL STUDY OF EVALUATION PROCEDURES AND TECHNIQUES USED

BY THE ARMED FORCES

Research Methodology

Building on the experience of the SAS-054 work group, Ciobotaru (2008) headed the panelthat produced the document ‘NATO Guidance for Life Cycle Cost’, which proposed a modelcorresponding to a generic LCC process that incorporated the elements shown in Figure 2, in

accordance with systems engineering logic.FIGURE 2 Generic Life Cycle Cost Estimating Process.FIGURE 3 Results Item 1.1.FIGURE 4 Results Item 1-2.FIGURE 5 Results Item 1-3.FIGURE 6 Results Item 1-4.FIGURE 7 Results Item 1-5.FIGURE 8 Results Item 1-6.FIGURE 9 Results Item 2-1.FIGURE 10 Results Item 2-2.FIGURE 11 Results Item 2-3.FIGURE 12 Results Item 2-4.FIGURE 13 Results Item 2-5.FIGURE 14 Results Item 2-6.FIGURE 15 Results Item 3-1.FIGURE 16 Results Item 3-2.FIGURE 17 Results Item 4-1.FIGURE 18 Results Item 4-2.FIGURE 19 Results Item 5-1.FIGURE 20 Results Item 5-2.FIGURE 21 Dendogram. Ward’s method. % mismatch.FIGURE 22 Groups and components. Generalized EM analysis.FIGURE 23 Positioning of each strategic group of countries.

FIGURE 2 Generic Life Cycle Cost Estimating Process.

515


10/37


The methodology applied in this paper takes as a reference point the scheme proposed byCiobotaru (2008) – see Figure 2 – considering this proposal to be one of the most advancedand innovative in the field of LCC. The proposal made in this document responds to the logicof systems engineering, and comprises inputs, instruments and outputs, thus serving as a guidefor identifying the areas of interest for our empirical investigation. To acquire the informationneeded to meet the goals set for this paper, and taking into account the NATO RTO (2003,2007) proposals and Ciobotaru’s scheme (2008), we carried out a survey using a questionnairewith the areas of interest (blocks) shown in Table II.

Annexe I contains the questionnaire used for this study. The question structure is based bothupon NATO RTO (2003, 2007) proposals and on previous studies. The inclusion of Block 1(decision taking and procedures) is justified by the need to study certain issues identified inprevious research, such as the absence of political resolution, the need for clear-cut areas of responsibility and the extent of acceptance of LCC techniques within the organizationalculture. Blocks 2 (cost estimation methods and models) and 3 (cost breakdown structure) are

intended to test whether, as suggested in previous studies, cost estimation models currentlypresent important weaknesses regarding implementation of the LCC technique, and if this isso, to identify them. Finally, Block 4 (risk and uncertainty management) analyses the suitabil-ity of the risk and uncertainty analysis instruments currently used to apply the LCC technique,and Block 5 (management indicators) examines whether appropriate indicators are used inestimating the LCC.

In order to avoid duplications in the text of this paper, the possible replies associated witheach item are identified only in the tables that accompany the Results section. To achieve thetargets set for this research, and bearing in mind the nature of the data being compiled, it wasnecessary to adopt three criteria for classifying the questions presented, as follows: (a)

depending on the degree of freedom of the answer (open/closed); (b) depending on thedegree of coincidence between the intention of the questions and their content (direct/indi-rect); and (c) depending on the correspondence with the specific reality of the respondent(conditional/unconditional). With these premises, items for all of these categories wereincluded in the questionnaire, as shown in Annexe I. For the open questions, the possiblereplies were not limited, while for the closed ones (the majority), the possibilities for replieswere pre-determined. The direct questions were formulated such that the respondent wasexpressly made aware of the object of the inquiry, while the indirect ones sought to obtaininformation without expressly revealing the purpose of the question, in order to avoid thepossible effects of the respondents’ expectations. Finally, the unconditional questions

referred to real situations, lived or experienced by the respondent, while the conditional onesreferred to hypothetical or future situations.Before sending the questionnaire to the respondents, a preliminary version was analysed

and tested by experts from the defence ministries of Chile and Spain, and also by two

TABLE II Areas of Interest Survey

Systems Engineeringvision Ciobotaru (2008) Related Process Areas of Interest Survey (Blocks)

Inputs Data Sources Assumption Planning Making Decisions and Procedures

Tools and Techniques Life Cycle Cost Model Assessment Methods and Models of CostEstimation

Generic Cost BreakdownStructure

Cost Structures

Risk Analysis Risk Management and Uncertainly

Outputs Life Cycle Cost Output Results Presentation Performance Indicators

516


11/37


university teachers who were experts in market research. As a pilot test, these collaborators,who made some suggestions that were incorporated into the text, checked that the question-naire met three conditions: (1) that it was drafted in readily understandable language, avoidingvague and/or ambiguous expressions; (2) that the format and structure were appropriate forresponse by specialists in the economic evaluation of defence projects; (3) that all the ques-

tionnaire items could be answered within a reasonable time period.Finally, let us observe that it is quite comprehensible that the confidentiality of the infor-mation held by any country’s defence ministry might constitute an important barrier to ourcontacting and obtaining a completed questionnaire from its staff, especially if one seeks tocompile information about the current situation and its deficiencies. To overcome thisproblem, after various conversations with diverse military officials, and on the basis of ourown professional experience, we established the following conditions, which facilitated thecollaboration by our respondents: (1) a commitment was given that the use of the datarequested would not prejudice the national or military security of the countries concerned; (2)one questionnaire per country was compiled, but in no case would individualized results bemade public – they would always be aggregated; (3) the official mailing of the questionnairewould be performed via a document transmitted by the corresponding Chilean DefenceAttachés, who undertook to handle all correspondence with the respondents. In parallel form,the respondents would mail their replies to the Chilean Defence Attaché and to the Depart-ment of Financial and Accounting Economics at the University of Granada, where the studywas developed.

Once the completed questionnaires had been received, they were validated, tabulated andanalysed. As shown in the Results section, three types of statistical analysis were performed:global for the whole sample, by geographic zones and by cluster analysis. This procedureenabled us to draw relevant, accurate conclusions for the objectives of this paper.

Hypotheses Proposed

Taking into account the goals of this study, and the critical evaluation of LCC previouslycarried out, we proposed four research hypotheses and established the corresponding items tobe used for testing each hypothesis (see Table III). The aim of Hypothesis 1 was to determinewhether current regulations are sufficient, from a technical standpoint, for the LCC techniqueto be applied. Our analysis of Hypothesis 2 examined to what extent the cost estimation meth-ods currently employed meet the technical requirements necessary for the LCC technique to

TABLE III Research Hypotheses

Hypothesis Items

H1There exist weapons programmes standards and evaluation institutionsthat are suited to application of the LCC methodology

1-11-21-31-4

H2The procedures used in weapons acquisition programmes facilitate thecost estimations necessary for application of the LCC methodology

2-22-32-42-6

H3Legislation on weapons acquisition regulates a cost breakdown structure

that meets LCC requirements

3-13-2

H4Weapons acquisition programmes make use of the risk and uncertaintyestimation instruments required by the LCC technique

4-14-2

517


12/37


be systematically implemented. As a complement to Hypothesis 1, the aim of Hypothesis 3was to verify whether the applicable standards on cost breakdown structures are compatiblewith LCC. In a similar way, Hypothesis 4 extends the findings of Hypothesis 2, testing the useof instruments to analyse risk and uncertainty.

Both the items in block 5 and the other items not included in Table III concern the analysis

of information that is relevant to the aims of this paper, although their multi-response natureled us to eliminate them from the hypothesis tests; nevertheless, they are of use in commentingon the results obtained.

The hypotheses were tested using a t -test. For a moderately large sample, the t -test becomesvery similar to the Normal-test (Z-test), and is therefore robust to violations of the normalityassumption or the presence of unequal variances in the sub-samples (Markowski andMarkowski, 1990). The independent one-sample test is among the most frequently used t -testsfor comparison with the population mean (µ 0) specified in a null hypothesis (Fadem, 2008).

The following expression was used in testing the null hypothesis:

where s is the standard deviation and n is the size of the sample. n − 1 degrees of freedom (d.f.)were used in the test.

We compared a single sample with the population percentages in a binary or dichotomousdistribution (values: 0 or 1). The t -test in proportions has been widely applied, has been foundto be robust, and successfully used in this kind of distribution. In all items, we tested allproportions against the reference value of 50% (0.5). Accordingly, the t -test rejects the nullhypothesis when the percentage observed in the sample differs significantly, above or below,from this reference value.

Description of the Sample

With respect to the sampling technique to be used, and following Berenson and Levine(1992), Levin and Levin (2001) and Arteaga (2008), two types of samples could be used:probabilistic and non-probabilistic. The former is a sample selection process in which theelements are selected using random methods, while non-probabilistic sampling includes allmethods in which the units are not selected by random procedures. For Berenson and Levine(1992), in most analytical studies, only one non-probabilistic sample can be obtained,

namely the intentional sample. Under intentional sampling, and following Levin (1988), theresearcher is able to choose the most representative elements of the population, dependingon the objectives of the study. This opinion concurs with the conclusions of Marín Ibáñezand Pérez Serrano (1985), who claimed that not all opinions are equally well founded.Accordingly, and taking into consideration the characteristics of the population (countries),the study goals and the restrictions imposed regarding accessibility to the information, weopted to develop our study under non-random conditions, and so the intentional samplingtechnique was adopted.

The United Nations (2001) recognizes two defence research bodies with a global repu-tation for excellence: the International Institute for Strategic Studies, based in London,

and the Stockholm International Peace Research Institute (SIPRI). The first of thesepublishes an annual Military Balance, which includes 170 countries and is, in our view,more oriented towards the analysis of conflicts and weapons technologies. On the otherhand, SIPRI publications have a focus that is closer to the economics of defence. Our

t xs n= −

µ 0 /

518


13/37


14/37


(1) Equation for a sample from a finite population

(2) Equation for an optimum sample from a finite population,

where:

N = Population sizeε = Sampling error

p = Success rateq = Failure rate

Z ∀ = Value of the normal distribution

Equations (1) and (2) were applied to the following data, assuming the least favourablesituation (when p=q=0.5), given that prior information for estimating the variance was notavailable.

Thus, the results are as follows.Equation for a sample from a finite population

n Z p q N

N Z p q= × × ×

− + × ×∀ ∀

2

2 21ε ( )

nn

n

N

optimal =+

1

n = × × ×

− + × × =

1 65 0 5 0 5 168

0 05 168 1 1 65 0 5 0 5104

2

2 2

. . .

. ( ) . . .countries

TABLE V Distribution of the Sample and Survey Respondents

Locations Countries in the Sample Countries survey respondents

1. Africa 1 0North Africa 0 0South Africa 1 0

2. America 12 7North America 2 2Central America 0 0South America 10 5Caribbean 0 0

3. Asia and Oceania 13 2Central Asia 0 0East Asia 8 1South Asia 3 0Oceania 2 1

4. Europe 32 19

Central Europe 12 6Eastern Europe 4 0Western Europe 16 13

5. Middle East 6 1Middle East 6 1

64 29

520


15/37


Equation for an optimum sample from a finite population

Segmentation by Clusters of Countries

The geographic areas that constitute our sample enable us to make a comparative analysis.However, in view of the aims of this study, we considered it interesting to define othercountry clusters which, based on the criterion of the similarity of replies, promote the rele-vance and accuracy of our conclusions. Therefore, the cluster analysis technique wasapplied. This involves designating an interdependent set of techniques – in which nodistinction is made between dependent and independent variables – with the aim of formingclusters from a set of elements (Luque, 2000). These clusters, which share common charac-

teristics, must be as homogeneous as possible internally, and heterogeneous between oneand another. When many cases or variables are presented, their classification is neither intu-itive nor obvious, and then it is necessary to apply multivariate techniques, such as clusteranalysis. Specifically, and in view of the characteristics of our data (categorical variables),we utilized the procedure of Generalized Expectative Maximization (EM) or Bayesian Clus-ter Analysis, based on the percentage of mismatch. However, one of the most importantquestions to be answered, before applying this type of technique, is how many clustersshould be obtained. To answer this, hierarchic clustering techniques are normally appliedbeforehand.

Generalized EM Analysis is similar to the k-Means algorithm method, but is appropriate for

categorical data. Specifically, the EM algorithm is a general method for finding the maximumlikelihood estimate of the parameters of underlying distributions from a given data set. Weassume all variables are independent of each other and all data are from k joint distributions(Hill and Lewicki, 2006). The fundamental algorithm iterates between two steps:

1. M-algorithm (Maximization step)

2. E-algorithm (Expectation step)

where:ui is the mean of distribution I ,

σ i is the variance of distribution I , Z ij is the estimated weight (probability) of observation j belonging to cluster I ,ci represents the cluster I ,

p( x j) represents the probability of observation j belonging to cluster i.

noptimal =+

=

104

1104

168

64 countries

Equation for the M algorithm− = =−∑

∑∑

∑u

Z x

Z

Z x u

Z i

ij j j

ij

j

i

ij j i j

ij

j

,

( )

( )σ 2

2

3

Equation for the E algorithm− = Z p x c p c

p xij j i i

j

( ) ( )( ) ( )4

521


16/37


If the increase in the likelihood value is less than the value specified, the iteration stops andobtains the final clustering. If the number of the iterations is equal to the maximum number of iterations, the iteration stops and obtains the final clustering. Finally, variables of a multi-response categorical type (1-6, 2-1, 2-5, 3-5, 5-3, 5-5) were converted into binary (Yes/No)variables in order to obtain a homogeneous basis for comparison and to be able to apply

Generalized EM Analysis.

ANALYSIS OF RESULTS

Descriptive Analysis

Following the information request process, 29 completed questionnaires were received, corre-sponding to 45.31% of the total sample, assuming that only one questionnaire was requestedand received from each country. In compliance with the commitment given to the respondents,the countries are not identified by name, each one being assigned a number within each

geographic sub-region.The number and composition of the countries from which replies were received are suffi-cient for the aims of this paper, and support the reliability of the findings made, for thesereasons: first, the defence spending by the countries that completed the questionnaireaccounted for 68.22% of worldwide defence spending in 2008, according to SIPRI (2009).Secondly, responses were received from countries corresponding to all the geographic areasconsidered of interest by SIPRI (2008), and they are appropriately represented in terms of thenumber of countries and of their annual defence spending. Thirdly, responses were obtainedfrom almost all the countries identified in previous research as having opinions grounded uponspecific experience and as being of interest for the application of LCC methodology.

In order to increase the accuracy of the descriptive analysis of the results, the respondingcountries were grouped by sub-regions, except those where there was only one element, whichwere included in the category ‘Others’.

• Western Europe: 12 countries plus NATO (13 in total)• Central Europe: six countries• North America: two countries• South America: five countries• Others: three countries (one from East Asia, one from Oceania and one from the Middle

East)

Beginning with Block 1, referring to Decision Taking and Procedures (see Annexe I), thereplies to Item 1-1 show that most countries (76%) have a classification system by acquisitioncategories, and thus the effort made can be distributed in accordance with objective criteria,versus those lacking such a system (t = 3.20; d.f.=28; p = 0.0034). Although more than a thirdof those surveyed failed to answer Question 2, most of those that did (74% of those whoanswered Yes or No) replied that their defence department possessed an independent organi-zation for exercising the function of evaluating military acquisitions, which is considered tobe a key factor for success (t = 7.10; d.f.=28; p = 0.0000). With respect to the existence orotherwise of standards for estimating the life cycle cost of military investments, 62% of respondents stated that such standards did exist in their countries ( t = 6.77; d.f.=28; p =0.0000).

The answers to Item 4 show that in the vast majority of countries, the life cycle stages andcontrol milestones are regulated by norms (t = 5.56; d.f.=28; p = 0.00001); however, this doesnot mean that the legislative and organizational barriers in each country may not limit the

522


17/37


adoption of best practices, such as those proposed by NATO. Most regions have benefitedfrom the obligatory use of the NATO system, which has even been adopted in some countriesin South America.

There is therefore empirical evidence supporting hypothesis H1, i.e. a high percentage of countries use a classification system, standards, regulations or independent institutions that

participate in the evaluation process; apparently, therefore, the LCC methodology may beapplied.With respect to the standard that best represents the life cycle phases, 59% of countries had

their own system for identifying life cycle phases, which corroborates the hypothesis that thePAPS system is merely employed as a reference. However, since the introduction of the ISO/ IEC 15288 ‘Systems engineering – System life cycle processes’ it is becoming quite widelyaccepted, and it is to be expected that this standard will provide the basis for establishing futurecommon procedures. Finally, regarding perspectives for future improvements in the decision-taking process, almost half the respondents (44%) recognized the insufficient standardizationof procedures as constituting their principal weakness.

The results from Block 1 show that in most countries there exist standards and institutionsthat, at least in theory, enable the LCC methodology to be applied, although in fact most of these countries represent the life cycle phases on the basis of their own standards system. Thenormalization of procedures is the main deficiency to be rectified. The consolidation of democracy within some countries, and growing acceptance of the Anglo-Saxon culture inothers, seems to have facilitated the existence of organizations that are more stable, togetherwith a constantly revised and improved regulation of standards and procedures. Thus, thepolitical will expressed in these countries has tended to favour the use of LCC techniques.Nevertheless, the solidity of the LCC approach in organizational culture still appears to beinsufficient, as the best results achieved continue to be those of countries with the highestlevel of participation by civilian specialists. These conclusions remind us of the need to study

the types of organizational structures and the technical requirements of standards that mostfavour LCC-based decision taking, with particular attention to the definition of standardizedprocedures.

In Block 2, Cost Estimation Methods and Models, the types of studies identified arederived from the report by the NATO SAS-054 Work Group termed ‘Methods and Modelsfor Life Cycle Costing’ (NATO RTO, 2007) and from the reported experiences of certaincountries. When appropriate, the results are presented as a histogram ordered by frequency of occurrence (or Pareto diagram), which is conceptually related to the Pareto principle, accord-ing to which a relatively small number of causes provoke or reflect most problems, or prefer-ences, respectively.

As shown by the replies to Item 2-1, about 80%, accumulated (right-hand axis) of repliesbeyond a doubt recognize the analysis of alternatives as the most commonly utilized individualoption (left-hand axis).

In the case of Item 2, almost half of the respondents (48%) confirmed the existence of aprocedure for determining costing methods over the life cycle of the product (t = −0.18; d.f. =28; p = 0.8564). In the replies to Item 2-3, 55% of the respondents acknowledged the lack of a standardized procedure for determining their technical baseline and their costs baseline (t =0.55; d.f. = 28; p = 0.5864). It should be taken into account that these two baselines provide abenchmark for measuring the progress of, and variations affecting, the majority of weaponsprogrammes. For Item 2-4, although the maintenance of a historical reference is a necessary

condition for learning from experience, almost half of the respondents (48%) had no depart-ment specifically responsible for this function (t = −0.18; d.f. = 28; p = 0.8564), and in thiscase, it is to be expected that the learning curve will be minimal and the same deficiencies thatoccurred in earlier programmes or projects will again be present.

523


18/37


As regards the computing systems utilized (Item 5), the MS Excel spreadsheet is by far themost commonly employed for cost estimation. Considering that the use of this instrumentnormally depends on the skills of the user, who must development calculation ‘algorithms’, itis then necessary to examine the degree of computer automation, in practice.

To conclude this block, let us refer to procedures for the verification and validation of cost-

ing models (Item 6). It can be seen that most countries (76%) do not possess a procedure forvalidating the costing model used (t = 3.20; d.f. = 28; p = 0.0034), which could influence thedegree of reliability of the information managed.

The results obtained do not support the involvement hypothesis, H2. Although there was atendency for standardized procedures to be used, the percentage of countries using them, incomparison to those that did not, was not significant. Most countries do not possess a proce-dure for validating the costing model used, and therefore the procedures used do not allowthem to estimate costs as required under LCC methodology. Among the possible causes of this shortcoming, perhaps the absence of political will is paramount; a mere declaration of intentions is no substitute for a resolute decision to facilitate the use of the LCC technique.Despite the results obtained for Block 1, current legislation does not adequately address thetechnical requirements imposed by the LCC methodology, especially as regards proceduresfor revision, verification and validation. These findings suggest there is a need to issue devel-opment rules whereby, in accordance with expert opinion, definitions are given of the proce-dure phases to be employed. In the absence of such rules, the LCC technique would losereliability and verifiability.

Turning to Block 3 (Cost structures), the answers to Item 3-1 show that most of the respond-ing countries do not possess a standardized (t = 3.20; d.f. = 28; p = 0.0034), codified WorkBreakdown Structure (WBS). This situation implies that, for each individual project, it isnecessary to invest time and effort in defining a structure to provide the contract, the produc-tion and subsequently the support for systems. The replies to Item 3-2 are similar to those for

Item 1, with 63% of respondents stating that their country has no system for presenting LCCwith a standardized, codified structure but the difference is not statistically significant (t =1.32; d.f. = 28; p = 0.1987). With respect to Item 3, the answers given show that, with a slightdifference regarding the fields for sub-activities and main product, greater consideration isgiven to the fields of the phases of the programme or of the project and main activity. Ingeneral, there is a consensus in accepting the codification method currently employed in theUnited Kingdom.

There is no empirical evidence in support of hypothesis H3. Therefore, there is no standard-ized cost breakdown structure with the characteristics needed for the LCC methodology to besuccessfully implemented. Far from what might be inferred from the results of Block 1, most

of the respondent countries do not possess a standardized or codified Work BreakdownStructure (WBS). These results may reflect the fact that the staff responsible for regulatingprocedures for evaluating military investments lack a long-term view of asset management,and that the corresponding regulations were issued more in view of maintaining control of legality than considering efficiency in the use of resources. The absence of such a standardizedstructure could make it impossible to carry out comparative analyses between differentprojects and to perform ex-post evaluation.

Block 4 concerns risk management and uncertainty. Very few countries (21%) possess anobligatory, standardized procedure for evaluating risk and uncertainty (t = −3.83; d.f. = 28; p= 0.0007) and 28% lack a systematic process for this purpose (t = 1.83; d.f. = 28; p = 0,0785).

There are differences within all regions, which shows that risk management and uncertainty isan instrument that has proved difficult to introduce and gain acceptance in the administrationof defence systems acquisition programmes. Sensitivity analysis is one type of risk analysis,and our results show that its use in the decision taking process presents the same pattern as for

524


19/37


Item 4-1 (t = −9.00; d.f. = 28; p = 0.0000). The results of sensitivity analysis are normallypresented in the form of three values: low, medium and high. This technique makes it possiblefor the decision taker to have more information at hand, which is utilized in the simulation of scenarios depending on the relevant variables defined. In the case of LCC analysis, the resultis not just a figure, but rather a distribution of possible values between the lower limit (opti-

mistic estimation) and the upper limit (pessimistic estimation), while the most probable LCCvalue represents the baseline.The results obtained do not support hypothesis H4. An extended analysis of the results from

Block 2 shows that very little use is made of risk and uncertainty estimation instruments; thismakes it difficult or impossible to gain full benefit from the LCC technique. The absence of sufficiently developed standards and procedures, together with the limited external evaluationof the estimation made, could be impeding the performance of risk and uncertainty analyses.This situation could bring about a loss of objectivity in the results obtained from the LCC tech-nique, to the extent that the latter might be applied on the basis of subjective criteria, thusgenerating mistrust of its accuracy among the staff responsible for evaluating militaryinvestments.

With respect to the final block, concerning management indicators, the first item (5-1)examines the existence or otherwise of an external body (governmental or private) thatperforms control audits and cost management regarding weapons systems acquisitionprogrammes. Most of our respondents (72%) stated that audits were carried out of defenceprogrammes by an organization that was independent of their respective defence ministries. Inresponse to Item 5-2, 83% of respondents stated that the economic-financial indicators mostfrequently used for economic assessment based on LCC were the Net Current Value and theNet Present Cost.

However, the question arises as to why two countries, according to their replies, didnot utilize any kind of indicator to carry out economic evaluation. From conversationsheld with the respondents, depending on the circumstances, this situation appears todepend on the relative importance granted to the cost/effectiveness analysis. When thecriterion of ‘effectiveness’ prevails over that of ‘cost’, the latter variable ceases to besignificant in decision taking. And cases may also occur in which some countries priori-tize the development of their domestic defence industry (typically, with regard to thenaval military industry), or the fulfilment of international cooperation agreements (e.g.regarding the Eurofighter, or Airbus A400M aircraft), or some other kind of political-stra-tegic concern.

Finally, the replies to Item 5-3 illustrate the utilization, to a greater or lesser degree, of theintegrated management system ‘Earned Value Management’ (EVM), which was implementedby the US Department of Defense in 1996 and is currently regulated by Directive 20069 of theDefense Contract Management Agency. Although six countries did not answer this question,in general terms it can be seen that EVM constitutes one of the most-widely employedsystems, which would apparently make it a key factor in the successful preparation and controlof weapons systems acquisition programmes.

We see, thus, that the existence of external bodies to supervise the progress of weaponsprogrammes could favour the transparency of LCC-based evaluations, although it would alsobe desirable to test the accuracy of forecasts made, and to improve it for future programmes.The presence of these bodies might depend upon the greater emphasis laid on regulations andprocedures in the control of legality, rather than that of efficiency. Moreover, the development

of control audits for such programmes could also favour the use of the LCC technique; thismight be a profitable line for future research regarding the performance indicators used andthe corrective measures adopted.

525


20/37


FIGURE 3 Results Item 1.1

FIGURE 4 Results Item 1-2


526


21/37





527


22/37




528


23/37





529


24/37





530


25/37





531


26/37


Analysis by Country Segments

Many different criteria can be employed to determine a reasonable, logical number of groupsto obtain, ranging from the most intuitive, based on a visual examination of the dendogram, toother more analytical ones, such as the examination of the amalgamation schedule. In anycase, all these criteria are open to criticism, and so we have preferred to make use of variouscritieria, as a means of validating the conclusions reached.

The above figure represents the first of the above criteria. The dendogram case shows,specifically, that at a distance of about 65–75% of the total distance, three groups may clearlybe distinguished. A good solution is that which considers a sudden jump, or gap, in the coef-ficient or measures the linkage distance. According to the corresponding amalgamationschedule, the largest jump takes place in phases 1 and 2, which is indicative of a two-clustersolution, and then in phases 2 and 3, indicating a three-cluster solution. As a solution that is

limited to a small number of clusters would give rise to unreliable conclusions, we decided toextract three clusters.

We now examine the characteristics of each of the clusters obtained.


TABLE VI Results Item 5–3

Frequency PercentageValid

percentage Accumulated percentage

Cost Baseline 21 22% 24% 24%

Actual Cost for Work Performed (ACWP) 17 18% 19% 43%

Budgeted Cost for Work Performed (BCWP) 13 14% 15% 58%

Budgeted Cost for Work Scheduled (BCWS) 13 14% 15% 73%Cost Performance Index (CPI) 10 11% 11% 84%

Schedule Performance Index (SPI) 7 7% 8% 92%

Cost Variance (CV) 7 7% 8% 100%

No opinion 6 0%

532


27/37


Group 1. Countries with regulatory norms, unlikely to use the approach of logistic support cost analysis. This group is constituted of Belgium, Denmark, France, Greece, Netherlands,Italy, Turkey, Colombia, Japan and NATO. A large majority of its members (80%) have anational standard for regulating life cycle stages and control milestones for weapons systemprogrammes. The majority of its members (60%) have a body that is responsible for maintain-

ing databases reflecting the costs and prices incurred in weapons systems programmes, to beused in audits or future programmes. Virtually all of the countries (90%) have developed astandard for presenting the Work Breakdown Structure (WBS), in accordance with a standard-ized, codified configuration. Concerning a codification of the life cycle cost to identify the costelements of a weapons systems programme, the principal product (WBS) or the sub-productderived from the WBS, this component was not found to be of the utmost priority (as,however, it was for the other groups).

Group 2. Countries with regulatory norms, systematized for cost evaluation with externalaudits. This group is made up of Germany, Spain, Norway, the UK, Sweden, Bulgaria,

Canada, the USA, Argentina, Brazil, Australia and Israel. All of its members recognize theexistence of a category-based classification in its weapons systems programmes. 83.33% of them present a national standard to regulate the life cycle stages and control milestones forthese programmes. On the other hand, none of them carries out budgetary planning in thisrespect. Virtually all (91.67%) of the countries in this group have a procedure for declaring,revising, adjusting and updating their evaluation methods during the different stages of theproduct life cycle, or for establishing the bases, restrictions, general assumptions or scenariosfor developing the cost evaluation of a weapons system programme. The same percentage of countries has developed standards for the presentation of a WBS. A further structured andcodified norm for LCC has been developed by half of the countries in this group. Two thirds

of the countries in this group possess a systematic process for performing risk and uncertaintyevaluation, but this procedure has not been standardized. In weapons acquisitionprogrammes, the net current value is the most commonly used management indicator(83.71%) in LCC-based economic evaluation (rather than the net present cost). The cost base-line and the real cost are the control-associated management indicators that are used (by100% of the group).

Group 3. Budget planning countries, with no regulatory norm, and focused on the analysisof alternatives and of integrated logistic support. This group is formed by Slovakia, Esto-nia, Latvia, Lithuania, Romania, Chile and Ecuador, and is characterized by the absence of a

national norm for regulating life cycle stages and control milestones for weapons systemprogrammes. In consequence, each country tends to develop its own system for identifyinglife cycle stages. There is no organization responsible for maintaining a database of thecosts and prices incurred in weapons system programmes, which might be available forfuture consultation. In this group, 71.43% have not developed a norm for presenting theWBS. There is a total absence of norms regarding LCC. 71.43% of its members recognizethe existence of a systematic process for risk and uncertainty estimation, but this procedureis not standardized. Furthermore, 85.71% do not perform a sensitivity analysis to facilitatedecision taking when faced with uncertainty; nor is any external organization employed tocarry out control audits and cost management regarding weapons system acquisition

programmes. For most of these countries (71.43%), the net current value is not utilized as amanagement indicator in economic evaluation, with preference being given to the netpresent cost for this purpose.

533


28/37


FIGURE 21 Dendogram. Ward’s method. % mismatch

FIGURE 22 Groups and components. Generalized EM analysis

534


29/37


CONCLUSIONS

When diverse alternatives are offered, the results of decisions on weapons investment projectsmay depend on the criteria used for selection, and on whether the evaluation is made withexclusive reference to the acquisition cost or whether LCC is employed.

In general, applicable regulations and procedures concerning the evaluation of militaryinvestments, at an international level, do not meet the technical requirements for the system-atic application of an LCC methodology, especially because of the following shortcomings:(a) the absence of methods for reviewing, fitting, validating and updating the cost estimation

methods employed; (b) the absence of normalized procedures for defining bases, restrictionsand general assumptions regarding cost estimation scenarios; (c) the non-existence of stan-dardized cost breakdown structures; (e) the lack of systematic processes and statistical toolsnecessary for the reliable measurement and evaluation of risk and uncertainty; (f) the absenceof historical databases on previous projects, from which experience lessons might be drawn.

Apparently, at a theoretical level, current administrative systems for evaluating invest-ments in military weapons are reasonably well advanced for facilitating the application of LCC. Most of the countries surveyed claim to use acquisition-category classifications,possess independent evaluation bodies, carry out regular cost control audits and have anational norm for estimating life cycle costs. Moreover, most of these countries use their

own standards for representing life cycle stages, and the types of studies most often used arethe analysis of alternatives, integrated logistic support and LCC itself. In practice, cost esti-mation procedures are limited in number and insufficiently regulated by legal norms, whichproduces a negative effect on the standardization and verification of input data and on the

FIGURE 23 Positioning of each strategic group of countries

535


30/37


generalized acceptance of a protocol for model validation. In addition, there is a generalizedlack of instruments for migrating data from databases, and a limited use of suitably standard-ized structures for costing projects, together with a limited, and inadequate, application of methods for risk and uncertainty management. Accordingly, the programme stage and theprincipal activity are two fields where it would be most necessary to consider establishing an

appropriate codification of LCC, while the standardization of procedures and upgrading theskills of civilian personnel are the two main areas where the need for improvement is recog-nized at an international level.

These deficiencies could be caused, at least in part, by some or all of the following factors:political resolution in this field has not been sufficient to produce the necessary developmentof regulation, at the technical level; current standards are oriented more towards the control of legality than towards the more efficient use of resources; the LCC methodology is still insuf-ficiently rooted in organizational structures; the limited technical preparation of staff respon-sible for cost estimation; the limited participation of civilian personnel in evaluationprocesses; and the scarcity of initiatives to design specific performance indicators for applica-tion of the LCC technique in evaluating investments in weapons programmes.

As concerns the implications of these findings, the definition and implementation of solu-tions to these limitations, which are less severe in North America and Western Europe, are thechallenges – and, at the same time, opportunities for improvement – that must be addressed bycountries wishing to introduce and systematically apply the LCC methodology. Achieving thismight entail: (a) changes in the organizational, regulatory and functional structures of admin-istrative systems for investment evaluation; (b) the adoption of innovative technical measuresin the models employed, especially as regards costing and risk and uncertainty management;and (c) the creation of technical standards for the application of LCC criteria, together with thecreation and maintenance of appropriate databases. Moreover, if countries reached agreementsto pool their LCC experiences with respect to military investments, and established lines of

collaboration with companies in the private sector that are experienced in the application of the LCC methodology to capital assets, this could favour the generalised use of this techniquein the area of national defence.

At a more detailed level, most of the countries surveyed possess an acquisition category-classification system, including participating bodies, financial resources and validating author-ities, thus enabling efforts to be distributed on the basis of objective criteria. With respect tocosting methods and models, the most commonly used are the analysis of alternatives and of integrated logistic support. Around half of the countries have no standardized procedure forsetting their technical baseline and costline, and a similar proportion have no organizationresponsible for database maintenance, which would allow them to learn from experience.

By geographic areas, the countries of North America and Western Europe (and especiallythe former) present the greatest concentration of best practices. In both regions, all the countriessurveyed had an acquisition category-classification system, and most of them used specific costestimation software (with the others using mainly Excel). However, and unlike the case of North America, most of the countries in Western Europe had an independent organization toperform investment evaluations. Many of the South American countries lacked an acquisitioncategory-classification system and/or an independent evaluation mechanism; these character-istics were shared with many Central European countries. In general, cost estimation proce-dures for defence programmes are not sufficiently supported by regulatory norms, althoughthis situation is less severe in the NATO countries. A striking contrast was that while a majority

of countries declared they used LCC, in practice they lacked standardized procedures forimplementing this methodology, and there was a generalized absence of procedures for thereview and validation of the estimation models and cost models adopted. Beyond a doubt, thisdeficiency could have a very negative impact on the reliability of the information handled.

536


31/37


Although most of the countries surveyed claimed to possess a standard for estimating theLCC of military investments, with the exception of the countries of North America, very fewpossessed a specific legal norm that enabled them to present a work breakdown structure basedupon a standardized and codified configuration. The countries of South America and CentralEurope revealed the greatest needs for improvement as regards the correct implementation of

the LCC technique, while those of North America presented fewest deficiencies, followed bythe countries of Western Europe. The non-existence of a procedure for risk and uncertaintyestimating represents a considerable deficiency in Western Europe and in South America, whilein Central Europe this type of procedure does exist, but is not standardized; the same circum-stance is to be observed in North America, although standardization is more common there.

On the other hand, the cluster analysis applied enabled us to isolate three groups of countrieswith different positionings as regards their evaluation of investments in national defence. Thegroups identified present the following characteristics: (a) countries with regulatory norms,unlikely to use the approach of integrated logistic support analysis or cost analysis; (b) coun-tries with regulatory norms, that systematically estimate costs and are externally audited; (c)budget planning countries, with no regulatory norms, that concentrate on the evaluation of alternatives and on integrated logistic support analysis.

Finally, our findings raise issues of interest for future research in the field of LCC, such as:(a) an analysis of factors that influence the use of LCC methodology, with respect to the inci-dence of financial and accounting variables; (b) extension of the research scope, analysingpractices in countries with different socioeconomic systems and different administrativecultures; (c) the design of cost calculation methods that would enable reliable estimations tobe achieved; and (d) a study of the disclosure made of control audits for weapons programmes,monitoring the corrective measures adopted.

ACKNOWLEDGEMENTS

This research was carried out with financial support from the Regional Government of Andalusia (Spain), Department of Innovation, Science and Enterprise (Research projectnumber P09-SEJ-5395) and from the Spanish National R&D Plan through research projectsnumbers ECO2010-20522-ECON-FEDER and ECO2010-17463-ECON-FEDER (Ministryof Science and Innovation) The authors also wish to thank the collaboration of all persons andinstitutions that answered the survey.

References

ANAO – Australian National Audit Office (1998) Life-cycle Costing in the Department of Defence. Sidney: Australia.Arnold, S. (2005) Standard systems engineering in the business environment. Conference of the Finnish Chapter of

the International Council on Systems Engineering. Helsinki, Finland. 26 May 2005.Arteaga Martinez, B.P. (2008) Estadistica Inferencial. Departamento de Estadistica. Universidad Carlos III de

Madrid, Spain.Barnes, N.M.L. and Wearne, S.H. (1993) The future for major project management. International Journal of Project

Management 11(3) 135–142.Barringer, H.P. and Weber, D.P. (1996) Life cycle cost tutorial. Fifth International Conference on Process Plant

Reliability. Houston, Texas, USA.Berenson, M.L. and Levine, D.M. (1992) Estadística Básica en Administración. Editorial: Prentice Hall. Mexico.Bhuiyan, N. and Thomson, V. (1999) The use of continuous approval methods in defence acquisition projects.

International Journal of Project Management 17(2) 121–129.Biermann, W. (2005) Análisis del Sistema de Asignación de Recursos para la Defensa de Chile y proposición de una

Metodología. Pontificia Universidad Católica. Santiago, Chile. Cited by Maturana Mena (2007).Birkler, J., Graser, J.C., Arena, M.V., Cook, C.R., Lee, G. and Lorell, M. (2001) Assessing Competitive Strategies for

the Joint Strike Fighter (MR- 1362-OSD). Santa Monica, USA.Blanchard, B.S. (1979) Life cycle costing – a review. Terotechnica (1) 9–15.

537


32/37


Blanchard. B.S. (1995) Ingeniería Logística. Serie Azul ISDEFE. Madrid, Spain, October.Blanchard. B.S. (1999) Cost management. In Handbook of Systems Engineering and Management, edited by A.P.

Sage and W.B. Rouse. New York: Wiley.Blanchard, B.S. and Fabricky, W. (1991) Life-Cycle Cost and Economic Analysis. Prentice Hall International Series

in Industrial and Systems Engineering. New Jersey, USA.Boudreau, M. and Naegle, B. (2005) TOC considerations in key performance parameters and beyond. Defense

Acquisition Review Journal, Defense Acquisition University (DAU), February–March, 109–121.

Calobrisi, C. (2006) Meaningful metrics for total life cycle costs. Defense Acquisition, Technology and Logistics Magazine. Defense Acquisition University. Washington, USA.Chou, Mei-Wu (1995) Decision Support Systems for Source Selection in the Procurement of Military Equipment.

Naval Postgraduate School. Monterey, USA.Christensen, P.N., Sparks, G.A., and Kostuk, K.J. (2005) A method-based survey of life cycle costing literature

pertinent to infrastructure design and renewal infrastructure design and renewal. Canadian Journal of Civil Engineering 32 250–259.

Ciobotaru, T. (2008) NATO guidance for life cycle cost. Conference on Life Cycle Management in NATO: The Basis for Efficiency in Project Management and Life Cycle Costing. Brussels, Belgium. 17–18 June 2008.

Clark, G., Piperias, P. and Traill, R. (1999) Life-Cycle Cost/Capability Analysis for Defence Systems. Aeronautical & Maritime Research Laboratory. Defence Science and Technology Organisation (DSTO). Australian Departmentof Defence. Melbourne, Australia.

Coleman, R.L., Summerville, J.R., DuBois, M. and Myers, B. (2000) Risk in cost estimating: general introduction &the BMDO approach. Briefing at the 33rd Annual DoD Cost Analysis Symposium. Williamsburg, USA

Dameron, M.E., Pullen, C.L., Summerville, J.R., Coleman, R.L. and Snead, D.M. (2001) NAVAIR Cost Growth:Overview of Analysis. Briefing at the Aeronautical Systems Center Industry Cost and Schedule Workshop.Wright-Patterson Air Force Base, USA

De Jong, G. and Van Westing, B. (2004) Fiscalización y armas, el papel de la Entidad Fiscalizadora en la supervisiónlegislativa de las Fuerzas Armadas. Revista EUROSAI, Organización de Entidades Fiscalizadoras Superiores deEuropa 10 40–44.

Defense Contract Management Agency (2006) Earned Value Management Implementation Guide. Virginia, USA:Defense Contract Management Agency, October.

Defense System Management College (1997) Acquisition Logistics Guide. Virginia, USA: Defense SystemManagement College.

Department Of Defense – DOD (2003a) Instruction 5000.2 Operation of the Defense Acquisition System. Washington,USA. 12 May 2003.

Department Of Defense – DOD (2003b) Extension to the Project Management Institute’s (PMI) A Guide to theProject Management Body of Knowledge (PMBOK Guide). Washington, USA.

Department Of Defense – DOD (2008) Joint Publication 1-02 Dictionary of Military and Associated Terms.Washington, USA, March.

Dhillon, B.S. (1989) Life Cycle Costing: Techniques, Models and Applications. Department of Mechanical EngineeringUniversity of Ottawa. London: Routledge.

Eskew, H.L. (2000) Aircraft cost growth and development program length: some Augustinian propositions revisited. Acquisition Review Quarterly 7(3) 209–220.

Fadem, B. (2008) High-Yield Behavioral Science (High-Yield Series), Hagerstown, MD: Lippincott Williams & Wilkins.Ferrín B.G. and Plank R.E. (2002) Total cost of ownership models: an exploratory study. Journal of Supply Chain

Management 38(3) 18–29.Fleischer, G. (1998) Ingeniería Económica. Manual del Ingeniero Industrial, Sección 9. Ed. McGraw-Hill, 4th edn.

Mexico.Flowe, R. (2002) Integrating Evolutionary Acquisition with PPBS. Virginia, USA: Defense Acquisition University.Flyvbjerg, B., Holm, M. and Buhl, S. (2002) Underestimating costs in public works projects. Error or lie? Journal of

the American Planning Association 68(3) 279–295Freeman, R.G. (1995) Adquisición y Apoyo Continuado durante el Ciclo de Vida. ISDEFE Serie Azul. Spain.Gates, J. and Greenberg, M. (2006) Cost Estimating Methodologies. Teaching Note. Virginia, USA: Defense Systems

Management College.Government Accountability Office (GAO) (2006) Defense Acquisitions: Actions Needed to Get Better Results on

Weapons Systems Investments. Washington, DC: General Accountability Office, GAO-06-585T. April.Gupta, Y. and Chow, W.S. (1985) Twenty-five years of life cycle costing – theory and applications: a survey. The

International Journal of Quality and Reliability 2(3) 51–76.Hartley, K. and Parker, D. (2003) Transaction costs, relational contracting and public private partnerships: a case

study of UK defence. Journal of Purchasing & Supply Management 9(3) 97–108.Hill, T. and Lewicki, P. (2006) Statistics: Methods and Applications. A Comprehensive Reference for Science,

Industry, and Data Mining. OK, USA: Statsoft.Hull, J.K. (1990) Application of risk analysis techniques in proposal assessment. International Journal of Project

Management 8(3) 152–157.

International Organization for Standardization (2002) ISO 10007 Quality Management, Guidance for Configuration Management. Geneva, Switzerland: ISO.

Kirkpatrick, D.L.I. (1997) Rising costs, falling budgets and their implications for defence policy. Economic Affairs Journal 17(4) 10–14.

538


33/37


Kirkpatrick, D.L.I. (2000) Life cycle costs for decision support – a study of the various life cycle costs used at differ-ent levels of defence policy and management. Defence and Peace Economics 11 333–368

Kirkpatrick, D.L.I. (2004) Trends in the costs of weapon systems and the consequences. Defence and Peace Economics15(3) 259–273.

Kirkpatrick, D. (2008) The future of UK defence research. Defence and Peace Economics 19(6) 479–491Levin, R.I. (1988) Estadística para Administradores, 2nd edn. Mexico: Editorial Prentice Hall.Levin, J. and Levin, W. C. (2001) Fundamentos de Estadística en la Investigación Social. Mexico: Oxford University

Press.Luque, T. (2000) Técnicas de análisis de datos en investigación de mercados. Ed. Pirámide. Madrid, Spain.Marín Ibáñez R.; Pérez Serrano, G. (1985) Pedagogía Social y Sociología de la Educación. Madrid, Spain: UNED.Martin, R. and Evans, D. (2000) Reducing costs in aircraft: the metals affordability initiative consortium. JOM – Journal

of the Minerals, Metals & Materials Society 52(3) 24–28.Markowski, C.A. and Markowski, E.P. (1990) Conditions for the Effectiveness of a Preliminary Test of Variance. The

American Statistician 44(4) 322–326Masiello G.L. (2002) Reliability, the life cycle driver: an examination of reliability management, culture and prac-

tices. MSc Thesis, US Naval Postgraduate School, Monterey, California.Maturana Mena, M.A. (2007) Análisis del Sistema de Asignación de Recursos para la Defensa de Chile y proposición

de una Metodología. Santiago, Chile: Pontificia Universidad Católica.Melese, F., Franck, R., Angelis, D. and Dillard, J. (2007) Applying insights from transaction cost economics to

improve cost estimates for public sector purchases: the case of U.S. military acquisition. International Public Management Journal 10(4) 357–385

Ministry of Defence (2005) The Acquisition Handbook – 6th edn. October. London, UK: MOD.Ministry of Defence (2006) JSP-507 Guide to Investment Appraisal and Evaluation. December 2006. London, UK:

MOD.Moore, D.M. and Antill, P.D. (2001) Integrated Project Teams: the way forward for UK defence procurement.

Journal of Purchasing & Supply Management 7(3) 179–185.Morris, P.W.G. and Hough, G.H. (1993) The anatomy of major projects: a study of the reality of project management.

UK: Wiley. Cited by Rodrigues and Bowers (1996).Naciones Unidas (2001) El uso de indicadores socioeconómicos en la formulación y evaluación de proyectos sociales

– Aplicación Metodológica.NAO – National Audit Office (2003) Ministry of Defence: Through-Life Management. The Stationery Office.NAO (2004) Ministry of Defence: Major Projects Report 2004. The Stationery Office.NATO (2002) Life Cycle Management in NATO. Edition 2/2002. Brussels, Belgium. (retrieved from www.nato.int)NATO (2008) SAS-076 NATO Independent Cost Estimating and its Role in Capability Portfolio Analysis. In

Execution. (retrieved from www.nato.int)NATO (2009) SAS-069 NATO Code of Practices for Life Cycle Costing. (retrieved from www.nato.int)NATO CALS – Continuous Aquisition and Life Cycle Support Office (2000) NATO CALS Handbook. Version 2,

Brussels, Belgium.NATO RTO – Research and Technology Organisation (2003) SAS-028 Cost Structure and Life Cycle Cost (LCC) for

Military Systems. Brussels, Belgium: NATO RTO. (retrieved from www.nato.int)NATO RTO – Research and Technology Organisation (2007) SAS-054 Methods and Models for Life Cycle Costing.

Brussels, Belgium: NATO RTO. (retrieved from www.nato.int)NATO Standardisation Agency (1992) Allied Administrative Publication AAP-20 Phased Armaments Programme

Systems (PAPS). Brussels, Belgium: NATO. (retrieved from www.nato.int)NAVSEA – Naval Systems Command- (2005) Cost Estimating Handbook. Cost Engineering and Industrial Analysis

Division (SEA 017) USA.Nelson. J.R. (2003) Life-Cycle Costing and Life-Cycle Analysis. George Mason University, USA: Institute for

Defense Analyses.

Pugliese, D. (2007) Cost overruns and delays spark calls for Canadian Procurement Oversight. Defense News Journal 1.Rand Corporation (2006a) ‘Why Has the Cost of Navy Ships Risen?’ Published for the US Navy with the collaborationof M.V. Arena, L. Blickstein, O. Younossi, and C.A. Grammich. Santa Monica, USA.

Rand Corporation (2006b) ‘Historical Cost Growth of Completed Weapon System Programs’. Published for the USAir Force with the collaboration of M.V. Arena, R.S. Leonard, S.E. Murray, O. Younossi. Santa Monica,USA.

Rodrigues, A. and Bowers, J. (1996) The role of system dynamics in project management. International Journal of Project Management 14(4) 213–220.

Rush, B.C. (1997) Cost as an Independent Variable: Concepts and Risks. Acquisition Review Quarterly—Spring1997. Virginia, USA: Defense Acquisition University (DAU).

Sipple, V., Whit

the application of life cycle costing

Documents