performance management of defence inventory

Performance Management of Defence Inventory 1

Performance Management of Defence Inventory Department of Defence

2

© Commonwealth of Australia 1996 ISSN 1036-7632 ISBN 0 644 This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without prior written permission from the Australian National Audit Office. Requests and inquiries concerning reproduction and rights should be addressed to the Publications Manager, Australian National Audit Office, GPO Box 707, Canberra ACT 2601.


Canberra ACT XX October 1997 Dear Madam President Dear Mr Speaker In accordance with the authority contained in the Audit Act 1901, the Australian National Audit Office has undertaken a performance audit in the Department of Defence and I present this report and the accompanying brochure to the Parliament. The report is titled Performance Management of Defence Inventory. Also presented is a report titled Defence Quality Assurance - Preliminary Study. Yours sincerely

P. J. Barrett Auditor-General The Honourable the President of the Senate The Honourable the Speaker of the House of Representatives Parliament House Canberra ACT

4

The Auditor-General is head of the Australian National Audit Office. The ANAO assists the Auditor-General to carry out his duties under the Audit Act to undertake performance audits and financial statement audits of Commonwealth public sector bodies and to provide independent reports and advice for the Parliament, the Government and the community. The aim is to improve Commonwealth public sector administration and accountability.

Auditor-General reports are available from Commonwealth Government Bookshops. Recent titles are shown at the back of this report. For further information contact:

The Publications Manager Australian National Audit Office GPO Box 707Canberra ACT 2601 telephone (02) 6203 7537 fax (02) 6203 7798

Information on ANAO audit reports and activities is available at the following Internet address: http://www.anao.gov.au

Audit Team

Tina Long Kate Shaw


Contents

Abbreviations vii

Part One ix

Summary xi Defence Efficiency Review xii Audit objective and criteria xii Overall audit conclusion xiii Key findings xiii Potential audit impact xix Departmental response xx Recommendations xxi Part Two 1

1. Background 3 ANAO preliminary study 3 JCFADT review of ADF stockholding 4 Defence Efficiency Review and Defence Reform Program 4 Cost of operating the supply chain 5 Potential savings identified by the Defence Efficiency Review 6 Recommendations of the Defence Efficiency Review 6 Important to consider characteristics of Defence inventories 7 Role of performance management in Defence Reform Program 7 Audit objective, criteria and focus 7 2. A balanced approach to performance management 9 Benefits to performance measurement 9 Leading logistics performance measurement characteristics 9 A framework for analysis 10 Measures not balanced 11 Impact of lack of balance 15 Development of logistics performance management in Defence 15 Need for strategic approach to performance measurement 16 Performance management across related functions 19 Logistics performance management ‘centre of excellence’ 20

6

3. Performance management of supply chain processes 21 Possible areas of process assessment in inventory management 21 Increasing awareness of need for process assessment 26 Need for formalised processes 26 Process ‘owners’ important to achieving integrated performance 27 4. Managing inventory as an asset 29 Total cost analysis 29 Impact of carrying costs 30 Focussing on expenditure effectiveness 33 Asset performance measures 33 Segmenting inventory to target stockholding strategies 35 Need for costs to be related to capability output 37 Cost attribution to support enhanced accountability 39 5. Internal and external performance targets 41 Need for consistent methodology 41 Supplier performance 43 6. Logistics information systems 45 Information systems not integrated 45 Need to consider performance management in system design 46 Defence executive information system 48 Data reliability and validity 49 Lessons of SDSS project 50 7. The use of benchmarking to enhance performance 53 Methods of benchmarking 53 Current benchmarking practice in Defence 54 Internal comparisons 55 External comparisons 56 Need for better coordination of benchmarking activities 58 8. Achieving cultural change 61 Personnel involvement in performance measurement 61 Training and education important for change 62 Use of reported information 63 Benefits of a strategic approach 64


Part Three 66

Defence Quality Assurance - Preliminary Study 66

Part Four 73

Appendix 1 ANAO analysis of a sample of Defence supply chain performance measures against better practice criteria

Appendix 2 Description of Defence supply chain performance measures analysed

Appendix 3 ANAO comparison of performance information availability in Defence with leading practice

Appendix 4 Performance audits in the Department of Defence

8

Abbreviations ABC Activity-based costing ABM Activity-based management ABW Air Base Wing ADF Australian Defence Force AF Log Comd. Air Force Logistics Command AIMS-BDS Advanced Inventory Management System - Breakdown Spares AMF Avionics Maintenance Flight, 501 Wing, Amberley ANAO Australian National Audit Office AOC Air Officer Commanding Air Force Logistics Command APS Australian Public Service AQC Australian Quality Council Army Log Comd. Army Logistic Command AUSMIMPS Australian Standard Materiel Issue and Movement Priority System CDF Chief of the Defence Force CDP Central Dispensing Point COMSPT Commander Support Australia CSF Critical Success Factor CSP Commercial Support Program Defence Department of Defence DER Defence Efficiency Review DER Date Equipment Required DLA United States Defense Logistic Agency DLRP Defence Logistic Redevelopment Program DNSDC Defence National Storage and Distribution Centre, Moorebank, NSW DRP Defence Reform Program EDI Electronic data interchange EIS Executive Information System FDC Freight Distribution Centre GAO United States General Accounting Office HQADF Headquarters Australian Defence Force


JCFADT Joint Committee of Foreign Affairs, Defence and Trade KRA Key Result Area LBSS Logistic Business Systems Section LCI 8-1 Logistic Command Instruction Business Management 8-1 LPR Logistics Performance Report MAB Management Advisory Board MIAC Management Improvement Advisory Committee Michigan study ‘World Class Logistics: The Challenge of Managing Continuous Change’, Global Logistics Team, Michigan State University, 1995 MIMS Mincom Information Management System MOA Memorandum of Agreement NSC Naval Support Command NZDF New Zealand Defence Force PIR Performance Information Review PMB Program Management and Budgeting RD Requirements determination RI Repairable item SDSS Standard Defence Supply System SLA Service Level Agreement SOR Statement of Requirement SSRP Supply Systems Redevelopment Project STRIDE Strategic Inventory Decision Environment study SQL Structured Query Language US Defense United States Department of Defense

10

Part One

Summary and Recommendations

12

Summary

A critical element in the preparedness of the Australian Defence Force (ADF) is the availability of various items at the time and place required to support operations or training. For the purpose of this audit, inventory management has been defined as the logistics process, or supply chain, through which items flow from initial supplier to end customer - the units, ships, bases and personnel. It encompasses requirements determination, procurement, warehousing, inventory control, transport and disposal. That supply chain forms an integral part of the provision of logistics support to the ADF.

The current Defence inventory of spares, consumables and repairable items (excluding explosive ordnance) is valued at approximately $3.9b, with $1b expended annually on further procurement and the maintenance of repairable items by external contractors.

Defence holds stocks of these items primarily to ensure their availability in support of ADF operational and training requirements. In some cases, this has meant Defence procuring sufficient stocks to maintain equipment to the end of its forecast working life (life-of-type). Retirement of equipment by other countries has also created opportunities to purchase bulk spares at very low cost. Other items are held as insurance stocks - items not normally expected to be used, but held as a matter of prudence.

Defence Efficiency Review The Defence Efficiency Review (DER) highlighted that there are significant opportunities to improve the management of Defence inventory. The DER logistic review team concluded that current levels of operating stock are far too high. This was considered to reflect a


‘just in case’ culture which had been exacerbated by a poor, disjointed and unresponsive distribution system.

The DER said that, in the time allowed it, its ‘best guess’ was that a book value of $1b of the $6.3b of items managed through the Standard Defence Supply System (SDSS) (which incorporates the $3.9b of spares, consumables and repairable items considered in this audit) could be dispensed with. However, it acknowledged that what that stock might fetch at sale was difficult to assess. This estimate was based on analysis that showed that approximately $4.2b in non-explosive stores recorded no usage for 1995-96.

The DER also estimated that one-off savings of between $100m and $140m and annual savings of between $61m and $89m could be generated through the expanded use of commercial management practices such as vendor-held stock.

The DER recommended establishment of a joint support command, a review of the value of Defence inventory and development of a more efficient storage and distribution system. Other findings and recommendations relating to changes in both resource and inventory management practices were included in the DER Secretariat papers, which the DER Senior Review Panel considered to be advisory only.

The experience of the DER review teams highlighted the need for improved information to support reliable analysis of the inventory, particularly in identifying the effectiveness with which the supply chain has operated in support of the ADF.

Similar difficulties identified in the ANAO’s preliminary study of Defence inventory management conducted in 1996 led the ANAO to conclude that an audit could add most value by focussing on the overarching issue of performance management. Improvements in this area would improve the future ability of Defence to identify practical opportunities for improvement in its inventory management.

14

Performance management is the process by which an organisation translates its vision and strategy into a set of objectives and measures which link all levels of the organisation, perform the internal and external monitoring of operations and provide organisational feedback and learning. It is an approach that has to be understood and accepted at all levels of an organisation. Its effective implementation takes time and should be embarked upon at the earliest opportunity.

Defence has now embarked on the Defence Reform Program (DRP) on the basis of the DER findings and recommendations. The strategic logistics planning conducted in support of the DRP should provide a structure for the success of the extensive change management process now under way within Defence.

The ANAO considers that an important element in the implementation of Defence’s strategic logistics planning would be an effective performance management strategy and framework that enables both its effectiveness and efficiency to be measured and managed. The absence of such a framework has contributed to many of the inefficiencies in inventory management identified by the DER.

Audit objective and criteria Based on the premise that what is not measured is not managed and that gaps in reliable information are often covered by holding large safety stocks, the objective of this audit was to ascertain whether Defence performance management strategies and practices contribute to the effective and efficient management of the supply chain. In particular, it focussed on examining the extent to which the latter demonstrate identified world-class practices.


Overall audit conclusion The ANAO found that, despite increased focus in this area in recent years, Defence performance management practices used in regard to the supply chain for its $3.9b inventory of spares, consumables and repairable items do not reflect best practice. As a result, Defence managers are not provided with adequate information or incentives to ensure their decisions are based upon consideration of the efficiency and effectiveness of the total supply chain in the provision of logistics support to the ADF. Many of the factors that have inhibited adoption of leading practices have related to the fragmented nature of logistics management within Defence.

Although there have been efforts to improve inventory management, each of the three Services has sought to address these issues separately. This may be seen to be counter to the unified approach on Defence strategy that has been increasingly recognised in recent years as desirable. There has been little attention to overall management of the performance and costs of the supply chain, including inventory carrying costs.

There have been worthwhile but isolated attempts to improve the performance management practices applied to the Defence supply chain, but a cultural change is needed to bring Defence practices closer to those identified as best practice. Implementation of the DER recommendations, particularly those concerned with achieving greater integration in the provision of logistic support and with relating expenditure on logistic support directly to the achievement of Defence outputs, would create an opportunity to address many of those factors. Implementation of the recommendations in this report would assist Defence in maximising the benefits accrued from those changes.

16

Key findings The ANAO identified a number of issues that would assist Defence in developing a more effective logistics performance management framework (incorporating supply chain management) which would improve its ability to realise the benefits envisaged under the DRP. Fundamental among these is the need for Defence to take a more strategic, coordinated approach to developing and implementing credible and useful performance measures for its supply chain.

The improvement of Defence’s logistics performance, consistent with the performance of leading organisations, is strongly dependent on quality and timely information management and supporting technologies. Notwithstanding the advances Defence has made in the area of logistics information systems, there is still much that can be achieved at both the strategic and operational levels. Progress in this area must be guided by the strategic development of information systems which, in turn, must be consistent with higher logistics planning. This has not always been the case.

The major issues detailed in the audit report which require Defence attention are summarised below.

Chapter 2 - A balanced approach to performance management

• Best practice in supply chain performance management is characterised by the use of a balanced and integrated set of performance measures. This involves a focus on comprehensive supply chain performance in satisfying customer requirements and expectations.

• ANAO analysis of a sample of 354 Defence supply chain related performance measures demonstrated that, despite increased attention to performance management issues in recent years, the measures


currently used in the management of Defence inventory do not provide managers with a full and balanced picture of relevant performance.

• The information produced predominantly relates to the quantitative, lagging measurement of results, with only 30 per cent of the sample identified as leading measures. Over 90 per cent portrayed an internal perspective. This reflects the approach historically used in many organisations but does not reflect the practice emerging in world-class organisations.

• Achieving a more effective balance, coverage and integration of the issues addressed by supply chain performance measures would support a stronger focus on improving overall performance in providing credible support to operational customers with benefits to all parties.

• Efforts to incorporate leading performance management practices within the Defence supply chain would be enhanced by:

- developing a clearly-articulated, uniform strategy for logistics performance management (including supply chain management); and

- establishing a single ‘centre of excellence’ responsible for deploying that strategy to each level of the organisation.

Chapter 3 - Performance measurement of processes

• An area in which there is particular need for development is in the use of performance measures that assist in pro-active management of key supply chain processes. World-class organisations have shown that, if performance within key processes improves, improved results will follow. Less than twenty per cent of the Defence supply chain performance measures analysed by the ANAO could be assessed as providing information regarding the operation of processes.

18

• Areas in which there is particular potential to generate efficiencies through the expanded use of process assessment include the repairable item ‘maintenance pipelines’ operated by each of the three Services and the procurement processes of inventory managers.

• An important element in determining how the performance of processes can be measured and influenced is the formalisation and documentation of those processes and of the key drivers within them. There has been progress in mapping the logistical processes within the existing Service logistic organisations, but there is not yet a full or common understanding of the key logistic processes undertaken within Defence.

• Completing the documentation of its logistics processes, including those related to the supply chain, would assist Defence in both identifying examples of better practice and in achieving greater standardisation for greater efficiency. Given that standardisation has been identified as one of the key capabilities that drive integration in world-class organisations, this will be an important element in the successful migration to a joint support command.

• Improving overall performance through enhanced integration would also be assisted by identification of a single point of authority with responsibility for the end-to-end, cross-functional performance of each key supply chain process.

Chapter 4 - Managing inventory as an asset

• Leaders in the practice of integrated logistics management emphasise the use of total cost analysis to make informed trade-offs between functions to improve overall performance. There has been little focus within Defence on developing a management approach for inventory from this perspective.


• Of fourteen cost measures taken by world-class firms, the ANAO identified examples within Defence of four commensurate measures, relating to outbound freight costs, direct labour costs, comparison of actual costs to budget and cost trend analysis. No measures were identified relating to total supply chain costs. There was only limited analysis of the costs associated with various elements of the supply chain.

• The ANAO demonstrated the importance of understanding the more wide-ranging costs associated with inventory management decisions by considering, for example, the impact of inventory carrying costs. Applying a conservatively estimated carrying cost of 12 per cent suggests that Defence incurs total annual carrying costs of approximately $472m for its $3.9b inventory of spares, consumables and repairable items.

• The DER logistic review team commented that a focus within Defence on the achievement of expenditure against allocated budgets as a performance indicator has resulted in sub-optimal inventory management decisions. There has been little emphasis on assessing the effectiveness of that expenditure.

• Slow inventory ‘turns’ can be an indicator of excessive or inappropriate stocks. A turn is a measure of how many days it takes the inventory in the system to change on average. For example, if average inventory on hand is 10 weeks’ supply, inventory turn would be 5.2 times per year. As turnover increases, both inventory levels and associated carrying costs decrease.

• More use of measures of asset performance, such as inventory turn analysis, may provide greater incentive for Defence inventory managers to actively manage stock levels. For example, recent inventory turn analysis conducted by Air Force Logistic Command suggested that its average stock levels of breakdown spares could eventually be reduced by

20

up to 75 per cent, that is from 288 weeks to 65 weeks of stock. The inclusion of insurance and other intentionally stockpiled items, together with doubt regarding the accuracy of existing inventory valuation and pricing data, makes it difficult to identify accurately the savings that may result from such a reduction.

• From the information currently available, the ANAO estimated that reducing stocks of breakdown spares by 75 per cent could eventually generate annual carrying cost savings in the order of $46m, together with a $340m capital release.

• Defence inventory has not been comprehensively segmented to support the application of appropriately differentiated stocking strategies. Nor does the existing resource management framework provide inventory managers with the information needed to relate their decisions adequately to Defence preparedness and operational priorities. As a result, it is difficult for inventory managers to ensure resources and management effort are focussed on items of highest priority.

• The ANAO supports recommendations by the DER review teams directed at establishing a resource management system that focuses on relating all costs to the achievement of Defence outputs. Implementation of activity-based management would assist Defence in achieving this.

Chapter 5 - Internal and external performance targets

• Defence would benefit from using clearer performance targets to provide meaning and direction to logistics performance measures. It would also benefit from developing a sound and consistent methodology to guide determination of the levels of performance required for internal operations, functions market-tested under the Commercial Support Program (CSP) and the supply


of items through supply contracts such as vendor-held stock and prime vendor arrangements.

Chapter 6 - Logistics information systems

• Effective performance management of the Defence supply chain has been restricted by the inability of existing information systems to support integrated management. There may be over sixty information systems involved in the management of the supply chain. These systems are not integrated and lack connectivity. Functions and data are replicated. Information cannot be easily shared between them due to the use of different hardware, operating systems, data dictionaries and data base management systems.

• The development of information systems that ultimately do not provide required information in a useable form has been due in large part to a failure to consider performance management issues as an integral part of systems design and implementation.

• There are significant problems with the validity and accuracy of a number of important data elements such as price, stock level and lead time. This was highlighted during a data verification exercise conducted in conjunction with the DER in which the Services were asked to confirm the price, quantity, category, location and physical condition of the 10,000 line items on SDSS identified with the greatest total stock value. DER Secretariat papers indicate that this exercise produced a net reduction of $628m in inventory values when compared with values used in the preparation of the 1995-96 Defence financial statements.

• Although it provided a number of benefits, including a level of asset visibility never before achieved, there have been problems and dislocation associated with the implementation of the Standard Defence Supply System (SDSS). There were considerable modifications made to the commercial package on which it is based in the design of SDSS, primarily to

22

allow existing practices to continue. As a result, each of the Services has developed different methods for operating the common system and has retained different inventory management business practices.

• Many of the efficiencies expected to be generated through the use of automated functions have not been realised. Moreover, a significant proportion of the functionality offered by the system is not being used. Identifying the extent to which perceived inadequacies relate to the system itself or to the use being made of it can best be determined by a post-implementation review. Such a review would be important for documenting the issues associated with its introduction that should be avoided in future similar projects.

• Defence should continue its embryonic efforts to plan strategically for its logistics information management needs. Optimal results will only be achieved through a holistic approach to integrating the management of logistics and supporting information. To this end the DER logistic review team recommended that the new joint support command develop and implement a funded plan for the rationalisation of existing systems and the migration to joint, integrated logistics and administrative information systems. In implementing that recommendation, the ANAO considers that:

− a clear statement of performance management requirements should be articulated prior to, or in conjunction with, the development of a rationalisation strategy; and

− the validity and accuracy of important data elements should be comprehensively addressed.

• The use of data warehousing and executive information system (EIS) technologies offers considerable potential to bring together data from heterogeneous sources, thereby providing more meaningful information to various logistics


managers. Although these technologies, in themselves, cannot provide a panacea for Defence’s management reporting requirements, they would improve corporate oversight and support consistent, coordinated performance management of the supply chain.

Chapter 7 - The use of benchmarking to enhance performance

• There is considerable scope for Defence to help increase its use of both internal and external benchmarking to promote the consistent application of better logistics management practices across the organisation. This process would be assisted by:

− promulgation of a Defence logistics benchmarking policy that identifies clearly the processes and procedures to be followed; and

− identification of a body responsible for coordinating internal and external benchmarking activities and for promoting the development of expertise in the use of benchmarking techniques.

Chapter 8 - Achieving cultural change

• Performance management can play an important role in achieving significant change in Defence’s inventory management culture, and in the successful transition to a joint, integrated logistics organisation. However, there has been no coordinated strategy for addressing the training and educational issues involved in successfully introducing a strong performance management culture. Problems identified by the ANAO included:

− inadequate involvement of logistics personnel in the development of performance measures and targets; and

− inadequate training for personnel involved in both the development and implementation of performance measures.

24

• These problems have been reflected in a poor level of acceptance and awareness of performance management principles and techniques among logistics personnel. The ANAO considers that Defence would benefit from developing a performance management deployment program for the Defence support command to ensure these issues are comprehensively addressed.

Potential audit impact Although the benefits directly attributable to performance management are generally non-financial in nature, it is only through them that financial benefits can be realised. Implementing the audit recommendations would assist Defence in developing a well-constructed measurement system that would help to:

• communicate priorities to ensure everyone acts to achieve corporate objectives;

• channel the actions of individuals, helping them to manage their own performance;

• provide a means to influence and recognise performance improvement; and

• improve fact-based decision-making.

Potential benefits for inventory management in Defence include:

• better logistical planning and control;

• better buying decisions;

• more accurate and timely recognition of obsolescent materiel; and

• more focused accountability.

The ANAO estimated that even a five per cent reduction in the current stock levels of spares,


consumables and repairable items could yield annual savings in inventory carrying costs of at least $23m and a potential one-time capital release of $197m. In the ANAO’s view, this is the minimum possible outcome that could be delivered.

Implementation of the audit recommendations would help to create an environment and culture in which supply chain managers were better equipped to achieve such savings, while maximising the timeliness, relevance and quality of the support provided to the generation and maintenance of military capability and other Defence outputs.

Departmental response Defence agreed to all twenty-two recommendations made in the report and commented that:

The ANAO report reinforces and expands the Defence Efficiency Review (DER). The DER findings and recommendations are being implemented, where appropriate, through the Defence Reform Program (DRP).

As part of the DRP, Strategic Logistics Division has been raised to develop policy and plans for national support. Support Command Australia has been raised and tasked with providing integrated distribution and joint materiel support to the Australian Defence Force, thus providing a focus for the development of innovative, world class inventory management practices. Action is being taken to implement reforms in Defence logistics in general and in the management of the supply chain in particular.

2

Recommendations

Set out below are the ANAO’s recommendations with report paragraph references and an indication of the Defence response. The ANAO considers that Defence should give priority to Recommendation Nos. 1, 3, 5, 7, 9, 10, 11, 15, 18, 20 and 22. Priority recommendations are shown below with an asterisk.

Recommendation No. 1* (para 2.55)

The ANAO recommends that Defence develop a coherent logistics performance management strategy and framework, incorporating supply chain management, that:

a) ensures performance measures at each level are logically derived from overall objectives;

b) provides for an integrated, consistent and balanced set of performance measures cascaded throughout Defence; and

c) links core business processes at each level to the achievement of higher level objectives.

Defence Response

Agree.

Recommendation No. 2 (para 2.62)

The ANAO recommends that the Defence logistics performance management framework include strategies to:

a) assess the impact of decisions made by capital acquisition projects on the performance of the supply chain as a whole; and

b) provide for the appropriate consideration of those assessments in the inventory-related decisions of future acquisition projects.

Defence Response

2

Agree.


The ANAO recommends that, as part of a rationalisation of duplicated functions across the Service logistic and support organisations, Defence form a single performance management organisation to be charged with deploying its logistics performance management strategy to each level within the Support Command and across Defence as required.

Defence Response

Agree.


The ANAO recommends that the Defence logistics performance management strategy emphasise the development of a limited number of credible performance measures to assist in pro-active management of key supply chain processes at both the corporate and unit levels.

Defence Response

Agree.


The ANAO recommends that Defence:

a) evaluate the progress of each Service in the documentation of logistics processes;

b) develop and implement a standard approach for completing that process; and

c) where there are gaps in information, document the core processes to be undertaken within the Defence support command.

Defence Response

Agree.



The ANAO recommends that Defence identify single points of authority with responsibility for the end-to-end cross-functional performance of each key supply chain process.

Defence Response

Agree.


The ANAO recommends that Defence develop and implement:

a) a methodology enabling inventory management decisions to be supported by the wider consideration of relevant costs; and

b) a quality assurance program to monitor:

i) adherence to the methodology; and

ii) the effectiveness with which cost information is used to make trade-offs within the logistic system.

Defence Response

Agree.


The ANAO recommends that the Defence performance management framework for its inventory of spares, consumables and repairable items provide for the regular review of inventory asset performance through measures related to, for example, inventory turn, inventory levels (number of days’ supply) and obsolete inventory.

Defence Response

Agree.

4


The ANAO recommends that Defence develop and implement a policy for the application of segmentation techniques in the management of its inventory of spares, consumables and repairable items.

Defence Response

Agree.


The ANAO recommends that Defence review the appropriateness of existing stockholding policies for spares, consumables and repairable items (including the service levels nominated in SDSS and AIMS-BDS) based upon an appropriately framed profile of inventory characteristics.

Defence Response

Agree.


The ANAO recommends that Defence develop and implement an activity-based management methodology to support the integrated management of the Defence supply chain, having regard to Navy’s work on such methodology.

Defence Response

Agree.


The ANAO recommends that Defence revise existing internal supply chain performance targets to ensure that they are:

a) relevant to stated objectives;


b) based on a sound and consistent methodology; and

c) realistic and practical within existing resource constraints..

Defence Response

Agree.


The ANAO recommends that, in order to improve the visibility of costs and outputs, Defence review and standardise the mechanisms used for:

a) determining a limited number of credible supply chain performance measures and targets for inclusion in CSP contracts and service level agreements; and

b) identifying and disseminating lessons learnt in this regard from the market-testing of supply-chain functions.

Defence Response

Agree.


The ANAO recommends that Defence develop user-friendly guidance for inventory management personnel regarding the use of performance measures and targets in procurement contracts.

Defence Response

Agree.


The ANAO recommends that a clear statement of performance management requirements, based on a Defence logistics performance management strategy, be articulated prior to, or in conjunction with, the development of a rationalisation strategy for Defence logistics information systems.

6

Defence Response

Agree.


The ANAO recommends that Defence articulate a strategy and timetable, including milestones and performance indicators, to guide the development of a logistics executive information system to support consistent, coordinated performance management of the Defence supply chain.

Defence Response

Agree.


The ANAO recommends that Defence review the cost-effectiveness of implementing a ‘data warehousing’ approach for the management of common logistics data.

Defence Response

Agree.


The ANAO recommends that Defence:

a) develop and implement a strategy to validate the accuracy and reliability of all important data elements in logistic information systems; and

b) initiate strategies, including relevant performance measures, to ensure that the accuracy of logistic data is maintained and regularly validated.

Defence Response

Agree.



The ANAO recommends that Defence conduct a post-implementation review of the Standard Defence Supply System (SDSS) with a view to providing guidance for the introduction of future corporate logistics information systems.

Defence Response

Agree.


The ANAO recommends that Defence develop and promulgate a Defence logistics benchmarking policy that identifies clearly the processes and procedures to be followed.

Defence Response

Agree.


The ANAO recommends that Defence identify a logistics benchmarking coordination body responsible for coordinating:

a) the establishment and operation of internal benchmarking networks across Defence;

b) Defence participation in logistics benchmarking or process study activities with other defence and commercial organisations; and

c) the provision of adequate training and guidance regarding the conduct of logistics benchmarking and comparison activities.

Defence Response

Agree.

8


The ANAO recommends that Defence develop and implement a logistics performance management deployment program for the Defence support command that encompasses:

a) provision of appropriate education and training at all levels regarding the development and use of credible logistics performance measures, and

b) implementation and training in the use of appropriate performance analysis tools and techniques.

Defence Response

Agree.

2

Part Two

Performance Management of Defence Inventory


Figure 1-1 Value of Defence stocks of spares, consumables

and repairable items

Total value as at Feb 1997 - $3935 million

Air Force RI$2086m

Army S&C$229m

Army RI$147m

Navy RI$370m Navy S&C

$360m

Air Force S&C$743m

Legend:

S&C Spares and Consumables

RI Repairable Items

Source: Data supplied by Department of Defence

Figure 1-2 Defence expenditure on procurement and maintenance

of spares, consumables, repairable items

0

200

400

600

800

1000

1200

94-95 95-96 96-97(budget)

$m

Navy Air Force Army

Source: Data supplied by Department of Defence

4

1. Background This chapter provides background information on the management of the Defence supply chain, discusses the relationship of this report to the Defence Efficiency Review and Defence Reform Program, identifies the evidence suggesting the need for more effective performance management of inventory, and details the audit objective, criteria and focus. 1.1. A critical element in the preparedness of the Australian Defence Force (ADF) is the availability of various items at the time and place required to support operations or training. For the purpose of this audit, inventory management has been defined as the logistics process, or supply chain, through which items flow from initial supplier to end customer - the units, ships, bases and personnel. It encompasses requirements determination, procurement, warehousing, inventory control, transport and disposal. That supply chain forms an integral part of the provision of logistics support to the ADF.

1.2. Defence inventory consists of a complex range of items, including:

• weapon systems or major items of equipment;

• items consumed in support of either personnel or weapon systems such as repair spares, explosive ordnance and domestic consumables;

• repairable items (RI) - components that are themselves repairable; and

• ‘insurance items’ not normally expected to be used, but held as a matter of prudence.

1.3. This audit was primarily concerned with spares, consumables and RIs, including insurance items. Explosive ordnance was not

2

considered, having been covered in Audit Report No.5 1993-94 Explosive Ordnance and its follow-up in Audit Report No.8 1995-96. 1.4. As of 30 June 1996, the Departmental financial statements reported in the Defence annual report identified inventories of $3.582b ($3.635b as of 30 June 1995). Some items included within the scope of this audit are classed as property, plant and equipment in the Departmental financial statements. Defence advised that, as of February 1997, the value of its stocks of spares, consumables and RIs was approximately $3.9b, as shown in Figure 1-1. 1.5. Figure 1-2 shows annual expenditure on the procurement of these items, which exceeded $800m in 1995-96. Over $200m is also spent annually on the maintenance of RIs, excluding internal workshop costs (budgeted to rise to $233m in 1996-97). 1.6. Control and management of this inventory have been dispersed across a number of organisations, primarily within Army Logistic Command (Army Log Comd.), Air Force Logistics Command (AF Log Comd.) and Naval Support Command (NSC).

ANAO preliminary study 1.7. The ANAO conducted a preliminary study of Defence inventory management between August and October 1996. The study identified a number of initiatives being pursued within Defence to improve the management of inventory. The most notable have been: • the staged introduction since 1991 of the Defence National Storage

and Distribution Centre (DNSDC), through the Defence Logistic Redevelopment Project (DLRP), accompanied by rationalisation of single-Service warehouses; and

• migration between 1993 and 1995 from single-Service systems to the Standard Defence Supply System (SDSS) under the Supply System Redevelopment Project (SSRP).

1.8. Individual Service initiatives have also seen improvement in inventory management. AF Log Comd. in particular has been working to develop a strategic environment for inventory management. This work is based on the outcomes of a 1994 Strategic Inventory Decision Environment study (STRIDE) which found: • an inventory management culture in which personnel could not

directly identify the effect their efforts had on weapon system availability;


• business processes falling short of commercial best practice; and

• existing information systems that limited performance rather than being ‘enablers’.

1.9. Initiatives pursued by Air Force to improve its ability to forecast and manage inventory requirements have included development of a decision-support system for breakdown spares, re-engineering of its RI pipeline and improved budgetary processes. 1.10. Army Log Comd. has concentrated on mapping its processes, introducing quality management concepts and increasing its use of innovative arrangements with suppliers. Navy has implemented an inventory optimisation program to identify surplus or obsolete items. Since 1991, 296,000 line items have been disposed of, for a revenue of $7.9million. 1.11. The preliminary study indicated that, despite these efforts, there was still considerable scope for improvement. Potential areas identified included: • requirements determination;

• procurement practices and strategies; and

• storage and distribution systems and strategies.

1.12. However, the study also indicated that improvement in these areas was being inhibited by inadequacies in management information. The ANAO considered that the absence of relevant and reliable information was also likely to make it very difficult for an audit to identify accurately potential efficiencies in these areas. Therefore, it was concluded that greatest value could be added by focussing on the overarching issue of performance management. Improvements in this area would improve the future ability of Defence to identify practical opportunities for improvement in inventory management. The audit of performance management of Defence inventory commenced in November 1996.

JCFADT review of ADF stockholding

1.13. In 1992 the Joint Committee of Foreign Affairs, Defence and Trade (JCFADT) conducted an inquiry into stockholding and sustainability in the ADF. Although not directly relevant to the particular scope and focus of this audit, the ANAO had regard for the findings and

4

recommendations set out in the report of that inquiry, Stockholding and Sustainability in the Australian Defence Force (December 1992).

Defence Efficiency Review and Defence Reform Program 1.14. On 15 October 1996 the Minister for Defence announced the Defence Efficiency Review (DER) under the chairmanship of Dr Malcolm McIntosh. When releasing its report, Future Directions for the Management of Australia’s Defence, and its Addendum in April 1997, the Minister announced that a Defence Reform Program (DRP) based on the 70 findings and recommendations of the report would be implemented as soon as possible.

Indications of need for better inventory performance management

1.15. Many of the findings of the relevant DER review teams coincided with those that had arisen during this audit. In particular, they supported the need for improved management of Defence inventory. The DER logistic and regional support review team concluded that current levels of operating stock are far too high. This was considered to reflect a ‘just-in-case’ culture which had been exacerbated by a poor, disjointed and unresponsive distribution system.1

1.16. A principal reason for this finding was the very slow movement identified in the inventory. Approximately $6.3 billion of non-explosive stock involving 1.6 million line items relating to inventories, minor capital equipment and combat hardware is managed through SDSS. The DER identified that $4.2 billion of that stock had shown no movement in 1995-96.2 Data provided to the ANAO confirmed this trend, with no movement recorded in 60 to 76 per cent of line items in the Services’ wholesale warehouses during 1996.

1.17. Slow movement is often an indicator of large quantities of unnecessary or obsolete stock. This appears to be the case to some extent for many items in Defence. Analysis by the DER showed that as at 30 June 1996 stock held for 21 per cent of items on SDSS (with a total value of $836 million) was 10 times greater than total usage for 1995-96. Over 50 per cent had a ratio of greater than 2 to 1.

1 Defence Efficiency Review Logistic and Regional Support Review Team Final Report, 1997, p 3 2 Report of the Defence Efficiency Review, 1997, op cit, p 43


1.18. Effective inventory management also requires managers to be able to identify whether stock levels are sufficient to satisfy requirements, particularly in regard to critical or priority items. As at 30 June 1996 there was no stock held for 48 per cent of line items managed through SDSS. This includes a number for which no further stocking is intended, such as items made obsolete. Others have not been stocked due to funding shortfalls for inventory acquisition.

1.19. Analysis by the DER identified that in January 1997 there were 71,854 requisitions (or demands) in SDSS unable to be satisfied from current stock, involving stock valued at $146.6m. The ANAO also noted that the satisfaction rate for low priority stores in Army Log Comd. ranged from about 20 per cent during the Christmas 1995 period, to 39 per cent by June/July 1996.

Cost of operating the supply chain 1.20. The most recent attempt to identify the full costs associated with operating the Defence supply chain occurred as part of an extensive study conducted by consultants Arthur Young in 1989. It identified an annualised supply services cost of $2.1b. There have been significant changes in the structure of Defence supply services since the Arthur Young study, most notably the changes associated with the SSRP and DLRP.

1.21. The DER identified $3.1b in personnel and direct costs associated with logistics support. However, this included costs of personnel involved in functions not directly associated with the supply chain, while not including consideration of indirect costs, such as facilities. The DER team reported that performing detailed analysis on the potential for greater efficiency in logistics was made particularly difficult by the absence of relevant data.3 It found the costs of Defence logistics processes difficult, and in some cases impossible, to determine. This mirrored the experience of a 1995 internal review of ADF freight movement which found there was no method of determining the total cost of the distribution function.

1.22. These difficulties meant that a significant investment of resources would have been required by the ANAO to identify fully the

3 Future Directions for the Management of Australia’s Defence, Addendum to the Report of the Defence Efficiency Review, Secretariat Papers, 1997, p 357

6

costs associated with the current Defence supply chain. The ANAO did not consider this to be a cost-effective exercise within the scope of this audit, particularly given the significant changes proposed by the DER.

Potential savings identified by Defence Efficiency Review 1.23. The DER estimated that one-off savings of between $100m and $140m, and potential annual savings of between $61m and $89m, could be generated through the expanded use of commercial management practices such as vendor-held stock. The one-off savings relate to a moratorium on the purchase of items prior to shifting to such arrangements. This did not include analysis of the relative costs of such strategies.

1.24. The DER report also commented that, in the time allowed it, its ‘best guess’ was that a book value of $1b of the $6.3b of items managed through SDSS could be dispensed with.4 A significant proportion of that inventory relates to spares, consumables and RIs.

1.25. The experience of the DER review teams highlighted the need for improved information to support reliable analysis of the inventory. The report acknowledged that it is difficult to assess what the sale of surplus stock might fetch or what reduction in the cost of ownership might result.5 In conjunction with the DER the Services were asked to confirm the price, quantity, category, location and physical condition of the 10,000 line items on SDSS identified with the greatest total stock value. DER Secretariat papers indicate that this exercise produced a net reduction of $628m when compared to values used in the preparation of the 1995-96 Defence financial statements.

Recommendations of the Defence Efficiency Review 1.26. In relation to inventory the report of the DER recommended that:

4Future Directions for the Management of Australia’s Defence, Report of Defence Efficiency Review, 1997, p44 5Report of Defence Efficiency Review, 1997, op cit p44


• a military Support Command arrangement be established with the specific objective of reducing the size of organisations providing logistic and administrative support;

• the value of the ADF inventory be reviewed to ensure its accuracy; and

• Defence, with the assistance of industry, should develop a more efficient storage and distribution system which can accommodate its operational requirements.

1.27. The ANAO supports these recommendations. With the formation of a single support command, the previously separate Service logistics and support functions will, for the first time, come under the control and management of single commander. Commander Support Australia (COMSPT) will report to Chief of the Defence Force (CDF) and, as the primary customers, the Service Chiefs. However, many of the detailed conclusions that the ANAO considers important if Defence is to improve its inventory management are contained in the DER Secretariat papers, including recommendations for:

• establishment of a joint, integrated logistics policy staff at the strategic level;

• rationalisation of inventory holdings;

• a resource management system that identifies, attributes and monitors the real costs of delivering outputs; and

• the rationalisation and migration to joint integrated information systems.

1.28. Referring to the Secretariat papers, the Senior Review Panel of the DER commented that ‘…some of the more detailed recommendations made in these documents should be regarded as being only advice to the Secretary and CDF in their task of implementing the outcomes of the Review.’6

Important to consider characteristics of Defence inventories

6 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p 1

8

1.29. Analytical review, such as that conducted by the DER, provides a ‘prima facie’ case of the need for improved management of Defence inventory. However, as the DER acknowledged, care is needed in interpreting such analysis. Many items that appear surplus actually reflect intentional stockpiling in response to operational or economic imperatives.

1.30. The principal purpose for which Defence holds inventories is to avoid stock-outs that would cause stoppages to mission-critical functions. Others relate to the need to:

• ensure life-of-type availability and take advantage of opportunity buys. In some cases, as the last operators, Defence has had to purchase sufficient spares to maintain equipment for its forecast life. Retirement of equipment by other countries has also created opportunities to purchase bulk spares at low cost, such as for the F-111 aircraft;

• hold ‘insurance’ stocks (as described above);

• gain quantity discounts which, although necessitating the holding of apparently ‘surplus’ stock for a time, will reduce the unit costs of the acquired inventories; or

• reduce order costs by reducing the number of orders placed.

1.31. Therefore, a complete appreciation of the performance of Defence inventory management can be achieved only by including consideration of effectiveness issues, such as the relationship between stock held and the military capability required to be supported, the nature and priority of each item, and the conditions affecting its supply or disposal. This is the sort of analysis that the Defence performance management framework should support.

Role of performance management in Defence Reform Program 1.32. In outlining its logistics implementation strategy, the addendum to the DER report stated that a performance measurement system containing service level agreements and key performance indicators would be


negotiated with all customers.7 However, this was the only direct reference to logistics performance management.

1.33. Experience in many organisations has demonstrated that the creation of a shared information resource maintained as part of continuous performance measurement is a key element in successful change management.8 Performance management is the process by which an organisation translates its vision and strategy into a set of objectives and measures which link all levels, and which perform the monitoring of operations to provide organisational feedback and learning. They communicate strategic direction, motivate change and form a basis to plan, budget and structure the organisation and control results. It is an approach that has to be understood and accepted at all levels of an organisation. Its effective implementation takes time and should be embarked upon at the earliest opportunity.

1.34. In order to successfully implement reform in its supply chain, Defence should develop a comprehensive performance management framework. This audit report is intended to provide support to the reform process by examining practices from which Defence would benefit in the performance management of its inventory.

Audit objective, criteria and focus 1.35. Based on the premise that what is not measured is not managed and that gaps in reliable information are often covered by holding large safety stocks, the objective of this audit was to ascertain whether Defence performance management strategies and practices contribute to the effective and efficient management of the supply chain. In particular, it focussed on examining the extent to which the latter demonstrate identified world-class practices.

1.36. The ANAO did not seek to define the supply chain performance management model or metrics most appropriate for Defence. Such a task was beyond the resources of the audit. More importantly, Defence’s underlying performance management philosophy should support development of a framework appropriate to its environment.

7 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p 368 8 World Class Logistics: The Challenge of Managing Continuous Change, The Global Logistics Research Team, Michigan State University, Council of Logistics Management, 1995, p 14

10

1.37. The ANAO approach was to extract the supply chain performance management practices of leading organisations to define a set of audit criteria which could be used to analyse Defence practices. The ANAO engaged the services of a consultant, Mr Robert Easton of Price Waterhouse Urwick, to research and prepare a report on those better practice principles and to assist the audit analysis. The audit criteria developed through this approach were discussed with Defence and received a high level of acceptance.

1.38. Particular use was made of the 1995 report of a research project on world-class logistics conducted by Michigan State University. The ANAO also drew upon:

• published audit reports by the General Accounting Office (GAO) on United States Department of Defense (US Defense) inventory management;

• a study of best practices for supply system management conducted by consultants Arthur Andersen in 1993 as part of a management audit of New Zealand Defence Force (NZDF) materials accounting and management (an internal report provided by NZ Defence Force);

• a statement of best practice on inventory management produced in 1993 by the Office of Public Management in the NSW Premier’s Department; and

• other general research.

1.39. Audit matters were discussed with Defence throughout the audit. A paper consolidating the findings from the audit was discussed with Defence in July 1997. Fieldwork in Headquarters Australian Defence Force (HQADF), the Service logistic and support commands, the DNSDC, selected logistics units and quartermaster stores was substantively conducted between November 1996 and May 1997, involving interviews, file examination, and comparison of Defence practices with leading practices. The audit was conducted in conformance with ANAO Auditing Standards and cost $382 700, including $77 800 for advice from consultants.

1.40.

2

2. A balanced approach to performance management

This chapter examines the clarity, relevance and coverage of performance measures used to manage Defence inventories. It assesses the current focus of performance measures within Defence and highlights that, to drive superior supply chain performance, an integrated and balanced view or set of measures is necessary. It also examines the reasons for delays in the adoption by Defence of best practice in supply chain performance management.

Benefits of performance measurement 2.1. In 1995 the Global Logistics Research Team at Michigan State University conducted a study on world-class logistics involving 3693 organisations across the world, including Australia (Michigan study). The third stage in a ten-year research stream, it was aimed at understanding how the world’s best firms achieve and maintain their logistical excellence. Performance measurement was identified as a key competency that, together with positioning, integration and agility, must be simultaneously achieved to enable an organisation to develop logistics as a core competency. It found that:

Comprehensive, almost compulsive, performance measurement in terms of functional assessment, process assessment and benchmarking are essential to world class logistics… In fact, world class logistics firms view measurement as a critical competency and they demonstrate proficiency at a substantially higher level than their competitors.9

Non-financial benefits of measurement give rise to financial benefits

9 The Global Research Team Michigan State University, 1995, op cit, p. 218, 332

2

2.2. Although the benefits directly attributable to performance management are generally non-financial in nature, it is only through them that financial benefits can be realised. A well-constructed measurement system can:

• communicate priorities to ensure everyone acts to achieve corporate objectives;

• channel the actions of individuals, helping them to manage their own performance;

• provide a means to influence and recognise performance improvement; and

• improve fact-based decision-making.

2.3. In terms of inventory management, potential benefits can include: • better logistical planning and control;

• better buying decisions;

• more accurate and timely recognition of obsolescent materiel; and

• more focused accountability.10 Leading logistics performance measurement characteristics 2.4. According to international research, the principal supply chain performance measurement characteristics displayed by world-class organisations include:

• a strong customer orientation, supported by measuring performance from the perspective of the customer and other external sources to augment traditional internal measurement (a balance of measures taken from external and internal perspectives);

• increased use of qualitative information to support a focus on quality and effectiveness (a balance of measures providing qualitative and quantitative information);

10 Management Audit of NZDF Materials Accounting and Management, Volume 1, Executive Report, 1993, Arthur Andersen, p 14


• complementing the traditional measurement of results and outcomes with measurement of the internal and external processes through which they are achieved (a balance of results and process measures); and

• the increased use of measures that provide a more immediate insight into process performance and assist in the early identification of potential problems in order to improve the outcomes achieved (a balance of leading and lagging measures).

2.5. Leading organisations have recognised that achieving the outcomes required by end-customers can only be realised through the integrated performance of the supply chain, and that achieving this requires an integrated and balanced performance measurement framework covering all relevant business processes. For example, a major supply chain benchmarking study conducted in 1994 reported that ‘…the business performance metrics used to drive operational improvement must support a balanced view and a “balanced metric” framework is necessary to measure supply chain performance.’11

A framework for analysis 2.6. The ‘Balanced Scorecard’ model, first proposed by two Harvard professors, Kaplan and Norton, in the early 1990s, promotes an integrated approach to performance measurement. It looks at performance from four perspectives:

• what is the outcome we must deliver? (customer perspective);

• what internal processes must we excel at? (internal business process perspective);

• can we continue to improve? (innovation and learning perspective); and

• how do we look to our stakeholders? (accountability and financial perspective). 12

11 Stewart, G. Supply Chain Performance Benchmarking Study Reveals Keys to Supply Chain Excellence, Logistics Information Management, 1995, 8:2, p. 38. 12 Kaplan, R.S and Norton, D.P., The Balanced Scorecard - Measures that Drive Performance, in Harvard Business Review, January-February 1992.

4

2.7. This approach enables organisations to balance and prioritise the sometimes conflicting demands of these perspectives. Figure 2-1 illustrates the type of issues covered by each perspective in a ‘better practice’ balanced scorecard for inventory management.

Figure 2-1 Illustrative better practice inventory management balanced

scorecard

FINANCIAL PERSPECTIVEFINANCIAL PERSPECTIVE

“How do we look to ourstakeholders”

CUSTOMER PERSPECTIVECUSTOMER PERSPECTIVE

“How do we look to ourcustomers”

ORGANISATIONAL LEARNING &ORGANISATIONAL LEARNING &GROWTHGROWTH

“How can we sustaininnovation, change and

improvement

INTERNAL BUSINESS PROCESSINTERNAL BUSINESS PROCESS

“What Business Process mustwe excel at, which create

value”

VISIONVISIONAND STRATEGYAND STRATEGY

What is thestrategic/operational

vision we seek toachieve?

• Number of new ideas from staff• Number of supply chain/improvement

ideas implemented• Time to develop inventory

management tools• New systems implemented• No. of Quality initiatives in Place• Extent of Supply Chain information

available electronically• Extent of data capture at source• % of data on systems accurate

• Order cycle time, supply chain speed• Order cycle consistency• Forecast/Planning, Inventory accuracy• Capacity utilisation• Rates of obsolescence• Perfect order (DIFOTEF) fulfilment rate• Inventory Dwell rate• Internal/External Supplier management• Defect and Wastage rates• Asset performance, productivity, e.g. transport,

warehouse or order entry labour

• Delivery, In Full, On Time,Error Free (DIFOTEF)

• Customer complaints• Customer ratings, overall

satisfaction• Order accuracy• Backorder, stockouts, fill rates• Shipping errors• Order completeness• Delivery to Commit Date• Customer inquiry response

time

• Total suppy chain costs• Inventory carrying costs• Order management costs• Material acquisition costs• Network and MIS costs• Inventory obsolescence• Inventory days supply• Inventory turns• Cost of backorders, returns, service

failures

Source: Derived from Report on Better Practice Principles for the Performance Management of Large and Complex Inventories, Price Waterhouse Urwick, 1997, commissioned by ANAO

2.8. The ANAO adopted the balanced scorecard framework to capture the best practice principles described above in a working model against which Defence practices could be assessed. A sample of 354 inventory-related performance measures used or planned at the unit and

2

corporate level in each Service were analysed. Although this suggests an excessive number of measures are in use, no one manager is required to use all of these as the sample was taken from a number of organisations across Defence. However, the ANAO did note that a number of the Defence organisations visited were using multiple performance measures. Experience in well-performed organisations has shown that the use of a limited number of credible performance measures produces the best results. This is an issue that Defence should consider in its future development of performance measures. A summary of the ANAO’s analysis of the sample is given at Appendix 1, with the sources of the sample listed at Appendix 2.

Measures not balanced 2.9. Overall, the analysis demonstrates that the performance measures used in the management of Defence inventory do not provide managers with a full and balanced picture of relevant performance. They do not demonstrate the balance identified earlier as characterising leading logistics performance measurement.

2.10. The information produced predominantly relates to the quantitative, lagging measurement of results. Of the 354 measures analysed at Appendix 1, 97 per cent related to quantitative measurement, only 30 per cent were identified as leading measures and 80 per cent focussed on the measurement of results. Over 90 per cent portrayed an internal perspective. This reflects the approach historically used in many organisations but does not reflect the practice emerging in world-class organisations.

2.11. The lack of balance resulting from the current focus in Defence on the internal measurement of results is demonstrated in Figure 2-2. The ‘best practice objective’ also illustrated highlights the movement in leading practice toward a stronger customer focus through the use of external performance measurement, together with a greater emphasis on the assessment of key processes. In contrast, Defence measurement is clustered around the internal/results quadrant.

2

Figure 2-2 Comparison of Defence measurement against ‘best practice’ objective 13

Assessment of Level of Balance in Current Performance Measures

Legend: 1 Overall, 2 AF corporate, 3 Army corporate, 4 Navy corporate, 5 AF unit, 6 Army unit, 7 Navy unit

RESULT

EXTERNAL

PROCESS

INTERNAL

100 %

100 %

100 % 100 % 3

2

6

3

2

7

41

1 1

1

3

6 6

6

2

2

4

4 4

77

7RESULT

RESULTPROCESS

INTERNAL

EXTERNAL

‘Best Practice’ Objective:

To complement and balancetraditional internal measurement of results with measures thatfocus on external and processmeasurement, resulting in movementupwards and to the left into theother quadrantsof the scorecard.

5

5

5

53

Source: Derived from Report on Better Practice Principles for the Performance Management of Large and Complex Inventories, Price Waterhouse Urwick, 1997, commissioned by ANAO and ANAO analysis

13 There are four graph points for each observation category, e.g. Army Unit measurement has four observation points, Results/Internal, Results/External, Process/Internal and Process/External. The points are derived from the analysis at Appendix 1.

2.12. As Figure 2-2 highlights, an area requiring particular development in Defence is the use of process assessment. To date there has been an emphasis on the measurement of results with limited focus on the development of measures that assist in analysing enabling processes in order to influence those results. This has contributed to the

2

current focus on lagging and quantitative performance measurement in the Defence supply chain.

2.13. Focusing on achieving the movement identified as the ‘best practice objective’ in Figure 2-2 has helped leading organisations to make greater use of leading measures (through the more timely measurement of process performance) and in complementing quantitative measures with the more qualitative information that can be provided by external parties. As is discussed further below, a greater emphasis on external measurement would assist in providing Defence supply chain managers with improved qualitative information regarding their performance, particularly from the customer’s perspective.

2.14. Balancing results measures with process measures would also help Defence to shift measurement from lagging to leading measures. Over two thirds of the measures sampled were lagging measures which show the results of completed operations over time. Although important for measuring success in achieving overall objectives, they are not as useful for pro-actively managing performance. Leading measures provide a more immediate insight into process performance and assist in the early identification of potential problems.

2.15. This issue was recognised by a 1995 Army Log Comd. performance management seminar which identified that the Command’s measures were in significant need of reform to provide units with more pro-active performance information. Despite a major review in 1996, there has been little change, with primarily lagging results still reported bi-annually and some not analysed for up to ten months after the event.

Coverage of balanced scorecard perspectives

2.16. Figure 2-3 maps the analysis in Appendix 1 against the balanced scorecard framework. A single measure may provide information relating to more than one perspective.

Figure 2-3 Assessment of Defence measures against Harvard balanced scorecard

2

% of Measures Assessed inFinance Domain

Overall 17%Navy Corporate 20%Navy Unit 18%Army Corporate 38%Army Unit 8%Air Force Corporate 32%Air Force Unit 13%

% of Measures Assessed inOrganisational Learning andGrowth Domain


FINANCIAL PERSPECTIVEFINANCIAL PERSPECTIVE

“How do we look to ourstakeholders”

CUSTOMER PERSPECTIVECUSTOMER PERSPECTIVE

“How do we look to ourcustomer”

ORGANISATIONAL LEARNING & GROWTHORGANISATIONAL LEARNING & GROWTH

“How can we sustain innovation,change and improvement

IINTERNAL BUSINESS PROCESSNTERNAL BUSINESS PROCESS

“What Business Processmust we excel at, which

create value”

VISIONVISIONAND STRATEGYAND STRATEGY

What is thestrategic/operational

vision we seek toachieve?

% of Measures Assessed inCustomer Domain


% of Measures Assessed inInternal Process Domain


Sample SizeOverall (N = 354)Navy Corporate (N= 10)Navy Unit (N= 56)Army Corporate (N= 53)Army Unit (N= 49)Air Force Corp (N= 19)Air Force Unit (N= 131)

Source: ANAO analysis

2.17. The analysis indicates that there is not a balanced coverage of these areas within any of the Services. In particular, only seven per cent of the measures analysed could be classified as relating to organisational learning and growth. As a result, there are few measures which reveal how Defence can sustain innovation and improvement in its supply chain management. Although over half of the measures overall were assessed as falling into the internal business process domain (as defined in the Harvard model), 72 per cent of these measured the results of those processes. Only 28 per cent were focused on process assessment. This issue is discussed further in Chapter 3.

2.18. The apparent low coverage of the financial perspective (17 per cent) warrants comment. Although most Defence organisations monitor expenditure, the connection between the collection of financial data and the management of inventory is tenuous, with very limited use made of data collected to assess cost-effectiveness of inventory management. This is discussed further at Chapter 4.

2

Customer perspective

2.19. Supply chain managers in world-class organisations have an integrated, comprehensive view which includes an external perspective on their environment. For example, they spend considerable effort measuring customer satisfaction with their process from the customers’ perspective, through the use of surveys, panels, focus groups etc.

2.20. There is an apparent emphasis on customer service within Defence logistics organisations, with 32 per cent of the measures examined falling into this domain. However, as noted earlier, the measurement perspective is predominantly internal. Over 90 per cent of measures with a customer perspective are internally-generated indicators such as the rate of successfully fulfilling customer demands for items.

2.21. Although providing a gauge of supply chain performance, such measures have been found to fail to address two important concerns. Firstly, they look inward at operating performance rather than outward to customer requirements. This results in a focus on customer service, rather than customer satisfaction.

2.22. This was highlighted to Army when in 1995 it initiated arrangements for its customers to report the time elapsed from raising a requisition to receipt of the item. This identified that fewer items were being delivered within required timeframes than had been identified through measurement by logistics units. Recent analysis by Army Log Comd. found that this is still the case.

2.23. A lack of faith in the service they are likely to receive has led units in each Service to submit several demands for the same item or to misuse the priority system in the hope of getting faster service. Increased external measurement may assist to address these issues.

Measuring average performance lacks customer focus

2.24. The second problem with typical service measures is that they measure average performance against goals set at significantly less than 100 per cent. In other words, service is administered to achieve an average overall performance rather than being focussed on the particular requirements of each customer. This has been the approach largely used in Defence. A lack of customer focus has been found by Headquarters Army Log Comd. to be a major cause of failures by Army logistics units in achieving those average target levels.


2.25. The Michigan study found that world-class performers are far more likely to emphasise absolute or specific performance, translating it into specific customer statements. For example, reporting that 1000 packages were delivered damaged puts a different perspective on performance from reporting that 95.5 per cent were delivered damage-free.14

2.26. Leading practice now seeks to measure performance relative to the perfect order standard; that is, the number of orders delivered in-full, on-time and totally error-free. Although research has shown that performance in even the best organisations can be as low as 40 to 50 per cent, world-class firms are more likely to measure how well they perform relative to the perfect order standard.15 They also take a more integrated view of its achievement. For example, achieving 97 per cent in each of ten measures (fill rate, on time, damage free, etc), may appear to reflect excellent performance, but the combined effect is (.97)10 or about 73 per cent of orders being filled without a service failure.

2.27. Despite the important role the supply function plays in the capability of operational units, there is no such measurement within Defence. The majority of existing measures focus on individual segments of the supply chain.

Areas of continuous measurement in leading practice organisations

2.28. The Michigan study found that world-class organisations undertake continuous measurement in five functional areas - asset utilisation, costs, customer service, quality, and productivity.16 It compared the logistic performance of 3,693 firms and reported on the relative availability of information on a range of measures in each of these areas in firms performing well and in firms performing at lower levels. Appendix 3 summarises the findings of the Michigan study and includes an assessment, based on ANAO observation and analysis, of the extent to which commensurate measures are used within Defence. The assessment shows major gaps in Defence performance measures in these areas.

14 The Global Research Team Michigan State University, 1995, op cit, p. 237-240. 15 The Global Research Team Michigan State University, 1995, op cit, p. 237-240. 16 The Global Research Team Michigan State University, 1995, op cit, p. 224.

4

Impact of lack of balance 2.29. The ANAO analysis indicates that Defence performance measurement has paid inadequate attention to the multi-dimensional nature of the environment in which its inventory must be managed. The impact arising from this lack of balance is that managers are not provided with adequate information or incentives to ensure their decisions are based upon consideration of total supply chain performance. The incentive is for personnel to optimise their own performance, often resulting in sub-optimal supply chain performance overall.

2.30. Moreover, current measurement does not provide Defence with an adequate appreciation of what its total support performance has been. For example, in a minute outlining issues for consideration at the 1997 Command planning conference, the Air Officer Commanding Air Force Logistics Command (AOC) commented that when he tried to judge how well the Command had performed in the past year, he was:

…left with the uncomfortable feeling that we do not quite know; I can subjectively say that we must have done fairly well because we seem to have basically met the customers’ needs with less resources (both dollars and workforce content).

Development of logistics performance management in Defence 2.31. The ANAO observed awareness and enthusiasm in a number of areas across Defence for both the balanced scorecard approach and the leading principles described above, with logistics performance management (which incorporates supply chain activities) having advanced considerably in recent years in some areas. However, as the ANAO analysis demonstrates, progress could have been better.

2.32. The 1994 STRIDE study mentioned earlier found that, despite much discussion of the issue over the years, performance measurement remained a critical failing of the Air Force logistics system. Since then, Air Force has focused on the development of a Key Result Area (KRA)/Critical Success Factor (CSF) methodology which has helped to provide greater clarity of purpose and the dimensions in which it needs to measure outcomes relevant to its customers. Measurement of performance against the KRAs has focussed on monthly reporting by logistics management units against measures set out in the Logistic Performance Report (LPR).


2.33. However, an internal paper prepared for the 1997 AF Log Comd. planning conference acknowledged that there is ‘…still room for a quantum change in the way performance management is managed across the Command.’

2.34. Army Log Comd. has also focused on the development of a command-level performance measurement set, published in the form of Logistic Command Instruction Business Management LCI 8-1 (LCI 8-1). The latest version, developed in December 1996, is the first to include inventory management measures. However, there are currently very few inventory-related measures within HQ Army Log Comd., with the performance of national inventory managers monitored largely through expenditure against budget.

2.35. In April 1997 the ANAO was advised that the measures in LCI 8-1 are again under review due to doubts regarding their validity. Many were identified as flawed in that they are too rigid in their measurement and consequently skew or distort the data presented. Others do not measure what they purport to measure. In March 1997 a project was established to promote the development of quality management in Army Log Comd.

2.36. Until 1995, Naval Support Command (NSC) produced a Half Yearly Logistics Report, but it was discontinued as it failed to satisfy internal and external requirements. It was replaced by an electronic database networked throughout the Command providing access to a number of discrete sets of information. These are drawn from databases manually maintained by individual managers. There is no common approach to performance management within the Command.

2.37. Materiel managers within NSC acknowledge that the Command is only scratching the surface of performance management, arguing that they have been largely forestalled in this regard by inadequacies in the available information systems. Information they would like but which is not currently available includes the impact on ship availability arising from the items they choose not to buy and the costs associated with over stocking.

2.38. Efforts at developing supply-related performance measures are also being undertaken within various units across the Services. However, much of this work has been uncoordinated and conducted without the benefit of appropriate training or guidance. As a result, there is inconsistency in the performance measures used in various units for similar processes.

6

Factors inhibiting adoption of leading performance management practices

2.39. A number of factors have inhibited the adoption of leading performance management practices within Defence. These have included issues associated with:

• coordination between the allocation and expenditure of logistic funds and the determination and achievement of capability output priorities (Chapter 4);

• the ability of information systems to effectively support integrated logistic performance management (discussed in Chapter 6);

• the use made of benchmarking techniques (Chapter 7); and

• the Defence inventory and performance management culture (Chapter 8).

2.40. Progress has also been affected by the fragmented nature of logistics performance management within Defence. The ANAO identified two areas that would assist Defence in this area. These relate to the need for:

• a clearly articulated strategy for logistics performance management (including supply chain management); and

• a coordinated development effort, appropriately resourced.

Need for strategic approach to performance measurement. 2.41. The performance of any function, including those that make up the logistics supply chain, cannot usefully be managed in isolation from an understanding of the end objective to which they are working or the costs and benefits that arise from different levels of performance.

2.42. The balanced scorecard model referred to in Figure 2-1 places vision and strategy at its centre. This reflects the practice in leading organisations in which the logistics performance management framework is derived from its overarching strategy and vision. Performance measures link “the board room to the back room” by aligning objectives and goals and integrating performance management into the planning


process. International experience has pointed to the need for a uniform approach to measurement in order to achieve such linkages.

2.43. For example, a 1993 management audit by consultants Arthur Andersen found that few appropriate and comprehensive performance measures existed in the inventory management processes of the New Zealand Defence Force (NZDF). The Arthur Andersen report identified ways in which the NZDF could improve performance measurement, make better use of modern inventory management, distribution and warehousing practices, and take advantage of technological advances. The report provided guidance on how improved definition of performance objectives, making them more appropriate to the nature of each Service’s business, could create better goal congruence and management decision-making. It recommended that NZDF-wide, uniform policy and practices be developed in the area of performance measurement.17 The NZDF supply system is in the final stages of being redeveloped and fully integrated with financial systems.

2.44. The ANAO also noted that there has been significant development in recent years in the performance management framework used by the US Defense Logistic Agency (DLA). The DLA buys and distributes food, clothing, medical, electronic and industrial supplies and many other consumable items for the military Services. Its experience demonstrates the value that can be extracted from taking a structured, strategic approach to the development of performance indicators. Important lessons identified by the DLA were that:

• performance management must be communicated and supported throughout the organisation;

• identification of meaningful performance measures that accurately portray and predict the value of the organisation requires a shift in the cultural paradigm; and

• realisation of the value of output versus outcome information occurs only after the organisation validates and defends its existing historical indicators.18

Opportunity for greater strategic control of Defence performance management

17 Arthur Andersen, Volume 1, 1993, op cit, p 25-26 18 Defense Logistic Agency, Performance Plan, 1996, p 23

8

2.45. The DER logistic review team found that, although overarching strategic logistics planning guidance is available in the form of the Strategic Logistics Planning Guide, there is a perception of a lack of executive authority for logistics matters at the strategic level.19 This is reflected in the absence of a Defence logistics performance management strategy or model. Appendix 1 of this audit report includes no measures at the HQADF level, the ANAO having found little evidence of supply chain performance management at that level and no mechanism for assessing performance across Defence.

2.46. With no area responsible or accountable for the performance of the entire supply chain, there has been little attempt to provide comprehensive performance information across the chain. As the DER found, a complete and consistent view of the cost of functions which cross management boundaries is difficult to obtain.20

2.47. In general, existing logistics performance measures are not well integrated with stated business strategies or plans. For example, Army Log Comd. has recognised that its corporate performance indicators are not yet integrated with business plans, key business processes or corporate action plans within a continuous improvement context.

2.48. In support of the Defence Reform Program, including implementation of the DER recommendation for the establishment of a joint logistics policy staff at the strategic level, Defence is undertaking a comprehensive strategic logistics planning exercise. This should provide a structure for the success of the extensive change management process now under way within Defence. The ANAO considers that an important element in the implementation of Defence’s strategic logistics planning would be an effective performance management strategy and framework for the Defence supply chain. The absence of such a framework has contributed to many of the inefficiencies in inventory management identified by the DER.

2.49. Where development of a coherent strategy to guide performance management has been attempted, the benefits have been apparent. For example, AF Log Comd. has recognised the need for a model to help translate its broad KRAs into meaningful performance measures. Its model has attempted to marry the already developed

19 Defence Efficiency Review Logistics and Regional Support Review Team Final Report , 1997, p 25 20 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p 25


understanding within the Command of the existing KRA/CSF framework to the key principles of the balanced scorecard approach.

2.50. The ANAO also observed a positive impact from the development of performance management model on the enthusiasm for performance measurement in organisations like the Avionics Maintenance Flight (AMF) at Amberley and the DNSDC. AMF found that applying the balanced scorecard approach had forced it to ensure it had a very good understanding of its processes and of the effect that changes in one area have on the performance of another. This had helped it to identify problems in its processes and to develop a standard maintenance process to enable internal benchmarking. However, the benefits of these largely isolated patches of work have yet to be felt outside their immediate areas.

2.51. The ANAO considers that a common approach to the development of performance measures across the joint support organisation would assist it in providing effective support to a joint ADF. In particular, a single strategy would assist in integrating existing logistics organisations and strategies. The need for a common approach will only grow as increasing use is made of single-Service logistic management of items or functions.

2.52. A uniform approach does not have to imply rigid application of the same measures. However, it provides the necessary framework within which measures can be tailored for local conditions while maintaining the integrity of the integrated management objective.

Advantages of balanced scorecard approach

2.53. The balanced scorecard approach may be a useful model within which to provide such a framework. Advantages arising from its use include that it:

• helps managers to understand the relationships between the four areas;

• is consistent with quality-related initiatives such as cross-functional integration and continuous improvement;

• strikes a balance between financial and operating performance measures;

• puts strategy rather than control as the central focus;

10

• helps managers to focus on the handful of measures that are most critical; and

• provides a practical, easy to read, condensed statement of performance.

2.54. However, it is not necessarily the specific model used that is of greatest importance. In many ways, it is the very process of developing and articulating the model that provides the necessary environment to guide the development of integrated and relevant measures.

Recommendation No. 1

2.55. The ANAO recommends that Defence develop a coherent logistics performance management strategy and framework, incorporating supply chain management, that:

a) ensures performance measures at each level are logically derived from overall objectives;

b) provides for an integrated, consistent and balanced set of performance measures cascaded throughout Defence; and

c) links core business processes at each level to the achievement of higher level objectives.

Defence Response

2.56. Agree.

Performance management across related functions 2.57. There is scope for the Defence logistics performance management framework to be expanded to encompass wider activities that impact upon overall performance. For example, any delays in the consideration of procurement contracts by the legal and contracting specialist areas within each Service can impact upon the performance of inventory managers. Currently there are no mechanisms for relating the performance of those areas to overall performance.

2.58. Perhaps the need for this issue to be addressed is greatest in respect to the role of the projects responsible for the purchase of capital equipment. Much of the inventory managed by the Services is inherited from these projects, either as part of an initial purchase of spares and


consumables or as a consequence of assumptions made by them with respect to the reliability, availability and maintenance profile of the capability. According to the DER logistic review team, about 80 per cent of through-life costs are determined in that process.21

2.59. This is an important issue which the ANAO could not pursue in detail within the scope of this audit due to resource and time restrictions. However, it did become clear that there has been inadequate communication between project offices and the relevant logistic and support commands.

2.60. For example, the ANAO noted a December 1996 minute in which AF Log Comd. noted that the Command did not have an appropriately robust relationship with Materiel Division (responsible for capital projects) to enable them to know their relative responsibilities. It was noted that Log Comd. is still facing serious problems with the funding implications of a number of recent aircraft modification projects. Another example noted related to the development of a unique supply and maintenance information system for the ANZAC class of ships that added to an already extensive range of such systems across Navy.

2.61. This issue highlights the need for an integrated performance management philosophy across all relevant areas of Defence that enables the causal effect of problems identified in one function to be clearly identified across the rest of the supply chain, and in subsequent activities.


2.62. The ANAO recommends that the Defence logistics performance management framework include strategies to:

a) assess the impact of decisions made by capital acquisition projects on the performance of the supply chain as a whole; and

b) provide for the appropriate consideration of those assessments in the inventory-related decisions of future acquisition projects.

Defence Response

2.63. Agree.

21 Defence Efficiency Review Logistic and Regional Support Review Team Final Report, 1997, p 23

12

Logistics performance management ‘centre of excellence’ 2.64. Another factor inhibiting the rapid adoption of leading performance management practices in Defence has been the fragmented nature of the development effort. Development within each Service has occurred largely in isolation, with little inter-Service communication. This has resulted in significant duplication of effort. Inconsistencies in both the performance measures and data collection mechanisms used make the consolidation of information and comparison of performance very difficult across a Service, and impossible across Defence.

2.65. The ANAO identified over ten organisations involved in this area within the Service logistic and support command headquarters, together with many quality and performance evaluation sections within units in each Service. The ANAO considers there is a need for a single team or ‘centre of excellence’ to guide the development and implementation of performance management across the new Defence joint support command.

2.66. Lack of continuity of skill and momentum has a primary factor contributing to the slow and patchy progress achieved so far. There have been inadequate resources committed to this issue, with the relevant areas significantly under-resourced. Efforts have been impeded by the transfer at a critical stage of the individuals with the relevant interest or knowledge. In other cases, individuals with little relevant training have been charged with developing performance measures.

2.67. A dedicated team would provide a means of addressing that issue, as well as preventing continuation of the current fragmented approach. Although sub-teams may be necessary to guide the adaptation of common performance measures for different operating environments, this organisation would provide an environment conducive to the open exchange of information and knowledge which has been largely absent to date.


2.68. The ANAO recommends that, as part of a rationalisation of duplicated functions across the Service logistic and support organisations, Defence form a single performance management organisation to be charged with deploying its logistics performance management strategy to each level within the Support Command and across Defence as required.


Defence Response

2.69. Agree.

2

3. Performance Management of Supply Chain Processes

This chapter discusses the potential for Defence to complement its current focus on the measurement of results with an expanded use of process assessment to support more pro-active management of key supply chain processes.

3.1. Leading practice in performance management is moving to balance the traditional focus on results with increasing measurement of the internal and external processes through which they are achieved.22 Although useful in identifying that a problem may exist, results measures seldom help to diagnose its cause.

3.2. Within each process there are certain factors or ‘drivers’ that have the greatest influence on its performance. For example, supply lead time may be a key performance driver of the inventory replenishment process. By focusing measurement on the key drivers within important processes an organisation can control and improve overall performance.

3.3. Of the 354 measures analysed by the ANAO, less than twenty per cent could be assessed as providing information regarding the operation of supply chain processes. The remainder measure the results or outcomes of functions. The command-level measures that have been the focus of development efforts provide broad indications of performance, but few units have successfully translated them into measures that assist them in pro-actively managing their supply processes. The application of techniques such as statistical process control and process variance analysis is also patchy.

22 The Global Research Team, Michigan State University, 1995, op cit, p 224-240

Case Study 1An internal review of performance management of logistic support for the C-130transport aircraft fleet conducted in November 1996 found there were few metrics inplace that identified the critical issues or the performance of processes andoperations. In the month available to it, the review was unable to determineinventory dimensions, performance trends, operating expenses, bottlenecks ormeasures of engineering integrity. Nor were there performance metrics to measurepreparedness. The review found few definitive mechanisms able to cause knownoutcomes in throughput, operating expense, inventory availability, life-cycle cost,airworthiness or reliability. As a result, logistic engineers were unsure how toinfluence directly and positively their key performance indicators.

There were no tangible measures of the gap between “good” and “bad” performance.The review found that the dispersed repairable-item and spares managers had littlepower over their suppliers and channels, and were unable to present anyperformance data to locate supply chain management problems.


Possible areas of process assessment in inventory management 3.4. A good example of the potential benefit of process-related performance measurement can be seen by considering the impact delays or poor coordination within and between processes can have on overall performance.

3.5. Speed and synchronisation provide significant opportunities for improved performance of the supply chain. Leading practice in replenishment logistics is moving from a supply-side ‘push’ orientation to a customer-driven ‘pull’ system in which demand drives replenishment activities.23 The aim is to reduce costs by cutting the time taken to complete replenishment. This includes seeking opportunities to move inventory through distribution facilities faster or allowing it to bypass them altogether. This requires a performance measurement framework that supports a process assessment focus.

3.6. Many organisations are placing increasing focus on inventory velocity, or ‘dwell time’.24 Dwell time is a measure of the ratio of days inventory sits idle to the days it is moving. The axiom is that once the movement of goods is initiated, they should continue in motion until arriving at their final destination. Under such ideal flow, dwell time would be zero. According to the Michigan study, typical ratios average about 20:1. In other words, items typically sit at rest in logistic pipelines for 20 days for each day they are moving. The best dwell times achieved are in the range of 8-10 days resting for each day moving.25

3.7. Longer dwell times push up the inventory levels that must be maintained to ensure customer service, and are a significant factor in the agility and efficiency of the supply chain. However, existing performance measurement systems do not provide the information required for this factor to be actively managed within Defence.

Dwell time in repairable item pipelines

23 Mercer Management Consulting, 1993, op cit, p 5 24 The Global Research Team Michigan University, 1995, op cit, p 94-100 25 The Global Research Team Michigan University, 1995, op cit, p 94-100

2

3.8. Over two-thirds of the $3.9b of inventory considered by this audit relates to repairable items (RI) - components or assemblies that are themselves repairable, such as engines and pumps. When they fail (or on a time-phased maintenance cycle), RIs are removed from the parent equipment for repair. A replacement is used to restore the parent equipment to serviceability. Over $233m was budgeted to be expended in 1996-97 on the maintenance of these items by external contractors. Additional maintenance work is conducted by internal workshops.

3.9. The velocity with which they move through this logistics pipeline has a direct bearing on the size of the pool of RIs needed to ensure support of the parent equipment. Dwell time is an important issue in this regard.

3.10. As Figure 3-1 shows, each of the Services is holding significant stocks of unserviceable items. In February 1997 over 61 per cent of Army’s $147m pool of RIs, 44 per cent of Navy’s $370m pool and 32 per cent of Air Force’s $2086m pool were unserviceable. This may in part be due to a cost-reduction strategy increasingly being adopted by Defence RI managers in which they seek to repair only enough items each year to provide the required level of serviceable items.

Figure 3-1 Serviceability of Defence repairable item pools as of February 1997

Source: Data provided by Department of Defence

0

50000

100000

150000

200000

Navy Air Force Army

No of items

Serviceable Unserviceable

3.11. Effectively managing the risks and costs associated with such strategies requires reliable information regarding forecast demand and the location and status of items in the pool, as well as a robust and reliable repair pipeline. However, until recently, this has not been an area in which the Services have focused their performance management.

Problems identified in Air Force repairable item pipeline

3.12. Problems in the Air Force pipeline have been recognised since the 1980s, with the 1994 STRIDE study again identifying deficiencies in the process including a lack of asset visibility, an inability to make trade-off decisions between pulling items through the pipeline, redistributing serviceable items or buying new ones, and forecasting deficiencies,

2

particularly forecasting when items need to be inducted into the pipeline for repair. Other key findings were that:

• there was no valid or consistent linkage between demand and supply for serviceable RIs and the breakdown spares used in repairing them;

• no person had end-to-end responsibility, and personnel working within the pipeline had very limited understanding of the overall process;

• there were a large number of steps in the process that added little value;

• each step operated reactively with limited knowledge of impending inputs; and

• although a great deal of data was recorded manually and electronically, its application to the provision of useful and timely management information was very difficult.

3.13. In 1995 Air Force commissioned a process re-engineering study which found there were too many RIs being maintained in the system. A significant reason for this has been the difficulty acquisition projects and inventory managers have had in accurately predicting the pool of items required. A more sophisticated modelling tool developed to assist with this is not yet widely applied, due to inadequate training and resources.

3.14. Another reason has been inadequate RI performance management. The problems described above have been manifested in a slow and cumbersome pipeline in which items remain idle at various points. The aim of the re-engineered process will be to achieve a better mix and disposition of the RI inventory by enabling managers to make better decisions regarding expenditure on procurement and maintenance.

Similar problems in Army and Navy pipelines

3.15. The ANAO observed similar problems in the RI pipelines of Army and Navy. For example, there are significant problems associated with the management of RIs for the Leopard tanks, with considerable difficulty experienced each year in ensuring adequate serviceable stocks are available as required. This may be due to an inadequate pool of items, which is difficult to address given the age of the tank fleet. However, it is also due to the inadequate flow of items between repair venues and operational units.


3.16. Army has recently undertaken a manually intensive stocktake of RIs for both the Leopard tanks and M113 Armoured Personnel Carriers. The stocktake identified considerable discrepancies between SDSS inventory records and those of the operational units, with errors occurring in both sets. As a result, inventory managers did not have an accurate picture of the stock available for use. It also identified that many units were holding stockpiles of RIs, with unit holdings exceeding approved levels for 76 out of 105 RIs for the Leopard tank. 3.17. Annual output targets for the maintenance of these items by Army workshops are consistently not achieved. Although the ANAO was advised that this is primarily due to a lack of spares, there has been little rigorous analysis of the causes or effects of this consistent failure or to identify the causes of other problems experienced in the pipeline. 3.18. There are currently few performance measures for the Navy RI pipeline. Many of Navy’s RIs relate to ageing ships for which additional items are difficult to procure. Therefore, the management focus is on the repair and maintenance of the existing pool. Most management effort is focused on the satisfaction of urgent defect demands from ships requiring urgent RI repair or replacement. The time taken to fill these demands is considered to be a crucial performance indicator and is measured manually.

3.19. However, there is little measurement of other aspects of the pipeline, such as repair and distribution times. The RI manager responsible for Naval air stores advised the ANAO that, although some reporting has recently been introduced, there has been little measurement of the performance of internal workshops. There has been no analysis to show whether the process is operating in the most efficient manner possible. For example, there appears to be no measurement of the nexus between the availability of spare parts and delays in the pipeline.

Process measurement would help to improve performance

3.20. A closer focus on improving the efficiency and speed of the pipeline process would assist Defence in improving the availability of both serviceable RIs and the equipment they support, as well as helping to identify opportunities for savings through the maintenance of smaller RI pools. The team responsible for implementing the re-engineered process in Air Force is seeking to embed both outcome and process measurement as an integral part of the new system. Although still under development, the ANAO considers that Defence should evaluate the potential for wider application of the work being undertaken by Air Force.

4

Dwell time in Defence freight movement

3.21. Assessment of distribution performance within Defence is based on notional maximum timeframes set out in the Australian Standard Materiel Issue and Movement Priority System (AUSMIMPS), adapted by Defence from a United States system. Demands are assigned one of four general priority classifications which determines the overall target time frame for their satisfaction.26 This overall target is broken down into maximum times for demand submission, supply unit processing and in-transit movement.

3.22. Measurement against AUSMIMPS does not show where the problem lies when a customer does not receive a demand within the assigned time frame. In particular, it provides no mechanism for measuring or analysing periods in which freight is idle. The performance of individual elements may be acceptable, yet fail to meet overall targets.

3.23. A major factor in this has been the lack of integration between the range of information systems involved. The transition of items between processes using different information systems consumes time that no system records. Also, with the overall time split between elements of the supply chain, there is no incentive for personnel actively to manage issues such as dwell time between elements. Manual investigation of instances of poor performance has highlighted that periods in which freight lies idle are a major cause of failure to meet required delivery times. For example, a review of distribution from Darwin by Army Log Comd. identified:

• delays in the receipting of freight by receiving Freight Distribution Centres (FDC); for example, a priority one package delivered to Albury from Darwin overnight via commercial freight sat in the Albury warehouse for nine days before being receipted; and

• freight not being given priority of movement by transit FDCs. The investigation report noted that one third of the delivery time for priority four freight is spent on the transit FDC’s warehouse floor. Delays in on-freighting transit freight do not reflect on the performance of the transit FDC. In one example, freight sent on 17 January from Darwin to Albury via Liverpool arrived in Liverpool on 24 January but was not delivered to Albury until 22 April. Although the report acknowledged

26 The AUSMIMPS system does not currently provide for measurement of the time taken to satisfy demands for items that are not in stock at the time the demand is received.


this was an extreme example, it noted that turnaround times from transit FDCs average six to ten days.

3.24. AUSMIMPS is to be reviewed by Defence as part of a project to market test the distribution function. The ANAO considers that Defence would benefit from ensuring that any replacement distribution performance management system provides for a greater focus on the factors affecting the speed and velocity of distribution processes within the supply chain.

Lead time measurement

3.25. Lead time within the procurement process is another area in which a greater focus on the analysis of key process drivers would be of benefit. Procurement lead time consists of:

• administrative lead time, including requirement identification and awarding of contracts;

• supplier lead time, including materiel acquisition, scheduling and production; and

• delivery of items.

3.26. An understanding of these elements is critical to the ability of managers to make valid cost/benefit assessments regarding procurement strategies and stocking policies. Reductions in lead time can enable an organisation to reduce the level of stock held. For these reasons leading logistic organisations regularly monitor lead times for key inventory items.

3.27. A report in 1992 by the Joint Committee of Foreign Affairs, Defence and Trade, Stockholding and Sustainability in the Australian Defence Force, recommended that Defence arrange for the development of a system to actively monitor production and supply lead times for combat-oriented and other important stock items.27 However, the ANAO could identify little overt measurement within Defence. Although SDSS records the lead time associated with the last procurement of an item and applies that in calculating purchase recommendations, there is little

27 Stockholding and Sustainability in the Australian Defence Force, Joint Committee of Foreign Affairs, Defence and Trade, December 1992, p 27

6

understanding or analysis of the various lead time elements associated with many items.

3.28. The ANAO considers that Defence would benefit from focusing on the analysis and management of each component of procurement lead times. Even for those items for which there is little scope for reduction in supplier lead time, such as military items procured from overseas, the administrative component may offer opportunities for greater efficiency. Improved understanding of each element would improve inventory forecasting.

3.29. Some inventory managers within Defence advised the ANAO that they had achieved significant lead time reductions in recent years. However, the absence of an effective performance management focus on lead time has reduced the opportunity for examples of leading practice within Defence to be identified and applied more widely.

Electronic commerce

3.30. An important element in lead time reduction is the use of electronic commerce, using mechanisms such as electronic data interchange (EDI), E-Mail, facsimile and electronic funds transfer. These are particularly useful for the replenishment of commercially available items for which forward purchasing agreements can be negotiated.

3.31. Inventory managers within the Services are aware of the potential benefits offered by the use of such tools and expressed considerable frustration at the delays associated with their implementation in Defence. A variety of interim arrangements using facsimile transmission of orders between units, inventory managers and suppliers have been examined or put in place, but these have not provided the efficiencies required. Many involve manual intervention and double entry of data.

3.32. The Addendum to the DER report included a recommendation that Defence implement electronic commerce.28 The ANAO supports this recommendation and considers that Defence would accrue significant benefits from applying the necessary resources to ensuring that effective electronic commerce arrangements are instituted as soon as possible.



Increasing awareness of need for process assessment 3.33. It has been recognised within Defence that unreliable supply performance has encouraged behaviour which further degrades overall performance. For example, the fact (or perception) that items may not be available when required has led to greatly increased use of high priority demands.

3.34. The DNSDC has identified this as a major cost driver that has a significant impact on their performance in other areas. However, other than through manual analysis, there is no means of quantifying its extent or identifying the main causes. A closer focus on the assessment of procurement and distribution processes would assist Defence in identifying how to improve customer satisfaction and faith in the system, and therefore overall efficiency.

3.35. There has been an increasing awareness of the need to address this issue within the Services. For example, a paper prepared for the 1997 AF Log Comd. planning conference argued that:

An understanding of business processes is considered essential to the development and use of a performance management system - because it is process, not functions, that should be understood and measured. A process rather than a functional approach will eliminate a tendency to produce ‘stovepiped’ performance measures….Also, an understanding of business processes will enable an appreciation (and hence measurement) of the drivers of outcomes, not just the outcomes.

3.36. The ANAO noted that the conference identified a number of objectives directed at the further development of performance measures, but there did not appear to have been an overt commitment to the need to focus on process performance measurement in that development.

3.37. Views similar to those in the Air Force paper were expressed by delegates to a 1995 performance measurement seminar in Army Log Comd. Participants recommended adoption of a process-based measurement system that monitors the key result areas of capacity, financial effectiveness and customer satisfaction. As noted earlier, however, the 1996 review of Army Log Comd. performance indicators produced little change in the existing focus on the measurement of results.


8

3.38. The ANAO recommends that the Defence logistics performance management strategy emphasise the development of a limited number of credible performance measures to assist in pro-active management of key supply chain processes at both the corporate and unit levels.

Defence Response

3.39. Agree.

Need for formalised processes 3.40. Determining the specific measures appropriate for each process is reliant upon the formalisation of those processes and identification of the key drivers within them. This helps to ensure measurement effort is appropriately directed and provides valid indications of performance, as well as helping to eliminate non-value adding activities. 29

3.41. Each Service has been documenting its logistical processes in recent years, including those relating to the supply chain. This has been due in large part to the pursuit of quality accreditation and/or the market testing of functions under the Commercial Support Program (CSP) process. A number of units observed that the real benefit of pursuing accreditation had been that it had required them to identify and map their processes. In many cases this was the first time processes had been critically reviewed or discussed. However, the work in this area by the Services has highlighted that there is not yet a full or common understanding of the key logistics processes undertaken.

3.42. Process mapping is a useful tool for identifying the different procedures and practices being used for the same process. Completion of this process in Defence would assist in both identifying examples of better practice and in achieving greater standardisation for greater efficiency. Given that the Michigan study identified standardisation as one of the key capabilities that drive integration in world-class organisations, this will be an important element in the successful migration to a joint support command.30


29 The Global Research Team, Michigan State University, 1995, op cit, p 182 30 The Global Research Team, Michigan State University, 1995, op cit, p 182


3.43. The ANAO recommends that Defence:

a) evaluate the progress of each Service in the documentation of logistics processes;

b) develop and implement a standard approach for completing that process; and

c) where there are gaps in information, document the core processes to be undertaken within the Defence support command.

Defence Response

3.44. Agree.

Process ‘owners’ important to achieving integrated performance 3.45. Achieving enhanced integration within key processes also requires the identification of ‘process owners’ responsible for the overall, end-to-end performance of a process, regardless of functional and organisational boundaries. Commercial organisations have found that by assigning responsibility for a process to a single owner they are able to reduce duplication of effort, and provide a total system focus rather than sub-optimise into ‘stovepipes’.31 One supply chain benchmarking study reported that outstanding performers in all five industry groups studied focused responsibility for total inventory in one organisation/leader.32

3.46. Many supply chain processes within Defence operate across functional and organisational boundaries. Prime examples are the repair pipelines and physical distribution system referred to earlier. It has been recognised within Defence that, in the absence of a single point of authority for these processes, their performance has been sub-optimal.


3.47. The ANAO recommends that Defence identify single points of authority with responsibility for the end-to-end cross-functional performance of each key supply chain process.

31 Report on Better Practice Principles for the Performance Management of Large and Complex Inventories, Price Waterhouse Urwick, 1997, p 9 - commissioned by ANAO. 32 Stewart, G, op cit, 8:2, p 38-44.

10

Defence Response

3.48. Agree.


4. Managing inventory as an asset This chapter examines factors that have inhibited the ability of supply chain managers within Defence to take a ‘whole-of-business’ approach. It highlights that world-class organisations emphasise the use of integrated logistics management and total cost analysis to support such an approach, and discusses potential benefits that could accrue to Defence from its wider application. 4.1. The Services’ inventories have characteristics quite different from those of the majority of commercial inventories. Points of difference include:

• the predominance of technologically complex items;

• slow stock turnover rates for many items due to the small fleets they support;

• overseas sources of supply and inherently long lead times;

• the large number of complex items that are no longer in production; and

• the particular importance of avoiding stock-outs for critical items.

4.2. These factors mean that commercial inventory management practices do not always represent an appropriate strategy or performance level for Defence. However, they do not negate the requirement for Defence to ensure it is obtaining the best return on its inventory investment achievable given its particular circumstances. In Defence terms, that return should be maximum contribution to military capability for the resources expended.

4.3. In order to maximise the contribution of their logistics investment, many leading commercial organisations are moving toward ‘integrated logistics management’. This involves integrating functions such as transport, warehousing and inventory management to promote a synergistic effect that enhances total performance.

12

4.4. In a 1991 study of private sector efforts to improve logistics operations, the United States General Accounting Office (GAO) observed the practices of seven companies identified as leaders in integrated logistics management. All seven, including those who had not yet completed implementation of integrated logistics management, had realised significant benefits. Most experienced notable inventory reductions, one from US$1b to US$569m.33 One company reduced 27 warehouses to six, and another assimilated several hundred warehouses into 6 distribution centres. Inbound and outbound transport costs at one company were reduced by US$310m.34

4.5. A key element in obtaining such benefits is a management framework that identifies the contribution each element makes to overall supply chain performance. In order to achieve this, organisations are increasingly using total cost analysis. Total cost analysis 4.6. Total cost analysis involves developing complete and accurate cost information for all logistics functions and operations. Its greatest benefit is that it enables informed trade-offs among those functions. For example, one company visited by the GAO used a higher-cost transport service to respond to decreasing customer satisfaction, finding that reductions in inventory and warehousing costs more than compensated for increased transport costs.35 The Michigan study found that firms with comprehensive measures of total cost are more apt to be world class.36

Total cost management of the Defence inventory 4.7. Making cost-effective use of Defence supply-related resources requires a full understanding of the requirement for items based upon capability, preparedness and safety considerations. These factors should then, particularly in peace time, be traded-off against the costs involved in various procurement, storage and distribution strategies.

33 GAO/NSIAD-91-210, Defense Logistics: Observations on private sector efforts to improve operations, 1991 34 GAO/NSIAD-91-210, 1991, op cit, p 2-11 35 GAO/NSIAD-91-210 1991, op cit, p 2-11 36 The Global Research Team Michigan State University, 1995, op cit, p. 227-237


4.8. However, there has been little focus within Defence on developing a management approach for inventory from this perspective. There are few incentives within the current resource and performance management frameworks for managers to consider wider supply chain costs. For example, inventory managers have little knowledge of the additional costs associated with procurement, such as freight and storage costs. 4.9. Other examples relate to the use of internal repair venues, the cost of which is not attributed to the RI manager, in comparison with repairs by an external contractor paid for by the manager. RI managers may also prefer to repair items even where this may be labour intensive as this is more cost-effective from their point of view than procuring a replacement item. In making such decisions, the costs of internal repair are not required to be traded-off against alternatives. Air Force advised that it was unable to identify the RI costs associated with its internal workshops.

4.10. Appendix 1 indicates that seventeen per cent of the measures analysed related to financial management. Only a third of those represented attempts to relate cost or financial performance to the achievement of overall objectives. As is shown in Appendix 3, of fourteen cost measures taken by world-class firms, the ANAO identified examples within Defence of four commensurate measures, relating to outbound freight costs, direct labour costs, comparison of actual costs to budget and cost trend analysis. No measures were identified relating to total supply chain costs. There was only limited analysis of the costs associated with various elements of the supply chain. For example, the ANAO identified no measurement of:

• inbound freight costs;

• administrative costs; or

• inventory carrying cost.

Impact of carrying costs 4.11. The importance of understanding the more wide-ranging costs associated with inventory management decisions can be seen by considering, for example, the impact of carrying costs. Normally expressed as a percentage of inventory value, carrying costs include storage and handling costs, the potential costs associated with damage, theft, spoilage or obsolescence and opportunity costs. These are incurred every year the inventory is held.

14

4.12. Research indicates that, in the commercial sector, total inventory costs run at between 14 and 50 per cent of the value of the product produced or sold on an annual basis,37 with inventory-related costs estimated to account for up to 25 per cent of total logistics costs.38 Defence managers in general have not attempted to quantify inventory carrying costs. However, an internal study conducted in 1992 estimated the annual costs associated with holding inventory for the Air Force training aircraft at 12 per cent of the inventory value, consisting of:

• opportunity cost -five per cent;

• storage - three per cent;

• handling - one per cent;

• administration - one per cent; and

• damage - two per cent.39

4.13. That estimate could be considered conservative when compared with the experience in the commercial sector. Defence, like other public sector agencies, is generally free of the pressures arising from the commercial costs of inventory, such as interest, rent and insurance costs. These costs can substantially increase inventory carrying costs.40

4.14. Applying an estimate of twelve per cent suggests that Defence incurs total annual carrying costs of approximately $472m for its $3.9b inventory of spares, consumables and RIs. As Table 4-1 demonstrates, even a small reduction in inventory can result in considerable annual savings in carrying costs, as well as reducing the level of capital invested.

37 Lambert D. M, The Development of an Inventory Costing Methodology: A Study Associated with Holding Inventory (Chicago, IL: National Council of Physical Distribution Management, 1976), cited in Bloomberg D.S., Murray A, 1996, The Management of Integrated Logistics, p 87, Prentice Hall , NSW 38 Coyle, Bardi & Langley, The Management of Business Logistics,p43, Bloomberg & Murray, 1996, op cit, p 87 39 Integrated Materials Management within the RAAF Training Logistics Group, T. Carmody, 1992, cited in Integrating the RAAF Trainer Aircraft Spare Parts Logistics Channel, T. Carmody, 1992, p 51 40 For example, the 1989 Arthur Young report on Defence supply services estimated the annualised cost of the Defence inventory investment to be between $642m and $714m, based on interest rates of between 13.5% and 15.24%. Current interest rates range between 6% and 9%.


Table 4-1 Potential savings to Defence from reduced inventory of spares, consumables and repairable items (based on 12% carrying cost)

* This relates to the reduction in future investment arising from maintaining lower inventory levels, not to any proceeds that mightresult from the disposal of existing stockholding. For many items in the Defence inventory, those proceeds are likely to besignificantly lower than the book value.

% change in Potentia l annual saving Value of Potentia l one-timeinventory in carrying costs inventory cap ita l re lease *

he ld (based on 12% )

-10% -$47.16m per annum $3541m $393m-5% -$23.64m per annum $3737m $197m-2% -$9.48m per annum $3855m $79m-1% -$4.68m per annum $3895m $39m

5Current carryingcosts

(based on 12% )

Currentinventory

value

$472m per annum $3934

6+1% +$4.68m per annum $3973m $39m+2% +$9.48m per annum $4013m $79m+5% +$23.64m per annum $4131m $197m

+10% +$47.16m per annum $4327m $393m

Potentia l add itiona l Additional capita lannual carrying costs investment in

(based on 12% ) inventory

Source: estimates by ANAO and Price Waterhouse Urwick 4.15. Given the lack of accurate costing data in Defence, such calculations can be estimates only. However, they do highlight the importance of considering the full costs involved in inventory and supply chain related decisions. 4.16. The value of fully costing supply chain activities has also been identified in other organisations. For example, the Michigan study found that firms that tracked transport cost and failed to report total logistics cost experienced a general management lack of interest in improving cross-functional productivity.41 A similar situation exists within Defence. Many of the costs associated with the ADF Line Haul Service are closely measured, yet the overall performance of ADF distribution has been

41 The Global Research Team Michigan State University, 1995,.op cit, p. 42

16

inadequate. The DER logistic review team described it as poorly managed, disjointed and unresponsive.42

4.17. The 1995 internal review of ADF freight movement referred to earlier found a lack of overall planning and historical data, with a related inability to measure performance across the system as a whole. The review found no method of quantitatively gauging the impact of one area on another or of determining the total cost of the distribution function.

4.18. Although the United States DLA also does not measure many of the costs identified in Appendix 3, the ANAO noted that it has recently implemented measures focusing on the supply chain cost per unit.43 This reflects an increasing emphasis on managing total costs rather than focusing on minimising costs for a particular segment.

Total cost analysis an evolutionary process

4.19. Implementing total cost analysis is an evolutionary process, particularly in terms of refining cost collection and analysis capabilities. A major reason for Defence inventory managers not considering wider supply chain costs is the inability of existing information systems to generate the necessary information. However, this need not be a barrier to the introduction of a management culture in which there is awareness of the need to focus, to the greatest extent possible, on the total cost to Defence of management decisions.

4.20. The GAO reported that companies who are leaders in the use of integrated logistic management have not let the lack of adequate accounting deter them from moving towards total cost analysis.44 Believing that the potential benefits could outweigh the costs, one company started making the necessary cost trade-offs manually. 4.21. The importance of full cost visibility was acknowledged by several of the DER review teams who recommended substantial changes to the way in which resources are managed within Defence.45 These were primarily aimed at achieving a cultural shift away from the current, almost

42 Defence Efficiency Review, Logistics and Regional Support Review Team Final Report, 1997, p 3 & 34 43 Defense Logistics Agency, Performance Plan Fiscal Year 1996, p 14 44 GAO/NSIAD-91-210, 1991, op cit, p 9 45 Defence Efficiency Review, Logistics and Regional Support Review Team Final Report, 1997, p 18


exclusive, focus on in-year cash management to incorporate a greater focus on outputs. The resource management and planning review team recommended that Defence seek external assistance in designing and implementing a new resource information and reporting system.46 It considered these changes fundamental to Defence embedding a culture of continuing reform and efficiency.

4.22. The ANAO supports the recommendations of the DER in this regard and considers that Defence would benefit from improving the capacity of inventory managers to include consideration of the costs that arise outside their immediate area of concern in the identification of appropriate inventory management strategies.


4.23. The ANAO recommends that Defence develop and implement:

a) a methodology enabling inventory management decisions to be supported by the wider consideration of relevant costs; and

b) a quality assurance program to monitor:

i) adherence to the methodology; and

ii) the effectiveness with which cost information is used to make trade-offs within the logistic system.

Defence Response

4.24. Agree.

Focussing on expenditure effectiveness

4.25. As noted earlier, the majority of financial information collected in Defence relates to the monitoring of cash budgets. However, there has been inadequate consideration of the effectiveness with which those budgets are expended. Indeed, a focus on expenditure achievement as a performance indicator has proved counter-productive, providing incentives for decisions that are not in the interests of overall Defence efficiency.

4.26. The DER logistic review team found that the ‘…need to satisfy an annual performance indicator of total expenditure against allocation…’


18

has been accompanied by the acquisition of short lead-time, low priority items, resulting in associated storage, maintenance and distribution costs. It argued this could be avoided by providing for funds roll-over for replenishment stores or greater flexibility between appropriations.47

4.27. Increasingly in recent years, inventory managers have experienced relatively sudden cuts in their approved funding levels, often followed by the release of additional funding later in the financial year. Managers commented to the ANAO that this places them in a difficult position in trying to deal rationally with customers and suppliers. 4.28. The ANAO noted anecdotal evidence suggesting that, in response to this uncertain funding, some managers are focussing on committing as much of their approved budget as early in the financial year as possible in order to avoid being targeted for reductions. This exposes them to the possibility of not having funding available to satisfy high priority requirements that arise later in the year. The DER team commented that there is clear evidence that high priority items with long lead times cannot be purchased because of the failure to release funding in time.48 Asset performance measures 4.29. Many organisations use indicators of asset performance to assist in analysing the effectiveness with which inventory-related resources have been managed. As is shown in Appendix 3, inventory ‘turns’, number of days’ supply and obsolete inventory are all widely measured in world-class organisations. 4.30. A turn is a measure of how many days it takes the inventory in the system to change on average. For example, if average inventory on hand is 10 weeks’ supply, inventory turn would be 5.2 times per year. Slow inventory turns can be an indicator of excessive or inappropriate stocks. As turnover increases, both inventory levels and associated carrying costs decrease. The GAO reported that several of the companies it visited had achieved an increase in inventory turn through integrated logistic management, one from 13 to 17, another from 1.8 to 2.5.49

4.31. Within Defence there has been limited focus on the measurement of inventory asset performance. Although fifteen per cent of the measures

47 Defence Efficiency Review Logistic and Regional Support Review Team Final Report, 1997, p 28 48 Defence Efficiency Review Logistic and Regional Support Review Team Final Report, 1997, p 3 49 GAO/NSIAD-91-210, 1991, op cit, p 11


analysed in Appendix 1 related to the utilisation of assets, about half focused on the use of equipment such as vehicles or aircraft. Such measures provide only broad information regarding the effectiveness with which inventory is utilised.

4.32. The ANAO noted a general perception within Defence that indicators such as inventory turn are not a relevant measure of its performance. This is because, given the characteristics of the inventory, the turns that can be achieved are very low. As a result, inventory turnover has not been actively measured or managed. However, as the latter example identified by the GAO above demonstrates, commercial organisations with low turn rates still consider this a worthwhile measure. Given the size of the Defence inventory investment, even small improvements could generate significant benefits.

Demonstrating the value of inventory turn measures

4.33. Inventory turn analysis provides a focus for management effort. This has been demonstrated recently in Air Force. Although SDSS provides a requirements determination (RD) capability, Air Force has developed an alternative system for its breakdown spares, known as AIMS-BDS, which is also based on a commercial off-the-shelf product. Air Force considered that the module within SDSS did not support its requirements, particularly in terms of forecasting and analytical ability.

4.34. The strategic management capabilities within AIMS-BDS have enabled Air Force to begin to profile its inventory against a number of parameters. Initial results have highlighted a number of potential problem areas. For example, it has identified that 56 per cent of the 280 000 line items in the breakdown spares inventory are overstocked (based on a parameter of two years’ usage). This ratio is expected to reduce when insurance stock, life-of-type stock and reserve stock are able to be excluded. A further six per cent are either out of stock or projected to be out of stock within procurement lead time.

4.35. In particular, the system has allowed Air Force to identify the turn being achieved in regard to breakdown spares. It identified a current inventory turn across the Command of 0.18 times per year; that is, on average it takes 288 weeks for the inventory to change. As of June 1997, the system reported that stock on hand was sufficient for 319 weeks of average usage.

4.36. There are limits on the extent to which Air Force can reduce these stock levels, particularly given the long lead times associated with

20

many items. However, there is clearly scope for savings through improved management. According to AIMS-BDS, the optimal turn that might be achieved through improved forecasting and management is 0.8, suggesting average stock levels could eventually be reduced by over 75 per cent to 65 weeks’ stock.

Indicative calculations highlight potential benefits

4.37. The inclusion of insurance and other intentionally stockpiled items, together with doubts regarding the accuracy of existing valuation and pricing data, makes it difficult to identify accurately the savings Air Force might gain from increasing inventory turns to that level. However, indicative calculations highlight the potential for considerable savings.

4.38. Data provided by Air Force indicates that the current value of its inventory of spares and consumables is $743m. Breakdown spares represent a considerable proportion of that, perhaps 70 per cent ($520m). Beyond the $340m reduction in capital investment that might result from an eventual 75% stock reduction, carrying costs would also be significantly reduced. For example, using the 12 per cent estimate identified earlier, potential annual savings in carrying costs for breakdown spares could be in the order of $46m.

4.39. The other Services have started to identify the ratios of their inventory that are slow or fast moving, and the percentage of items for which there are outstanding demands. However, it is not clear how this will be applied in on-going management of inventory. 4.40. Obsolete inventory is another widely-used asset management performance measure that has received limited focus within Defence. For example, Defence maintains significant stocks of insurance items. Given the associated holding costs, these and other items potentially affected by configuration changes should be regularly reviewed for continued relevance. However, only Navy appears to have an active program for such on-going review. 4.41. The Navy Inventory Optimisation section has begun to utilise analysis of inactive stocks to identify items for disposal. HMAS Stirling, for which that analysis has been completed, now has a level of inactive stock of 22 per cent, well below the average of 49 per cent in other Navy districts. 4.42. Regular measurement and analysis of the asset management aspects of inventory may provide greater incentive for inventory managers to actively manage stock levels. Recommendation No. 8


4.43. The ANAO recommends that the Defence performance management framework for its inventory of spares, consumables and repairable items provide for the regular review of inventory asset performance through measures related to, for example, inventory turn, inventory levels (number of days’ supply) and obsolete inventory. Defence Response

4.44. Agree.

Segmenting inventory to target stockholding strategies 4.45. Analytical review such as that described above can help to identify areas where inventory management can be improved. However, in order to then ensure appropriate management strategies are applied, a more focused approach is also needed. Effective inventory management requires that items be grouped or segmented according to common characteristics in order to design the most appropriate flow through the supply line.50

4.46. Profiling or segmentation strategies need to be tailored by an organisation to its particular circumstances. General characteristics that should be considered include:

• predictability and level of demand for items;

• special characteristics, such as dangerous or perishable goods;

• item criticality;

• item value; and

• order and delivery lead times.51

4.47. One commonly used technique, known as A,B,C analysis, segments items in decreasing order of annual dollar volume. Another is critical value analysis in which items are segmented based upon their relative priority. As is illustrated in Table 4-2, the characteristics of each segment are then used to determine the business strategy, including performance targets, that will be applied to that inventory.

50 Statement of Best Practice, Inventory Management, Office of Public Management, Premier’s Department, NSW, 1993, p 11 51 Office of Public Management, Premier’s Department, NSW, 1993, op cit, p 12

22

Table 4-2 Illustrative stockholding policy for various inventory segments

Group 2Surge flow, high criticiality,low value, long lead time,small size

Hold high level of stock, allowing safety stock fordelivery lead time and demand fluctuations.

Group 3Surge flow, low criticality,high value, bulky item

Minimise stock, difficult and costly to store. Lookat opportunities for “make to order”.

Group 4Wave flow, slow moving,high criticality, perishable

Minimise stock holdings, building only duringperiods of peak demand (should be predictable).Direct delivery ideal.

Source: Statement of Best Practice, Inventory Management, Office of Public Management, Premier’s Department, NSW,1993, p 12.

4.48. There has been only limited segmenting of the Defence inventory. Although Air Force logistics units indicated that they currently make no conscious effort to segment their inventory, the budgetary process now requires them to divide it into high and low-cost items. As the high-cost items are financial drivers, management effort can be focused on them.

4.49. Navy is developing the Common Management Code which is intended to enable managers to relate stock items to weapon systems. It is also considering partitioning its inventory to allow better procurement. Possible categories are:

• system management items - items requiring greater than routine management; and

• commodity management items - items which can be managed less intensively.

Applying segmentation to differentiate stockholding policy

4.50. In general, however, the inventory has not been analysed from a segmentation perspective. As a result, there has been little differentiation between items in terms of stockholding policy and performance measures, particularly for items managed using the automated processes in SDSS.

4.51. Following the 1992 JCFADT inquiry mentioned earlier, Defence produced an ADF Reserve Stockholding policy. However, that policy has not been effectively implemented. An operating stockholding policy was also planned, but has not been produced. HQADF now plans to revise the existing Reserve policy with the intention of re-issuing it as a single policy addressing both reserve and operating stocks at the strategic level.


4.52. In the absence of an endorsed policy, inventory managers in each Service have relied on historical usage as the principal stockholding guide. However, there has been inadequate information available to them regarding both the relative priority to be attached to the supply of various items and the likely demand for them. As a result, item managers assign priorities based on their own understanding.

4.53. A defacto stockholding policy has been applied through the nominated service level defined against each item in SDSS (and AIMS-BDS for Air Force). Service level is a percentage measurement of demand satisfaction. For example, a service level of 95 per cent means that, on average, five out of every 100 demands will not be able to be satisfied from stock. A higher nominated service level increases the level of safety stock held and the associated capital and carrying costs, not necessarily in a linear relationship.

4.54. SDSS and AIMS-BDS apply a default nominated service level, currently 87 per cent. The ANAO could identify few examples of service level, and therefore stocking policies, being tailored to the characteristics of particular items or segments. Item managers have been provided with little training in this aspect of inventory management.

4.55. In February 1997 Army Log Comd. issued the first guidance to its inventory managers on the use of a form of A,B,C analysis to differentiate stocking strategies. That policy has yet to be implemented. Air Force expects that, over time, individual managers will tailor the service levels within AIMS-BDS, but guidance on this is not yet available. A pilot project conducted within NSC has highlighted the extensive effort required to determine such guidance. A coordinated effort across Defence may be more cost-effective.

4.56. The ANAO considers that Defence should expand its use of inventory segmentation techniques to provide a basis for ensuring the cost-effective use of resources and that management effort is concentrated on items of most importance to military capability. Promoting greater consistency in the methodologies used would enable more effective internal benchmarking and assist in the implementation of a Defence stockholding policy.

Recommendation No. 9 4.57. The ANAO recommends that Defence develop and implement a policy for the application of segmentation techniques in the management of its inventory of spares, consumables and repairable items.

24

Defence Response

4.58. Agree.


4.59. The ANAO recommends that Defence review the appropriateness of existing stockholding policies for spares, consumables and repairable items (including the service levels nominated in SDSS and AIMS-BDS) based upon an appropriately framed profile of inventory characteristics.

Defence Response

4.60. Agree.

Need for costs to be related directly to capability output 4.61. An essential consideration in appropriately segmenting Defence inventory and determining procurement and distribution strategies should be the role it plays in military capability. In Audit Report No.17 1995-96 Management of Australian Defence Force Preparedness the ANAO commented that Defence was unable to adequately relate logistic expenditure to different levels of preparedness. As a result, there was inadequate integration between defined preparedness objectives and logistics resource management decisions.

4.62. Given that long lead times can mean that inventory expenditure relates to capability achievable two or three years later, there is a need for a greater predictive ability to assess the impact of current procurement decisions on future capability. There have been a number of studies within Defence directed at improving its understanding of those linkages, but this is still an area requiring considerable development.

4.63. AF Log Comd. considers that its revised approach to budget forecasting for 1997-98 will provide its first reasonably accurate indication of the shortfall between available logistics funds and those needed to provide the required level of capability. The first attempt in 1996 identified an estimated shortfall of over $370m across the five-year budget horizon, but there were doubts about the accuracy of the data used. There is clearly still considerable development to be done in this area. The Air Officer Commanding, AF Log Comd. commented in a December 1996 minute discussing planning imperatives that perhaps AF


Log Comd.’s most crucial performance inadequacy is that they are unable to substantiate the effect of resource shortfalls.

4.64. Materiel managers in Navy advised the ANAO that a fundamental outstanding requirement for them is clear articulation by the operational customer, Maritime Command, of its requirements and priorities. Navy hopes to address this issue through the In-Service Support project, initiated in 1994 with the aim of optimising support to fleet units.

4.65. The project will re-engineer Navy logistic support into class logistic offices, linked to logistics service agencies and customers through service level agreements. This also aims to put the onus on Maritime Command to articulate its requirements. Ultimately, it is intended to achieve an understanding of the link between resources and capability. However, there have been considerable delays with this project and no agreements have yet been established.

DER and PIR support for output focus

4.66. The need for greater transparency in the cost of delivering defence outputs was supported by the DER review teams. It was suggested that development of a statement of Defence outputs in the context of a Defence Long Term Plan would greatly assist the development of performance indicators for support organisations. Reporting against those indicators was expected to allow a better understanding of decision-making processes and assist in resource prioritisation on a Portfolio basis.52 The corporate planning and resource management review team recommended development of a resource management system that identifies the full costs of delivering outputs, attributes them to the entities responsible and reliably monitors them over time.53

4.67. This view has also been reflected in the Defence approach to its participation in a review of performance information across the Australian Public Service (APS). The Department of Finance established the Performance Information Review (PIR) across a number of departments and agencies following several recent assessments showing that this is an area which needs considerable improvement.

4.68. The objective of the PIR is to improve the quality of performance information to the point where sound judgements can be made about

52 Defence Efficiency Review Logistic and Regional Support Team, Final Report, 1997, p 30 53 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p 26

26

program achievements. Although the principal focus is on publicly reported information, improvements in this area hold the promise of better information for internal management purposes. 4.69. Defence is participating in the 1996-97 round of the PIR and submitted its report and recommendations to the Minister for Defence and the Minister for Finance on 30 July 1997. This review has highlighted to Defence the need for fundamental change in its approach to the production and use of performance information. In advice to the ANAO in March 1997 regarding its progress, Defence acknowledged that its performance information generation has been ad hoc and driven more by external reporting requirements than internal management needs. Taking a new approach to performance information 4.70. The Defence PIR team’s approach is that contributions to Defence outputs should be used as the ultimate basis of performance measurement. A comprehensive performance information framework is seen as closing the Program Management and Budgeting (PMB) loop by providing the basis for quality feedback to managers which can then be used to inform resource planning for subsequent years. 4.71. In a March 1997 progress report to the Defence Audit and Program Evaluation Committee, the Defence PIR team argued that ‘…agreement to the introduction of outputs as the primary focus of, and added dimension to, resource management is the most important business decision that Defence needs to make.’ It recommended a top-down shift to an output focus supported by statements on capability and performance levels.

4.72. The ANAO supports this recommendation. Adoption of such an approach would enable Defence to embrace many of the performance management practices utilised in leading organisations across the world. The evolutionary nature of such a change is accepted. The PIR has estimated that it will take at least three years before an output focus will be fully integrated into the Defence PMB cycle. However, the ANAO considers it important that a decision to pursue this direction be taken in order to maximise the benefits that can potentially be derived from the Defence Reform Program. The ANAO noted that in July 1997 the Defence Management Committee agreed to a number of recommendations by the Defence PIR outlining the adoption of such a direction.

Cost attribution to support enhanced accountability 4.73. In recent months the Services have made increasing use of customer service agreements between logistic and operational units


and/or commands. Such agreements can be a useful mechanism for clearly articulating the requirements of the customer and measuring the extent to which they are satisfied. However, their value is significantly diminished if the requirements set out in them have not been appropriately derived. In particular, the costs associated with providing various levels of service should be understood and analysed in the context of both available funds and overall objectives. This has not been the case to date.

4.74. The ANAO agrees that there must be a strong dialogue with the customer in setting supply performance priorities and requirements. However, at present, the customer has no incentive to develop a true trade-off between cost and capability, because there is no accountability on behalf of the customer for the inputs provided. Creating such accountability requires Defence to adopt either:

• a sophisticated attribution model through activity-based management; or

• a ‘user pays’ model, in which the capability owner incurs costs for all inputs received.

4.75. Under the latter option the capability owner as customer would be allocated funds to acquire goods and services by ‘buying’ them internally or by purchasing them externally. This option would provide capability owners with the greatest flexibility to make cost and quality trade-off decisions to maximise the capability generated for a given level of funding. However, the analysis conducted during both the DER and this audit have highlighted that Defence does not yet have an adequate understanding of the costs associated with military outputs in order to support the implementation of a true ‘user-pays’ system at this time.

4.76. The use of a sophisticated cost attribution technique would provide Defence with an effective means of making such decisions. Leading-practice organisations are making increasing use of activity-based costing and activity-based management to achieve greater integration in their logistics management and to support total cost analysis.

Activity-based costing 4.77. Under activity-based costing (ABC) an organisation seeks to identify all costs related to a specific activity and to identify the causal factors, called drivers. ABC allows decisions to focus on specific policies. 4.78. ABC is not applied universally within Defence. However, its potential benefits have been demonstrated. For example, the Avionics

28

Maintenance Flight at Amberley has started to apply limited ABC, attributing personnel costs to activities. As a result, it has been able to focus on the areas contributing to its internal costs. For example, it identified that it spent $233 000 in the last year on the rework of RIs for which no fault was found; that is, items were withdrawn from the aircraft prematurely. This was identified as a significant driver of their costs, and can now be targeted for management attention. Activity-based management

4.79. The overall process of taking total cost analysis and ABC to the next step is called activity-based management (ABM). World-class firms use total cost and ABC measurement capabilities to develop activity-based management goals. Such goals serve to focus on specific opportunities for improving effectiveness and efficiency. 4.80. Although the potential benefits of ABM have been recognised within Defence for some years, efforts to introduce it have been spasmodic and uncoordinated. A 1995 Army trial concluded that the information gained was useful at the unit-level, but that the software developed was not suitable for Army-wide implementation. This, together with funding constraints, resulted in an incremental approach initially using costing data from existing systems. 4.81. Navy initiated a project to implement ABM as a strategic management tool in January 1996, producing a prototype ABM model in June 1996. In January 1997 an 18 month implementation stage commenced which aims to map all activities and resources used in Navy. The full costs associated with each activity will be compiled to allow attribution of all costs to the end-stage activities at which capability outputs are produced.

4.82. Introduction of such a system is potentially complex and requires analysis to ensure associated management overheads do not negate potential benefits. However, significant investments such as the Defence inventory must be driven by capability requirements as reflected in the demand of the operational customer. This can only be achieved by ensuring there is a full understanding of the costs associated with each activity, and accountability on both sides for the trade-off decisions made in the provision of logistics support.


4.83. The ANAO recommends that Defence develop and implement an activity-based management methodology to support the integrated


management of the Defence supply chain, having regard to Navy’s work on such methodology.

Defence Response

4.84. Agree.

30

5. Internal and external performance targets This chapter discusses the need for Defence to increase its use of clearly identified performance targets to provide meaning and direction to supply chain performance measures. It would also benefit from developing a sound and consistent methodology to guide determination of the levels of performance required for internal supply operations, supply-related functions market-tested under the Commercial Support Program (CSP) and of suppliers. 5.1. Performance targets provide meaning and direction for a performance measure. They function both as a benchmark against which to measure current performance and as an incentive to improve. Target performance levels or parameters have not been set for over 80 per cent of the measures analysed by the ANAO. The absence of formal performance targets is an indicator of the relatively recent development of performance management across the Defence supply chain. In many cases, targets have not been set because there is not yet adequate understanding of the performance that can be realistically expected.

5.2. As this understanding increases, however, the ANAO considers that it will be important that sound and realistic performance targets are clearly identified for more of the supply chain performance measures used within Defence.

Need for a consistent methodology 5.3. To be useful as a management tool, targets must be underpinned by a sound and consistent methodology in which the costs of achieving them is understood and fully considered. They should be logically derived from stated objectives.

5.4. There does not appear to have been adequate emphasis on these issues in the setting of inventory-related performance targets within Defence. In many cases, targets have not been derived from a rigorous analysis of the actual performance levels required or of the related practicalities or costs.


stocked with all sizes, as advance information regarding the sizes needed for eachintake of recruits is not provided. Achieved performance against a kit out required ofover 14 000 items has been around 99 per cent. Although the item managers advisedthe ANAO that they believe a target of 95 per cent immediate fit-out with directdelivery arrangements for any outstanding items would be more cost effective, theyhave not costed the impact of either arrangement.

5.5. Each of the Services is pursuing service level agreements with operational customers as the principal means of determining future performance targets. However, there does not appear to have been a methodology identified for the determination of targets that are cost-effective from a Defence-wide perspective.

5.6. In July 1996 Army Log Comd. entered into a customer service agreement with its two primary customers (Land and Training Commands) requiring specific levels of performance against measures such as satisfaction of inventory demands and equipment repair. Logistic units have now been tasked with negotiating agreements with their particular customer units, but are not to set targets below those identified at the Command level. The ANAO could identify no underlying methodology used in developing those targets. Logistic unit personnel told the ANAO they were unachievable within current funding constraints.

5.7. In 1996 Air Force logistic units were also directed to negotiate performance agreements with their customers. However, some units experienced difficulty in negotiating targets of less than 100 per cent performance. As a result, no targets have been identified for some indicators. As noted above, Navy also hopes to use service level agreements as the primary mechanism for managing logistic support in the future. However, there is little evidence that the necessary cost and capability understanding exists to support the negotiation of such agreements.

CSP increases the need for sound methodology

5.8. The requirement for a sound methodology for the determination of performance targets will increase as Defence market tests more of its supply chain functions. Exposure to market testing has been a positive influence on acceptance of the need for performance measurement, but there appears to have been inadequate work in identifying performance requirements.

5.9. A good example of this relates to the Memorandum of Agreement (MOA) developed in 1993 to set out the performance requirements of the Defence National Storage and Distribution Centre (DNSDC), operated as an In-House Option under the CSP. The MOA performance measures and targets have been recognised as inadequate

32

and inappropriate. They provide little information regarding the extent to which customer requirements are satisfied and were produced without consideration of volume, hourly rates or time. It has been acknowledged within Defence that it is unrealistic and prohibitively expensive to demand 100 per cent performance, as has been the case with the DNSDC and a number of other statements of requirement (SOR) developed under CSP.

5.10. A working party was established in 1996 to review the performance measures and targets within the MOA, but progress has been slow. However, the ANAO noted that the DNSDC has now adopted many of the leading principles outlined in this audit report to guide development of a sound performance management strategy.

Need to improve dissemination of lessons learnt

5.11. There was recognition within Defence in 1995 that the DNSDC experience highlighted the need for a thorough overhaul of its approach to performance measurement in CSP contracts to improve the visibility of cost and outputs. This does not appear to have occurred. The mechanisms for ensuring that lessons such as those identified at the DNSDC are widely disseminated appear to have been inadequate. There is not yet a standard basis for performance measurement in SOR for logistics functions.

5.12. The ANAO visited an Army logistic unit whose functions, including supply-related activities, are scheduled to be market tested. The unit is developing Service Level Agreements (SLA), including performance measures and targets, to support the SOR. However, there was little guidance or training available to the personnel involved in this process. The performance targets used have largely reflected those used within Army Log Comd. As noted, these do not provide an effective and balanced picture of performance. There has been no cost-benefit analysis applied in identifying the levels of performance that will be required of the successful contractor.

5.13. It was acknowledged at an Army Log Comd. SLA seminar held in October 1996 that military logistic services performance measures and benchmarks are immature and need to be developed interactively as experience is gained with requirements and contract management. It was also acknowledged that lessons learnt need to be better managed and communicated to staff involved in CSP activities.

5.14. The seminar identified that current corporate benchmarks would be more useful for managing CSP contracts if they were more end-user


or customer focused. There is a need to translate blanket internal performance targets such as 85 per cent achievement against Date Equipment Required (DER) into more relevant performance requirements. This could be done by requiring 90 per cent achievement of DER where 100 per cent of the repair parts are provided; or 100 per cent achievement of DER for high priority units and lower performance levels for lower priority units.

5.15. The seminar concluded that there was a need to develop better benchmarks for military logistic services that are more customer and outcome focussed. The ANAO considers that the use of more tailored performance targets that include consideration of cost, item priority and customer requirements would also be beneficial in the more effective management of internal performance. The ANAO also considers that Defence should review existing supply chain performance measures to determine the scope for applying relevant performance parameters.


5.16. The ANAO recommends that Defence revise existing internal supply chain performance targets to ensure that they are:

a) relevant to stated objectives;

b) based on a sound and consistent methodology; and

c) realistic and practical within existing resource constraints.

Defence Response

5.17. Agree.


5.18. The ANAO recommends that, in order to improve the visibility of costs and outputs, Defence review and standardise the mechanisms used for:

a) determining a limited number of credible supply chain performance measures and targets for inclusion in CSP contracts and service level agreements; and

b) identifying and disseminating lessons learnt in this regard from the market-testing of supply-chain functions.

Defence Response

34

5.19. Agree.

Supplier performance 5.20. Defence has begun to make increased use of long-term supply contracts that incorporate leading commercial practices such as forward purchasing arrangements (also known as standing orders, bulk orders or common-use contracts). Such contracts enable the use of electronic commerce and vendor-held stock arrangements.

5.21. For example, Army Log Comd. has entered into a prime vendor arrangement under which pharmaceuticals are supplied directly to 35 Central Dispensing Points (CDP) in the larger ADF units. In return for a distribution fee, the prime vendor has taken over a large proportion of the storage and distribution aspects, with the fee calculated as a percentage of the cost of pharmaceuticals distributed.

5.22. Army Log Comd. negotiates standing offers for commonly required pharmaceutical items, but CDPs are now able to place orders directly with the prime vendor, daily if required. The prime vendor is then responsible for obtaining the items through those standing offers and delivering them directly to the requesting facility.

5.23. This innovative arrangement was one of the first of its kind entered into by Army and is an example of the benefits that can be achieved through the use of better practice. US Defense has used such arrangements to achieve considerable improvements in service to its medical facilities while reducing inventory holding costs.

5.24. Army has entered into similar arrangements for the supply of non-combat clothing and spare parts for non-combat vehicles. Air Force logistic units have also entered into vendor-held stock arrangements, primarily relating to the supply of spare parts for the repair and maintenance of aircraft and RIs. Navy has made some use of such arrangements for items with limited shelf-lives.

Performance requirements in vendor-held stock contracts

5.25. Strategies of this type make effective performance management even more important. Defence inventory personnel will require enhanced skills in managing the supply chain, including the monitoring of supplier performance. Performance targets are being progressively included in such contracts. However, Defence personnel are still relatively inexperienced in negotiating and administering this sort of arrangement.


5.26. For example, in Audit Report No.34 1996-97 Australian Defence Force Health Service (p 84) the ANAO commented on a number of issues associated with the performance targets included in the contract with the pharmaceuticals prime vendor. The measures included in the original contract were imprecise. The customer satisfaction level required (95 per cent), does not take account of the criticality or relative demand of items not able to be supplied as required. This reflects the general approach taken to the setting of internal performance targets within the Command, which can assess only average performance.

5.27. Another issue identified related to the supply of more expensive brands by the prime vendor where the standing offer supplier was unable to satisfy demands. As the distribution fee paid to the prime vendor is based on the cost of items supplied, this resulted in a double increase in costs to Army. Audit Report No.34 recommended that Defence enter into negotiations with the prime vendor to amend the standing offer contract, including the development of more demanding performance measures with the aim of minimising costs to Defence. 5.28. The DER estimated that one-off savings of between $100m and $140m and annual savings of between $61m and $89m could be generated through the expanded use of practices such as vendor-held stock. However, the pharmaceuticals example highlights the need to ensure that a sound methodology is available to guide the framing of performance measures and that the performance required of vendors is understood and included in the contract. The lessons learnt in this exercise would be valuable for those across Defence involved in the development of similar arrangements in the future.


5.29. The ANAO recommends that Defence develop user-friendly guidance for inventory management personnel regarding the use of performance measures and targets in procurement contracts.

Defence Response

5.30. Agree.

36

6. Logistics information systems This chapter discusses factors that have diminished the capacity of Defence logistics information systems to support integrated and effective performance management. It outlines issues that the ANAO believes Defence should consider in implementing the recommendation of the Defence Efficiency Review that a rationalisation plan for the systems be developed and that Defence make greater use of common information systems. 6.1. The improvement of Defence’s logistic performance, consistent with the performance of leading organisations, is strongly dependent on quality and timely information management and supporting technologies. Leading performance management practice cannot be achieved without an integrated information network that provides a common information base and full visibility across the supply chain. The inability of information systems to provide such information has been a critical factor limiting the ability of the Services to adopt improved practices.

6.2. Notwithstanding the advances Defence has made in the area of logistics information systems, there is still much that can be achieved at both the strategic and operational levels. Progress in this area must be guided by strategic development of information systems which, in turn, must be consistent with higher logistics planning. This has not always been the case.

Information systems not integrated

6.3. The development of information systems in Defence has not been well managed. This was recognised by the DER, which reported that:

…We discovered from the difficulties we encountered throughout the Review, and from the strongly voiced complaints at all levels, that Defence cannot provide its managers with information in a form and manner to inform


decisions adequately. Too much of the available data is fragmented, irrelevant or incomplete, and is overly focused on inputs rather than outputs.54

6.4. Despite the significant achievement of implementing a common supply system across Defence, these problems have been particularly evident in Defence’s logistics information systems. The DER observed that there are over forty logistics and administrative information systems in use by the Services and the Department with more planned or under development. The ANAO understands the actual number may exceed sixty.

6.5. Many of these systems have been developed in isolation at the Service or unit level to address specific problems or needs. They are not integrated and lack connectivity. Functions and data are replicated. Information cannot be easily shared between them due to the use of different operating systems, data dictionaries and data base management systems. There are many over-lapping support arrangements and maintenance contracts.

6.6. The Addendum to the DER report recommended that the newly formed Commander Support Australia (COMSPT) develop and implement a funded plan for the rationalisation of these systems and migration to joint, integrated systems.55 The ANAO supports the need for rationalisation. However, to optimise the results achieved, a number of issues need to be addressed, including:

• clear articulation of performance management requirements;

• development of a Defence logistics executive information system to support integrated management;

• comprehensively addressing data integrity issues; and

• ensuring the movement to common systems is accompanied by commonality of process.

Gaps in available supply chain performance information

6.7. The existing network of logistic information systems does not support effective performance management of the Defence inventory. As

54 Report of the Defence Efficiency Review, 1997, op cit, p 52 55 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p 360

38

the DER team observed, managers have great difficulty accessing appropriate information in a useful format and within reasonable time frames.56

6.8. In the course of the audit the ANAO requested data from Defence regarding its holdings of spares, consumables and RIs, including valuations, stock movement rates and warehouse utilisation. It was requested that the data be provided at a fleet or inventory manager level. Defence found it necessary to invest considerable time and effort in extracting the data from a variety of information systems. Evidently this sort of information is not readily available to inventory managers.

6.9. A 1997 internal AF Log Comd. paper acknowledged that few of the primary logistics systems provide adequate aggregate-level performance data useful to managers. Various personnel commented on the inability of the Standard Defence Supply System (SDSS) to provide the management reports needed. It cannot provide an inventory manager with a report on the actual customer service level achieved for a specific range of items. Available reports summarise average performance for all items in a given district. This provides no ability to differentiate between items. Therefore, although inventory managers can vary the required service level for items, they cannot measure achievement against it.

6.10. Outstanding information requirements identified by various personnel included:

• summary of items needing reordering;

• summary of item managers’ workloads;

• summary data that pertains to inventory requirements of specific aircraft or ships;

• the value of inventory held in each district;

• identification of all districts holding excess stock by fleet; and

• inter-district transfers of inventory by fleet and Service.

6.11. Other significant gaps in information relate to the lack of system connectivity. For example, managers cannot measure the time taken between initial lodging of a demand by customer units and arrival of the



item in the customer’s hands. Inventory managers cannot accurately oversee the location and condition of all stock.

Need to consider performance management in system design

6.12. The development of information systems that ultimately do not provide required information in a useable form has been due in large part to a failure to consider performance management issues as an integral part of systems design and implementation.

6.13. For example, performance management was not an integral part of the design and implementation of SDSS by the Supply Systems Redevelopment Project (SSRP). Many of the performance measurement requirements now identified by the Services were not articulated at the time of its design in the early 1990’s. SDSS was designed and implemented to be a transaction-based system and was not specifically designed for performance management.

6.14. Although the commercial package on which it is based includes a suite of management reports, most of these proved useless in the Defence context because they had not been designed to support the management of items across multiple districts. The ANAO was advised that this had not been identified during the pilot testing of the system because the pilot had been conducted at single sites.

6.15. Army migrated to Version 1 in November 1993 but became frustrated with extensive delays in obtaining relevant management reports. The implementation of Versions 2 and 3 in the subsequent two years was the primary focus of SSRP. Although this can be understood in terms of its stated scope and budgetary obligations, the practical result was that Army (and progressively the other Services) were managing an extensive inventory with little relevant management information.

6.16. Each Service, and elements within them, have undergone separate learning processes in the use of structured query language (SQL) to extract management information. It is only in recent months that significant progress has been achieved. A more powerful query functionality exists through the use of the SAS query language. However, it is not clear that its availability has been adequately promoted to personnel.

6.17. Following extensive discussion, SSRP undertook to deliver a reporting tool (MIMSVu) by 1 July 1997 to replace SQLs. User-friendly and fully integrated with the commercial package on which SDSS is

40

based, it is planned to be distributed to middle managers (together with templates for standard reports and scope for specific ‘what if’ analysis).

6.18. Although this should provide the Services with a better capacity to generate relevant and timely management information, there is a need for a coordinated approach to its implementation and use. If this does not occur, the cultural problems that have resulted from the difficulty of extracting data will be entrenched, and personnel at various levels will continue to implement their own solutions.

6.19. As part of the logistics strategic planning now being conducted, Defence is attempting to address the need to plan strategically for its logistics information needs. The ANAO considers continuation of these embryonic efforts to be critical to the effective rationalisation and redevelopment of existing logistics information systems. Optimal results will only be achieved through a holistic approach to integrating the management of logistics and supporting information.

6.20. To support this, an essential element of any rationalisation strategy should be the clear articulation of the performance information requirements for management of the supply chain. This will help to ensure the systems that result provide better support than is currently available. Performance management requirements should then form an integral part of all future development of logistics information systems.


6.21. The ANAO recommends that a clear statement of performance management requirements, based on a Defence logistics performance management strategy, be articulated prior to, or in conjunction with, the development of a rationalisation strategy for Defence logistics information systems.

Defence Response

6.22. Agree.

Defence executive information system 6.23. Although MIMSVu should improve the timeliness and relevance of the performance information available to users of SDSS, it will not improve the ability to generate information relating to the performance of the broader supply chain. A Defence executive information system (EIS)


would provide a mechanism for drawing together information from a variety of information systems.

6.24. An executive reporting level has not been developed within SDSS to replace those that existed in the single-Service supply systems. As a result, many of the reports previously used by inventory managers were no longer available. Personnel in each Service expressed the view that their performance management capabilities had initially gone backwards with the introduction of SDSS. Each has subsequently taken a separate approach to addressing their information needs through the development of a Service-level logistics EIS.

6.25. However, the most appropriate environment to support integrated management would be a Defence-wide EIS that supported the provision of comprehensive performance information to managers at various levels. This would help to improve corporate oversight of the supply chain.


6.26. The ANAO recommends that Defence articulate a strategy and timetable, including milestones and performance indicators, to guide the development of a logistics executive information system to support consistent, coordinated performance management of the Defence supply chain.

Defence Response

6.27. Agree.

Data warehousing

6.28. A Defence EIS would need access to data resident within a variety of unconnected information systems managed by organisations which have different priorities and requirements. As part of a strategic planning process, HQADF in May 1997 embarked on a project to model the existing logistic system, identifying all information systems used, the data held, the uses made of each system and exiting connections and interfaces. Expected to take at least six months to complete, this should provide Defence with its first comprehensive understanding of the information systems currently used in the management of the supply chain. This is an essential requirement before progress can be made in developing a Defence logistics EIS.

42

6.29. A cost-effective option for achieving greater integration across these existing systems may be the use of a technique known as ‘data warehousing’ - recording and maintaining common data in one or more central warehouses from which it can be extracted and manipulated for various purposes. The DER logistics review team acknowledged the need to move towards data warehousing to reduce the number of separate logistic systems.57

6.30. However, Defence will need to evaluate the costs and benefits associated with this approach. Establishing a data warehouse approach may require significant capital outlay. In particular, there would be a need to purify and standardise data elements. This could be resource intensive, but the DER analysis has highlighted the need for such purification regardless of the approach taken.


6.31. The ANAO recommends that Defence review the cost-effectiveness of implementing a ‘data warehousing’ approach for the management of common logistics data.

Defence Response

6.32. Agree.

Data reliability and validity 6.33. The accuracy and validity of relevant data are critical to effective performance management. There are significant problems in this regard with much of the data resident within many of the information systems used in the Defence supply chain, affecting data elements such as stock level, price and unit of issue.

6.34. Defence has been aware of the poor state of its data relating to inventory for some years. For example, a 1989 report on Defence supply services by consultants Arthur Young noted that:

The pricing of inventory valuation has been identified as a matter of major concern by the Services and this impacts negatively on the reliability of the inventory investment cost…discussions with Department personnel have

57 Defence Efficiency Review, Logistic and Regional Support Review Team, Final Report, 1997, p 27


generally revealed that an estimate of the rate or degree of error in the Service databases cannot be established.58

6.35. None of the inventory managers the ANAO spoke to could identify the value of their inventory. Indeed, this was not an issue they regularly sought to monitor, because they had little confidence in the accuracy of the information generated by SDSS.

6.36. The valuation of inventory for the annual Defence financial statements does not rely entirely upon the pricing data in SDSS. For financial statement purposes, the quantities recorded in SDSS are merged annually with costing information from a number of sources.

Reasons for corrupt data

6.37. The ANAO could not identify any comprehensively attempt to analyse the reasons for data being corrupt. Possible causes suggested to the ANAO include:

• the progressive transfer of Navy and Air Force units to SDSS which necessitated the maintenance of data on two separate systems. Although the existing single-Service systems remained the official record for some time, they received diminishing attention from personnel anticipating the imminent migration to SDSS. Also, training on the use of the old systems was eliminated for new recruits for some months prior to their ‘turn-off’;

• no perceived need to correct information in the old systems prior to the transfer to SDSS;

• the generation of inaccuracies in the transfer of data to SDSS and in on-going transactions; and

• the lack of a culture that encourages and promotes data accuracy.

6.38. There has been no central area with authority or responsibility to coordinate or resolve these problems. Project Price Clean-up was initiated in the 1980s as a tri-Service program to correct data inaccuracies on the systems that pre-dated SDSS. It was never fully applied to the information systems, and initiatives to reactivate it were overtaken by the implementation of SDSS. Since then, there has been a

58 Arthur Young, Defence Supply Services report, July 1989, p 32, 3.69

44

single-Service approach to the clean-up of data. The progress of this has been very slow.

6.39. Following extensive negotiation, a code was applied in April 1997 to purify SDSS data for Air Force items in seven areas, including price and stock on hand. This had been identified as a requirement before the AIMS-BDS information system would operate effectively. However, there is also a need for appropriate strategies to maintain the data. The ANAO could identify no articulated strategy for ensuring this occurs.

6.40. As noted earlier, a verification exercise on the top 10,000 line items in SDSS conducted in conjunction with the DER produced a net reduction of $628m in inventory values and identified significant inaccuracies in a number of data elements. The Financial Reporting and Development Directorate in Defence acknowledged the value that had been obtained from the exercise and argued that such management attention will need to be continued. However, it is not clear whether this exercise will be extended to the remaining items on SDSS.

6.41. The DER recommended that the value of the ADF inventory be reviewed to ensure its accuracy.59 The addendum to the report also recommended that corporate guidance concerning uniform standards for data sets across Defence be developed and implemented.60 The ANAO supports these recommendations and considers that, as a matter of priority, a coordinated data purification and validation exercise of critical data elements should be conducted. This should form an integral part of the rationalisation of information systems.

6.42. Although the top 10,000 items account for nearly 70 per cent of the total value of items managed on SDSS, unreliable data diminishes the efficiency and effectiveness of the considerable management effort and resources expended on the other 1.6 million items.


6.43. The ANAO recommends that Defence:

a) develop and implement a strategy to validate the accuracy and reliability of all important data elements in logistics information system; and

59 Defence Efficiency Review, 1997, op cit, Annex E-7 60 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p 114


b) initiate strategies, including relevant performance measures, to ensure that the accuracy of logistic data is maintained and regularly validated.

Defence Response

6.44. Agree.

Lessons of SDSS project 6.45. SDSS was delivered through the SSRP, which was initiated in 1984.61 In 1990 MINCOM Pty Ltd was awarded the contract for the supply of its software package, Mincom Information Management System (MIMS). Although the original intention was to implement the software with minimum customisation, significant modifications were made.

6.46. The modified MIMS package was implemented as SDSS in three major stages between 1993 and 1995. Army converted to SDSS in a single event in November 1993. Navy and Air Force used a staged conversion over 1994 and 1995. Defence advised that between 1993 and 1997 approximately $134m was expended by the SDSS project and the Services on the implementation of SDSS, the Ship’s Logistic Information Management System - Supply (SLIMS(S)) and several other smaller logistics systems. Defence also advised that Air Force has expended a further $5.5m in project and personnel costs in the development implementation of the AIMS-BDS requirements determination system. SDSS replaced over a dozen single-Service supply systems which meant that, for the first time, the Services managed inventory through a single supply system, providing asset visibility never before achieved.

6.47. The ANAO did not conduct a comprehensive review of the SSRP within the scope of this audit. However, in the course of the audit it became clear that there have been a number of problems associated with the implementation of SDSS. Significant dislocation was experienced within the Services during and after its implementation and it

61 The Joint Committee of Public Accounts’ Report 317 ‘A champagne appetite but only a beer income - Defence’s Supply Systems Redevelopment Project’, June 1992, criticised aspects of Defence’s management of SSRP and expressed concern about its lack of progress. Audit Report No.19 1993-94 Project Audits: Defence Computer Environment, Supply Systems Redevelopment Project concluded that the evidence available indicated that administrative and management arrangements in place for the SDSS stage of the SSRP provided a reasonable basis to expect that project budgetary and event milestone targets would be met. It was noted that consideration would be given to undertaking an efficiency audit of the outcome of the SDSS stage at a later date.

46

has not ensured the consistent adoption of leading inventory management practices. As noted, the DER logistics review team recommended that Defence make greater use of common systems. In implementing this recommendation, it is important that Defence avoids the problems experienced with the introduction of SDSS.

No movement to common business practices

6.48. There was no compulsion placed upon the Services to fundamentally review or re-engineer their processes to take advantage of the functionality provided by SDSS. Each continued to operate in its existing configuration, changes being made only where necessary to make the new system work. Consequently, each developed its own methods for using the system.

6.49. The DER logistics review team noted that the Services have exhibited a focus on “retaining the way we do our business” rather than exploiting the opportunity to develop a greater core of commonality based on commercial best practice. The team commented that the absence of a common business approach, strategy or work practices has been a major impediment to the efficiency with which logistics services have been provided to the ADF.62

6.50. The business case for SDSS was based upon the generation of personnel savings resulting from the use of automated procurement functions. Each inventory management area lost positions following its introduction. However, the majority of managers spoken to by the ANAO expressed the view that they have not yet reaped increased efficiencies from its use. 6.51. One reason for this has been the large number of screens involved in each transaction, many with a cluttered and unfriendly layout. Another has been the inability of many inventory managers to rely on its automated functions. In some cases, as available funds are inadequate to satisfy all requirements, procurement recommendations must be manually prioritised or modified. In others, managers do not trust the system’s recommendations due to data inaccuracies. As a result, these managers are manually reviewing the majority of recommendations generated by the system. 6.52. As noted earlier, inventory managers have expressed particular dissatisfaction with the delay in providing them with an effective

62Defence Efficiency Review, Logistic and Regional Support Review Team Final Report, 1997, p 41


electronic commerce functionality. The use of mechanisms such as EDI and electronic funds transfer is seen as the principal means through which Defence will be able to generate the efficiencies expected of SDSS. For example, the DER analysis showed that over 87% of issues from stock other than the top 10,000 line items (737,198 issues in 1995-96) involved issues valued at less than $100. Electronic commerce can help to reduce the transaction and holding costs associated with those items. 6.53. Each of the Services also expressed dissatisfaction with the timing and extent of training provided to them on SDSS. A particular complaint is that the training did not provide deeper understanding regarding the nature and structure of the resident data. As a result, extensive trial and error was involved in understanding how to extract information.

Recognition of the need to address issues

6.54. The Logistic Business Systems Section (LBSS), formed in HQADF in July 1996, has worked at establishing a framework to guide and improve the operation of SDSS. It has produced a through-life support plan, draft business management document, draft security plan and is drafting a paper on business continuity. This sort of framework should have been completed before the delivery of the system to users.

6.55. LBSS advised the ANAO that it recognises the need to improve other aspects of the operation of SDSS and other logistic information systems, including:

• process re-engineering to ensure use of the functionality provided is being maximised;

• data accuracy; and

• full development of the system; e.g. introduction of EDI to enable process re-engineering.

Lesson of SDSS project

6.56. The fundamental lesson to be learnt from the SDSS project is that sup-optimal results occur when an organisation implements a common information system without ensuring there is also a common

48

process for its use. Establishment of joint logistics policy and support organisations as recommended by the DER should bring greater cohesion to the development and use of logistics systems. However, the experience with SDSS provides little evidence that the different Service elements absorbed into COMSPT will swiftly or willingly adopt common practices without committed leadership in this regard.

6.57. Although many of the issues identified by the Services are valid, it has been acknowledged that a significant proportion of the functionality offered by SDSS is not being used. Some estimates put this as high as 90 per cent. Identifying the extent to which perceived inadequacies relate to the system itself or to the use being made of it can best be determined by a post-implementation review. Such a review would also be important for documenting issues that should be avoided in future similar projects. This has yet to occur. The ANAO considers that Defence should conduct a post-implementation review of the SDSS project as soon as possible


6.58. The ANAO recommends that Defence conduct a post-implementation review of the Standard Defence Supply System (SDSS) with a view to providing guidance for the introduction of future corporate logistics information systems.

Defence Response

6.59. Agree.

49

1. The use of benchmarking to enhance performance This chapter examines the potential for Defence to increase its use of benchmarking to improve its logistical processes, including those relating to inventory management.

1.1. Benchmarking, or the search for practices that lead to superior performance,63allows an organisation to use functional and process comparisons to identify better practices in a valid and practical way. A ‘better practice’ is one which accomplishes specific work in a superior manner. Performance comparisons can assist an organisation to maintain continuous improvement and can play an important role in developing relevant performance measures. The Michigan study found that world-class firms are committed to the benchmarking of both metrics and processes.64

1.2. Benchmarking forces organisations to thoroughly review and understand their internal processes, determine current performance levels and define future objectives. The outcome of benchmarking should be the removal of non-value adding activities and a plan of continuous improvement based on the better practice identified.65

1.3. As noted in Chapter 4, the GAO in 1991 studied private sector efforts to improve logistics operations, visiting seven companies identified as leaders in integrated logistics management. All but one used benchmarking. Benefits reported by them included improved operations, increased efficiency, and reduced costs.66

1.4. Benchmarking by Commonwealth departments, both between each other and with the private sector, has been strongly advocated by the Government’s Management Advisory Board (MAB) and Management

63 Camp, R.C. , Benchmarking, ASQC Quality Press, Wisconsin, 1989, p. 12. 64 The Global Research Team Michigan State University, 1995, op cit, p. 241-246. 65 Office of Public Management, Premier’s Department, NSW, 1993, op cit, p. 17 66 GAO/NSIAD-91-210, 1991, op cit, p 2

50

Improvement Advisory Committee (MIAC). MAB/MIAC have reported that the most persuasive APS benchmarking work delivers practical recipes for demonstrable outcome improvements.67 The DER report supported the use of benchmarking by Defence.68

Methods of benchmarking 1.5. Benchmarking can be used to compare internal operations. It can also involve comparisons with external parties in both related and non-related fields. In most cases internal comparisons should be conducted first in order to maximise the benefits of existing corporate knowledge and experience.

1.6. Internal benchmarking involves comparing similar operations within an organisation. This allows for the identification and dissemination of better practice. Results can include:

• development of a common understanding of better practice within the organisation;

• documentation of internal business processes;

• increased asset utilisation;

• improved cooperation and morale; and

• higher productivity.69

Case Study 3 - Example of Internal Benchmarking

One company found that it had 43 different order processing systems indifferent business units and/or different countries of operation. Anotherfirm, organised into over 160 different business units, found widelyvarying logistic performance among the units. For example, fill ratesranged from a low of 80 per cent to a high of 98 per cent. Similardifferences were discovered on a broad range of performance measures.Some difference were justified by unique business unit circumstances butthe major cause was simply different levels of logistical competencies.No business unit exhibited best performance across the board. Theresult has been an extensive business process benchmarking initiative toidentify and disseminate best practice throughout the organisation.

Source: The Global Research Team, Michigan State University, op cit, p241-246

External benchmarking

1.7. Competitive benchmarking involves making comparisons against organisations in the same field of endeavour, and can be used to identify where improvement is most needed.

67 MAB MIAC Report No. 21, Raising the Standard: Benchmarking for Better Government, June 1996, p. 7. 68 Addendum to the Report of the Defence Efficiency Review, 1997, op cit, p. 130, 236, 295, 362 69 Logistic Benchmarking Service, cited in Price Waterhouse Urwick report, Report on Better Practice Principles for the Performance Management of Large and Complex Inventories, commissioned by ANAO.

51

meeting the target of 400 staff, the department was reduced to 75.

Ford may not have been able to achieve Mazda’s level of five due to organisationaldifferences, but the process of benchmarking against better practice helped it identifyand achieve greater savings than did internal examination.

Source: Leibfied H.J., McNair C.J., Vision Book Summary, Benchmarking - A Tool for Continuous Improvement, 1996, p 4

1.8. The potential benefits of competitive benchmarking are clear. However, in the commercial world, external benchmarking has also been expanded to include a wider, but selective, focus on the processes and performance metrics of top performing companies regardless of their industry sector.

Current benchmarking practice in Defence 1.9. Defence has made very limited use of benchmarking in the development of its inventory management practices. The ANAO could identify no formal benchmarking or comparison activities between Services. There has also been little benchmarking between supply-related organisations within Services, with most such activities ad hoc and informal.

1.10. The ANAO could identify no consolidated performance reporting that compared the performance of inventory management areas within Naval Support Command. The six-monthly Army Log Comd. performance report identifies the best and worst performing units against each performance measure. However, the report has focused on the reasons for poor performance rather than on identifying practices that have led to good performance.

1.11. A 1995 business process modelling study recommended that Army Log Comd. internally benchmark its logistic units and issue policy to promote wider adoption of the better practices identified. However, the ANAO found that there is still no formal framework that allows for the communication of better practices between Army logistic units or inventory management sections, although there has been increased focus recently on the importance of benchmarking. A recent Army Headquarters analysis of the Command’s performance concluded that:

… an examination of the units involved demonstrates a marked variation between one set of units at the top end (98.56% satisfaction) and others at the bottom (-16.73%). This is a classic case of the need for internal benchmarking to use the effective processes of the leading units in pulling into line the trailing units.70

70 Brief on Statistical Analysis of Performance Indicator Report, SO2 PM, HQ Army Log Comd., 21 Mar 97, p 2

52

1.12. Air Force has avoided direct performance comparisons to promote the pursuit of continuous improvement by each unit. However, this has resulted in limited focus on the sharing of better practices.

1.13. There have been some attempts at benchmarking with external organisations, but these have also generally been informal and ad hoc, and appear to have resulted more from individual initiatives than a structured approach to benchmarking.

1.14. The ANAO considers that there is considerable scope for an organisation with a supply operation as large and diverse as that of Defence to benefit from the increased application of the benchmarking techniques described above. Internal comparisons 1.15. Some areas within Defence indicated that they were concentrating on the stabilisation of internal processes and the development of a relevant performance measurement system before moving to a benchmarking program. This approach has some advantages.

1.16. However, the ANAO considers that more active use of cooperative comparisons across logistics units would facilitate continuous improvement within Defence’s inventory management processes and functions. For example, as noted in Chapter 3, many of the same problems appear to exist in each Service in the management of RIs. But RI managers in Army and Navy, including those responsible for aircraft-related items, were unaware of the work being done in this area by Air Force. There appears to be potential for inter-Service benchmarking, or at least wider dissemination and analysis of the findings of the Air Force re-engineering study.

1.17. This form of internal benchmarking should result in better practice across the board. Such forms of benchmarking are relatively simple to arrange, but require coordination to maximise the benefits. There has been no such coordination in Defence to date.

1.18. An example of the sub-optimisation that has resulted from this relates to the development of performance measures for warehousing activities. In June 1995 304 Air Base Wing (ABW) developed a set of measures for warehouse activities. At the time this represented leading practice in performance measurement for warehouses in Air Force, with most having none. In August 1995 it recommended to Air Command, which operates Air Force warehouses, that its approach be used by

53

others as it had realised considerable productivity and morale benefits from its implementation.

1.19. It was not until late 1996 that Air Command decided to implement the approach across other warehouses, due for completion by June 1997. An internal benchmarking network would have enabled the benefits realised by 304 ABW to be leveraged across Air Force more quickly and efficiently.

1.20. Had a Defence-wide benchmarking network existed, the 304 ABW measures, which were developed in isolation, could have been compared with the practices used across all Defence warehouses in order to identify Defence leading practice. As it is, they are being implemented with little modification to reflect the improved expertise developed by various elements within Defence, particularly AF Log Comd. and the DNSDC.

1.21. This example demonstrates the potential that internal benchmarking has for Defence. Much of the duplication of effort that has occurred in the development of performance measures and in the identification of better inventory management practices could have been avoided if internal benchmarking groups had been established.

External comparisons 1.22. There is also potential for Defence to benchmark with other defence and commercial organisations to assist it in designing improved inventory management practices. For example, the GAO has found inefficiencies in the RI pipelines of each of the US Armed Services similar to those experienced in Australia. Each of the US Services is developing initiatives to make their processes faster, better and cheaper.71

1.23. The GAO has also recommended that the US Armed Services examine the extent to which they can apply leading commercial practices, particularly those used in the airline industry, which it considered had the potential to generate significant savings.

71 GAO/T-NSIAD-97-109, Defence Inventory Management: Problems, Progress and Additional Actions Needed, 1997, p14; GAO/NSIAD-97-82, Inventory Management: The Army Could Reduce Logistics Costs for Aviation Parts by Adopting Best Practices, 1997, p 2-23; GAO/NSIAD-96-156, Inventory Management: Adopting Best Practices Could Enhance Navy Efforts to Achieve Efficiencies and Savings, 1996, p 2-20

54

1.24. The ANAO considers that Defence could obtain valuable information by studying the improvement initiatives being pursued in other organisations. The ANAO is aware that Air Force has had some contact with commercial airlines in this regard, but there is scope for such exercises to be much more extensive and coordinated.

1.25. The process of the CSP has acknowledged that many supply-related functions within Defence are also carried out in the private sector. There are significant opportunities to use benchmarking as part of the process of market-testing Defence logistics support services. The CSP also creates a strong imperative for achieving better commercial practice.

Utility of external benchmarking for Defence

1.26. Many personnel within Defence have expressed some scepticism about their ability to transfer specific practices from other organisations. It is argued that the unique nature of Defence renders comparisons meaningless. Few commercial organisations share the size or complexity of Defence, and other defence forces operate in different strategic and budgetary environments. Other factors identified to the ANAO included that:

• the unique nature of public sector requirements regarding the authorisation of expenditure can limit the usefulness of comparisons with industry, particularly in the use of electronic commerce;

• military specifications and requirements on Defence for air worthiness and safety are different from those required of commercial organisations; and

• Defence operates in a funding environment different from that of private industry.

1.27. These factors do make external comparisons more complicated. However, it is considered that much of the scepticism within Defence regarding the utility to be gained from such activities results from the fact that its comparison efforts to date have tended to concentrate on the specific metrics achieved by other organisations rather than the processes used.

Important to focus on process and capabilities

1.28. Although the comparison of metrics may be useful in identifying possible performance gaps, the real gains to be made from comparisons with non-related organisations result from studying the process by which

55

the metrics are obtained in order to gain insight into what drives superior performance. 72 Indeed, research has shown that the capabilities and competencies that define better practices are universal, transcending differences in supply chain characteristics.73

1.29. An important part of developing better practices involves understanding the most important processes determining performance, and identifying sometimes quite different organisations that perform similar processes. The recent MAB/MIAC report mentioned earlier supported this view, stating that:

…to participate [in benchmarking] your agency need not have direct equivalents in the public or private sectors. Part of benchmarking is to break down results and processes into elements that can be compared across a number of different types of organisations.74

1.30. Defence efforts at benchmarking have not taken this approach. An example relates to the use made of a 1996 benchmarking study of the standard time frames assigned for the movement of various freight priorities in other organisations. The study was used in defining the terms of reference for a review of AUSMIMPS, the Defence freight movement system.

1.31. Because the study found there is no universal standard for these measures, with required time frames varying by industry, commodity, company and location, it was considered questionable whether further benchmarking would add value to the review. However, this conclusion highlights a focus on metrics. Differences in the performance targets used by other organisations should not render further benchmarking useless. Rather, a benchmarking or comparison study that focused on identifying the processes and technology used by organisations that have excelled in reducing distribution cycle times would be of considerable benefit to Defence in its efforts to design a more effective distribution management system.

Benchmarking networks

1.32. The ANAO noted that, in many cases, Defence personnel have attempted to benchmark with an external organisation only to find it is not

72 The Global Research Team Michigan State University,1995, op cit p 20-23. 73 The Global Research Team Michigan State University, 1995, op cit p 13. 74 MAB MIAC Report No. 21, 1996 , op cit p 10.

56

appropriate. Not all potential partners are appropriate. Indeed, benchmarking with inappropriate partners can cause problems rather than solve them. It is for this reason that organisations which gain the most out of benchmarking compare themselves with a range of organisations.

1.33. There are two major organisations in Australia that facilitate logistic benchmarking - the Australian Quality Council (AQC), of which Defence is a member, and the Logistic Benchmarking Service, an Australian company that specialises in benchmarking logistics within Australian companies. These organisations facilitate studies aimed at benchmarking specific processes. Some use of the AQC’s services has been made separately by Air Force, Army and the DNSDC. More extensive use of such services may assist Defence in improving its use of benchmarking.

Need for better coordination of benchmarking activities 1.34. However, in order to successfully utilise benchmarking Defence needs a focused and coordinated approach. As Figure 7-1 illustrates, benchmarking is part of the continuous improvement and change management process.

Figure 7-1 Benchmarking in the context of continuous improvement and change

57

Review internal operations

Identify non-value added activities

Internal and external benchmarking

CHANGE

Improve performance, better/best practice

Immediateoperational

improvements

Key driversidentified

Performancemeasuresdeveloped

What tobenchmarkidentified

Source: Logistic Benchmarking Service

1.35. As a process, it should be governed by practices which ensure that ad hoc, non-value adding activities are avoided. However, there is no global policy or endorsed approach to guide benchmarking activities. Many attempts have treated it as a stand-alone activity, isolated from the overall performance management process. This does not maximise the value to be gained.

1.36. The lack of policy direction and coordination has led to costly duplication of effort in activities directed at identifying better practices. Failure to share experiences has led elements in each Service to repeat activities in which others have already invested time and effort.

1.37. This has been particularly evident in Defence in the development of performance management methodologies. As noted, each Service has been developing its methodology in isolation, communicating with each other infrequently and often only informally. Although many of the leading performance management principles

58

described in this report have been identified by personnel in each Service, they have done so separately and in isolation.


1.38. The ANAO recommends that Defence develop and promulgate a Defence logistics benchmarking policy that identifies clearly the processes and procedures to be followed.

Defence Response

1.39. Agree.

Role of benchmarking coordination agency

1.40. The ANAO considers that establishment of a body responsible for coordinating logistics benchmarking activities would assist in improving future efforts at improving supply chain management in Defence. It could provide a reference point for benchmarking expertise, as well as coordinating external benchmarking to avoid duplication of effort.

1.41. For example, internal benchmarking may identify the DNSDC as the centre of excellence for warehousing in Defence. Through the coordinating body, the DNSDC’s processes could be compared with those of external organisations, and any findings passed on through an internal warehousing benchmarking network. This process, depicted in Figure 7-2, could be used for each key logistical process within Defence.

Figure 7-2 Possible approach to coordinating Defence logistics benchmarking

59

DefenceLogistic

BenchmarkingCoordination

Agency

Internal BenchmarkingDefence BestPractice byFunction

Service BestPractice byFunction

Betweenlogisticunits

BetweenServices

External Benchmarking

Other DefenceForces

OtherIndustries

Suppliers

World BestPractice byFunction

BenchmarkingFacilitators and

Groups

1.42. The ANAO considers that Defence has much to gain from the use of both internal and external benchmarking as part of its logistics performance management strategy. It would provide a useful mechanism for enhancing progress already made in improving inventory management practices. It would also facilitate the major change process confronting Defence. It is acknowledged that the use of formal benchmarking activities should be subjected to cost/benefit analysis to ensure their use is warranted, particularly during the current period of significant organisational change. However, the use of more informal and less expensive comparison studies and activities, particularly internally, would assist Defence in ensuring it obtains the maximum benefits from the current reform process.


1.43. The ANAO recommends that Defence identify a logistics benchmarking coordination body responsible for coordinating:

a) the establishment and operation of internal benchmarking networks across Defence;

60

b) Defence participation in logistic benchmarking or process study activities both with other defence and commercial organisations; and

c) the provision of adequate training and guidance regarding the conduct of logistics benchmarking and comparison activities.

Defence Response

1.44. Agree.

61

2. Achieving cultural change This chapter highlights the need for Defence to take a more strategic and coordinated approach to addressing the training and educational issues associated with developing a strong performance management culture.

2.1. An organisation’s culture is the underlying assumptions, beliefs, values, attitudes and expectations shared by its members, which affect their behaviour and the behaviour of the organisation as a whole.75 As in many other organisations, inventory management in Defence has historically been dominated by a ‘just-in-case’ culture in which the focus has been on holding significant levels of safety stock to ensure demands can be met as needed.

2.2. Private sector companies studied by the GAO have found that achieving cultural change in their inventory management philosophy was critical to making inroads against problems such as excessive inventory. If Defence is to achieve significant and lasting change in its inventory management, it is critical that it takes a strategic approach to developing a culture which encourages a ‘whole-of-business’ approach.

2.3. There is evidence of change occurring in the Defence inventory management culture, but this has been uneven across the Services and the inventory. The establishment of a joint support organisation should assist Defence in capitalising on that progress. However, the ANAO considers an essential element in achieving this will be the embedding of an integrated performance management within the policy and processes of the COMSPT organisation.

2.4. International experience has shown that one of the key requirements for successfully managing change is commitment at all levels.76 Managers must demonstrate commitment by giving full support for the change, setting a strategy for its implementation and allocating the resources necessary to achieve it. The measures then used to assess performance will serve to communicate desired behaviour.

75 GAO/NSIAD-94-193 Organizational Culture: Use of Training to Help Change DOD Inventory Management Culture, 1994, p 1 76GAO/NSIAD-94-193,1994, op cit, p 4

62

2.5. As was discussed earlier, the ANAO considers that development of a Defence performance management strategy will be important in that regard. However, identifying what to measure will not in itself produce significant improvements in performance. Important issues that would need to be addressed to achieve the cultural change discussed above include:

• personnel involvement in the development of performance measures and targets;

• provision of appropriate training and guidance; and

• that performance information is seen to have been used appropriately.

2.6. These are all issues that have not been well managed in the past in Defence.

Personnel involvement in performance measurement 2.7. Performance measurement is most effective when personnel have input into the systems used to measure them, including involvement in the analysis that leads to an understanding of the processes to be measured. This ensures there is a full understanding of how the process is actually working. It also supports the continuous improvement focus that only results when performance is accurately and appropriately measured. 2.8. Both Army and Air Force Logistic Commands have concentrated their supply-related performance management efforts on the central development of common measures against which their units report. A consistent theme identified by the ANAO in these units was that these measures had little relevance to their internal management issues.

2.9. The units acknowledge it is important that Headquarters obtain information that allows it to understand the performance of their support function. However, there is a strong feeling this should be drawn from measures useful at the unit level. This would allow units to retain a feeling of control over the process and reduce the data collection burden. Many of the current supply-related measures are seen as labour intensive tasks imposed for little benefit.

2.10. Balancing that need for personnel involvement with the need to ensure personnel are not overly diverted from their principal tasks has been difficult for the Commands. Late in 1996 HQ Army Log Comd. developed the first effective inventory status reports that had been

63

available from SDSS (some three years after Army migrated to the system). These reports are to be produced bi-annually directly by Headquarters but available to units only on request. Although this may have been intended as a mechanism to reduce the units’ reporting requirements, some units view it as representing a loss of ownership of performance data.

2.11. The ANAO considers information such as this should be more readily accessible by personnel directly responsible for inventory. Indeed, its use to improve inventory management should be actively encouraged rather than relying on the initiative of individuals to seek it.

2.12. The involvement of large numbers of people in the development of performance measures can cause excessive delays in the process. However, experience in the Services has shown that personnel at all levels need to be sufficiently involved to ensure performance measures and targets are perceived as valid and useful. Without that ‘buy-in’, they are unlikely to generate true performance improvement.

Training and education important for change 2.13. Training has been shown to be a key vehicle for helping organisations change their culture. For example, to understand better what US Defense needed to do to change its inventory management culture, the GAO obtained views from academic experts and officials from nine large private sector companies. They indicated that, together with top management support for desired values and beliefs, a key to effecting true cultural change is the provision of training to convey those values and beliefs and develop the skills needed to implement them.77

2.14. Greater accountability for performance must also be accompanied by an environment in which personnel feel they are able to affect the outcome. Education helps to achieve this by improving understanding as to how measures were derived and how the performance achieved against them can be affected.

2.15. Comprehensive performance management training has not been provided across Defence and resource shortages have limited access by many staff to the training that has been available. An effective performance management training and education program would assist Defence in two ways.

77 GAO/NSIAD-94-193, 1994, op cit, p 4

64

Improved validity of reported information

2.16. Training would improve acceptance of the performance management strategy, helping to improve the validity of the information generated. Experience in the Services has shown that imposing performance measurement without providing the training needed to gain its acceptance can result in sub-optimal information.

2.17. A 1995 consultancy study in Army Log Comd. reported that:

‘…there is evidence, both anecdotal and directly observed, that there is some misrepresentation of performance indicators…the study team were quoted the example of standard delivery dates being deliberately changed or falsely entered on the Cargo Visibility System to report better than actual achievement.’ 78

2.18. Units provided the ANAO with anecdotal evidence that some misrepresentation of supply chain related performance information may still occur. Analysis by HQ Army Log Comd. of reported performance for July to December 1996 also highlighted the need for education to overcome issues that were contributing to poor performance or rendering reported results unreliable.

2.19. For example, demand satisfaction rates were found regularly to decline between December and May. A number of possible causes were identified including a lack of initiative by staff in moving freight efficiently and inputting data. It was also noted that 74 per cent of units had failed to provide responses to this performance measure. The report considered that this indicated responses may be supplied only by units with the greatest interest or best results, with others perhaps hiding poor results or simply lacking interest or faith in the system. It was concluded that there is a lack of understanding of why the data is needed.

2.20. Similar problems have been shown to exist in AF Log Comd., where it has been acknowledged that there is a poor level of understanding of how to manage performance effectively through the collection and analysis of data. A survey conducted in one logistics unit in 1996 showed that 44 per cent of personnel were not aware of the concept of key performance indicators, with many seeing no benefit in performance measurement.

78 Logistic Business Modelling Study, Simsion Bowles & Associates, 1995, p 28

65

Support cascading of integrated performance measures

2.21. Comprehensive performance management training would assist personnel in developing appropriate and consistent lower-level process and outcome performance measures that are relevant to the overall strategy. Although each Service logistic or support command has directed its units to develop performance measures, none has provided those units with the guidance and training needed to ensure the resulting measures are appropriate, relevant or consistent.

2.22. The ANAO observed other examples in which personnel responsible for developing performance measures for use in supply-related CSP contracts were provided with little relevant training. It has been up to individuals to pursue training through their own initiative.

2.23. The ANAO considers that Defence needs to include performance management skills among the core competencies required of logistics personnel. Ensuring that performance management is incorporated as an integral part of training would help to develop a culture in which its application is viewed as a natural and indispensable part of logistics management.

Use of reported information

2.24. Gaining cultural acceptance of performance management also requires personnel to see that information collected is used appropriately and valued by management. Personnel must see their contribution to measuring performance as an effective use of their time. This has often not been the case in Defence.

2.25. There is little evidence of analysis of the monthly reports provided being fed back to Air Force logistics units. Army units expressed similar concerns. Although HQ Army Log Comd. produces a bi-annual report summarising the performance reported, in most cases, feedback generally occurs only when explanations are sought for poor performance.

2.26. A concern expressed by one unit related to the use by Headquarters of the standard performance report to make decisions that are based on inappropriately-framed performance measures or invalid assumptions. Other complaints relate to the failure of the performance report to weight reported performance to account appropriately for differences in the scale and nature of activities of different units.

66

2.27. The accuracy and validity of reported information may be undermined if personnel perceive that it will not be used appropriately or that it serves no purpose. Defence would benefit from establishing improved mechanisms for the analysis of achieved performance. The use being made of performance information should also be reviewed to ensure its value warrants the effort involved in its collection.

Incentives

2.28. AF Log Comd. has discussed the potential to employ some form of ‘gainsharing’ to provide greater incentives for units to produce savings through their own initiative. Although acknowledging the need to establish policy and guidelines, the principle of units and directorates retaining a proportion of those savings to re-invest in further improvement activities has been accepted. The other Services could also consider such a system to facilitate cultural change and promote a focus on process improvement.

Benefits of a strategic approach

2.29. There have been few attempts within Defence to articulate a specific strategy for addressing the educational and cultural issues identified above; yet international experience has pointed to the need for an effective overarching strategy to achieve lasting cultural change in inventory management.

2.30. For example, the GAO recently reported that half of US Defense’s US$69b inventory of spare parts, medical supplies, hardware, food and clothing is either obsolete or rarely used. Underlying causes included inefficient inventory management practices, inadequate inventory oversight and overstated requirements. The GAO concluded that, to overcome these problems, US Defense must establish goals, objectives and milestones for changing its culture and adopting new management tools and practices.79 The ANAO considers that Defence would benefit from adopting a similar approach.

2.31. The benefits of a structured approach to this issue have become apparent at the few locations within Defence where this has occurred. For example, the ANAO noted that at both the DNSDC and the Avionics Maintenance Flight at Amberley, personnel at all levels are involved in the development and use of performance measures to improve overall performance. As a result, genuine cultural change appears to be

79 Extract from Reports & Testimony, March 1997, p 38 - GAO/T-NSIAD-97-109

67

occurring. These initiatives have succeeded in giving staff ownership of their processes, helping to build a continuous improvement culture.

2.32. However, the difficulty of implementing comprehensive change through isolated efforts was exemplified in a 1996 article in the magazine, NavalSupply, by the officer responsible for developing a quality system for the logistic systems directorate. He argued his task had been made more difficult by the lack of a quality ethos in the surrounding NSC culture and that this had contributed to the development of a range of incompatible systems.80

2.33. A more coordinated and strategic approach to addressing these cultural issues will assist in achieving lasting improvement in inventory management.


2.34. The ANAO recommends that Defence develop and implement a logistics performance management deployment program for the Defence support command that encompasses:

a) provision of appropriate education and training at all levels regarding the development and use of credible logistics performance measures, and

b) implementation and training in the use of appropriate performance analysis tools and techniques.

Defence Response

2.35. Agree.

80 ‘Screaming Into the Abyss’, LCDR Steve Hood and LEUT Dave Dykstra, NavalSupply newsletter, Dec 1996, p 27, advance copy provided to DER

68

PART THREE

Defence Quality Assurance Preliminary Study

69

Defence Quality Assurance Summary

1 The Department of Defence has for many years performed quality assurance (QA) of supplies ranging from major capital equipment through to replacement stores and consumables such as fuels. The conduct of QA in Defence has been subject to wide-ranging change in recent years, and further changes are proposed as part of the Defence Reform Program.

2 Because of this major reform program, the ANAO’s preliminary study did not proceed to a performance audit. The preliminary study made several observations which Defence could consider in implementing the Defence Reform Program. These observations include:

• relationships between Defence Quality Assurance (DQA) and its customers throughout Defence have improved, but could be improved further by means such as improved cost visibility;

• there is no comprehensive statement of quality policy, or statement of relevant performance indicators (measures of QA performance);

• services offered by DQA do not match those requested by DQA’s customers in Defence, and there needs to be a reallocation of DQA resources according to need; and

• DQA might improve assessment of its overall effectiveness by combining assessments of staff skills, DQA procedures and customer feedback.

3 The Department of Defence stated it had no significant disagreements with the contents of this report.

Background

4 The Department of Defence has for many years performed quality assurance (QA) of supplies ranging from major capital equipment through to replacement stores and consumables such as fuels. QA is

70

defined as ‘all those planned and systematic activities necessary to provide adequate confidence that goods or services will satisfy requirements for quality.’81 QA is primarily the responsibility of the supplier; the role of DQA is mainly to audit the QA systems of suppliers and to provide advice to Defence on specifying QA requirements when purchasing supplies.

5 The conduct of QA in Defence has been subject to wide-ranging change in recent years. Individual QA services for Navy, Army and Air Force were combined into a single Defence Quality Assurance Organisation (DQAO; later renamed DQA) and the total number of QA staff declined from some 1100 to about 300. The role of DQA was changed from a quality control function of testing and inspection to a quality assurance role emphasising auditing of suppliers’ quality systems. In 1996 DQA regional offices were merged with the Defence Acquisition Regional Offices.

6 DQA performs its tasks in response to requests by its customers such as staff of Defence’s major capital acquisition projects and purchasing staff within the three armed Services. The current cost of operation is approximately $20 million per year.

7 Late in 1996 the ANAO began a preliminary study of DQA to assist in deciding whether to conduct a performance audit.

Defence Reform Program

8 In October 1996 the Minister for Defence established the Defence Efficiency Review (DER) under the chairmanship of Dr Malcolm McIntosh. It was to make recommendations for reforming Defence management and financial processes. The DER report was released in April 1997. It mentioned that quality assurance was among the procurement services that could be outsourced.82 When releasing the

81 International Organisation for Standardisation, Quality management and quality assurance - vocabulary. ISO 8402.

82 Future Directions for the Management of Australia’s Defence - Report of the Defence Efficiency Review 10 March 1997 (Directorate of Publishing and Visual Communications - Defence Centre Canberra) p25.

71

report the Minister announced that a Defence Reform Program based on DER findings and recommendations would be implemented as quickly as possible.83

9 In a separate Addendum to the report, the DER Secretariat made detailed recommendations, of which some are regarded as being only advice to Defence in implementing the outcomes of the DER. In respect of QA the DER Secretariat recommended that:

• QA services should be delivered to Defence customers on a ‘user-pays’ basis;

• QA policy should differentiate between QA requirements according to bands of procurement risk;

• QA resources [i.e. personnel and funding] should be dispersed among Defence customers, and management overheads (in the order of 60 positions) harvested in the short term, leaving a small cell to maintain QA policy; and

• further rationalisation of QA should be pursued in the longer term, with an aim of an overall 50% reduction from current staffing levels.84

Preliminary Study Conclusion

10 Because of this major reform program, the ANAO’s preliminary study did not proceed to a performance audit at this time. This brief report records audit observations made during the preliminary study which Defence could consider together with the DER recommendations. These observations include:

• relationships between DQA and its customers throughout Defence have improved, but could be improved further by means such as improved cost visibility;

83 Minister for Defence statement MIN61/97 11 April 1997 McLachlan Announces Defence Reform Program.

84 Future Directions for the Management of Australia’s Defence - Addendum to the Report of the Defence Efficiency Review - Secretariat Papers (Directorate of Publishing and Visual Communications - Defence Centre Canberra) pp158, 199 and 200.

72

• there is no comprehensive statement of quality policy, or statement of relevant performance indicators (measures of QA performance);

• services offered by DQA do not match those requested by DQA’s customers in Defence, and there needs to be a reallocation of DQA resources according to need; and

• DQA might improve assessment of its overall effectiveness by combining assessments of staff skills, DQA procedures and customer feedback.

Defence Response

11 The Department of Defence stated it had no significant disagreements with the contents of this report. Defence also stated it had carried out a review to consider the various approaches to QA identified in the Addendum to the Defence Efficiency Review report, and to recommend key changes which would optimise the cost effective provision of QA. The recommendations of that review will be considered by senior management within the Defence Acquisition Program, and the agreed way ahead will then be implemented as part of the Defence Reform Program.

Conduct of the preliminary study

12 This preliminary study was conducted between December 1996 and March 1997. The audit approach was first to review public information, and hence establish some issues to examine. These issues were:

• performance information and cost-effectiveness of DQA;

• DQA’s relationships with its Defence customers;

• DQA’s services; and

• QA practices and procedures.

13 A fifth issue, industry relationships, was identified during the preliminary study but considered only briefly. Most of these issues would be affected by the DER recommendations if implemented.

73

14 Criteria were developed during the preliminary study covering each of the four issues listed above. These criteria were forwarded to DQA for comment, and used as a basis for field work.

15 Fieldwork included interviews with customers and DQA staff, examining previous reviews and inspecting relevant files and data from management information systems. Fieldwork was conducted in Canberra, Sydney, Melbourne and Adelaide. The preliminary study was conducted in accordance with ANAO auditing standards and cost $90 000.

16 The audit team was assisted by two consultants: Mr B.R. Hore of Quality Management Services Group Pty Ltd and Mr D.P. Hurst MBE.

Performance information and cost-effectiveness

17 The ANAO considers that performance information should be linked clearly to top-level statements of objectives, through clearly defined strategies and a process of establishing appropriate performance indicators and measuring results against such indicators.

18 However, the ANAO found that Defence has not established a comprehensive statement of quality assurance policy. Neither has DQA assessed its own performance, other than through customer satisfaction reports. This limits DQA’s ability to objectively assess its performance and may have detracted from its ability to market itself to customers. Should DQA resources be dispersed to customers, the marketing need would no longer apply, but the need for managers to understand the performance of quality assurance work will remain.

19 Assessing performance would be aided by a statement of QA policy and objectives, establishment of appropriate performance indicators and targets and the implementation of mechanisms which review progress against these targets. Prime responsibility for establishing and reviewing this information lies with line management. However, the monitoring process in particular can be aided by ensuring periodic independent reviews such as internal audit or program evaluation.

20 One measure of effectiveness is the extent to which QA advice is accepted by customers. In the preliminary study, the ANAO found that almost all QA advice was accepted.

74

Customer relationships

21 The ANAO considered aspects of customer relationships such as mechanisms for agreeing on work to be done and whether DQA services met agreed requirements. Other criteria considered included consistency between regions of DQA, and visibility of costs to customers.

22 We found that DQA had made considerable improvement in its relationships with Defence customers since early 1996, but that some customers still perceived difficulties. Customer satisfaction reports indicate that services usually met requirements although there were a few complaints about DQA performance. Most complaints were that QA services had not been supplied as requested or had not detected defects in supplies. A lack of consistency between regions had caused difficulties for some customers; for example a service agreed to in one State would be refused in another. Also, there was no visibility of the costs of specific QA services to either the customer or to DQA. DER proposals for dispersal of QA resources to customers and for ‘user pays’ should help to overcome these concerns.

DQA’s services

23 The ANAO considered plans for the resourcing of QA services and whether customers were informed of the range of services offered.

24 We found that the range of services required by customers does not match the range offered by DQA. Nor are the services offered well understood by customers. DQA has, however, been changing its views on what services should be provided.

25 DQA and its predecessors have been responsible for providing certification of selected Defence suppliers to recognised quality standards (the ISO 9000 series). This service will cease for new contracts in 1999 but meanwhile there is an ongoing role in ensuring that these suppliers meet the requirements of the standard. DQA can also certify particular organisations in Defence to ISO 9000 and provide risk consultancy. These services are appreciated by the elements in Defence that have used them but are not widely known.

26 QA services relating to overseas purchases are provided either by relying on manufacturers’ documentation stating compliance, through arrangements with foreign QA authorities or through a limited presence

75

of DQA officers overseas. These mechanisms appear less rigorous than those for QA of local purchases. Differences between QA for overseas supplies and local supplies could be reviewed in the context of the DER recommendation for differentiation of QA according to procurement risk.

27 DQA’s customers reported that most requests for QA services accepted by DQA were performed in a timely fashion. Similarly DQA managers reported that there was no significant resource constraint in meeting current demands except for some specific skills such as software quality auditing. The DER has suggested that a 50% reduction from current staffing levels should be pursued. Given these considerations, as well as concerns raised above on the matching of services to customer demand, there needs to be a reassessment of the desirable level and mix of QA resources on the basis of expected workload taking into account any changes in demand which might arise should user charging or distribution of QA resources to customers be implemented in the near future.

QA practices and procedures

28 The ANAO considered a number of aspects of the conduct of individual QA tasks. These included the effectiveness of QA audits, the skills of QA staff and mechanisms for continuous improvement of practices.

29 DQA conducts surveillance of suppliers’ quality systems, including the critical review of suppliers’ internal QA audits and corrective and preventive action programs. In the preliminary study the ANAO was unable to assess directly the effectiveness of DQA in this respect. This was due to a lack of analyses by DQA of QA results (for example, there was no systematic recording of the frequency and type of non-conformances detected) and to the inherent difficulties of measuring the impact of QA work. The impact is difficult to measure because the quality of supplies becomes apparent only after supplies have entered service, and is affected more by the manufacturer’s quality control systems than by DQA.

30 One approach to assessing QA effectiveness would be to evaluate customer complaints, QA staff skills and DQA procedures. As noted previously, most customer satisfaction reports were positive. Staff skills and procedures are discussed below.

76

Skills and knowledge of QA staff

31 Given that many DQA tasks are carried out by staff working individually and using their judgment at a supplier’s establishment, the question of appropriate skills and knowledge is important. With arrangements for quality assurance subject to significant change in the context of the Defence Reform Program, a strategic staff development plan is needed. It might consider whether there will be a shift from QA of production to QA of planning and design, which could require skills at a professional engineering level rather than at a trade level.

32 The relevance of technical background to quality assurance has not been well defined. Both DQA managers and customers felt that QA people should have a technical background appropriate to the type of industry for which QA services are provided. However, a DQA skills survey emphasised generic tasks, and did not address specialist technical knowledge despite this being listed as a customer need. DQA has attempted to recruit people with appropriate technical background and skills but has not then ensured that these technical skills are maintained.

33 There has been a long-term problem in developing software QA skills. DQA should review, in the context of the implementation of the DER proposals, whether it would be more efficient to deliver software QA services through a single software QA group that provides services to all Defence customers and regions.

DQA procedures

34 DQA has a program of continuous improvement (CI) of its activities which is managed centrally through a CI section with a register of recommendations and actions. Steps have been taken to sharpen the focus of the CI program, and these could be taken further. Reviews of completed QA services offer another learning opportunity for DQA. DQA should confirm the requirement for these reviews, and bring their analysis within the CI program. If QA services are to be devolved to customers, the continuation of some CI process to enable sharing of information would seem even more valuable.

35 DQA’s information system is inadequate to support its work. A proposed replacement system could help to provide basic information on the costs of operations and to monitor the performance of specific

77

suppliers. DQA could also examine the option of using the Standard Defence Supply System, which would enable immediate exchange of information with customers.

Canberra ACT P.J. Barrett xx xxxx 1997 Auditor-General

78

PART FOUR

Appendixes

79

Appendix 1 ANAO analysis of a sample of Defence supply chain performance measures against better practice criteria

(Refer to discussions at Chapter 2, 4.10 and 4.29)

A sample of 354 inventory and supply chain related performance measures used or planned for use at the unit and corporate level in each Service were analysed against the leading practice principles identified in Chapter 2. The sources for the sample are described at Appendix 2. In order to achieve some comparability across the very different Service structures, ‘unit level’ has been defined as including national fleet managers within Army Log Comd., materiel management sections in NSC and logistic management squadrons and units in AF Log Comd. ‘Corporate level’ relates to measures defined at Command headquarters level and required to be reported against by logistic managers.

The four parts of the table below summarise this analysis which considered:

1. the extent to which Defence performance measures provide a balance between:

• qualitative and quantitative measures; • measurement of results and measurement of enabling

processes; • internal measurement of performance and external

measurement; and • lagging and leading performance measures (refer para 2.4);

2. the coverage provided of each of the four perspectives in the balanced scorecard (refer Figures 2-1 and 2-3);

3. the coverage provided of three of the five functional areas in which continuous measurement is required for leading practice (refer para 2.28); and

4. the ratio of measures examined for which a performance target had been articulated.

No of 354 53 49 19 131 10 56 36

80

Criteria Overall

(%)

Army

Corp

(%)

Army

Unit (%)

AF Corp (%)

AF Uni

t (%)

Navy

Corp

(%)

Navy Unit (%)

DNSDC

(%)

Quantitative 97 100 100 100 97 100 88 97 Qualitative 3 0 0 0 3 0 13 3 Result 80 75 69 95 84 80 75 89 Process 19 25 31 5 16 20 25 6 External 7 0 2 26 8 0 14 0 Internal 93 100 98 74 92 100 86 100 Leading 29 4 33 21 39 10 30 33 Lagging 69 87 67 79 61 90 70 67 Customer service perspective

32

26

27

68

32

67

30

22

Internal business process perspective

55

47

63

16

60

29

59

67

Internal growth perspective

7

0

0

11

9

0

18

0

Financial perspective

17 38 8 32 13 20 18 6

Asset Utilisation

15 32 4 21 11 40 14 6

Quality 14 4 35 21 8 0 21 6 Productivity 21 32 2 16 28 0 14 28 Targets Set 20 15 71 0 6 0 5 47

Source: ANAO analysis Note 1: Army Corporate includes ADF Line Haul Service Note 2: Some totals< 100% due to: • inclusion of DNSDC CSP-specific measures in overall scores but not in unit/corporate breakdown • inability to make assessment due to lack of information, particularly for measures not implemented.

81

Appendix 2 Description of Defence supply chain performance measures analysed

Service

Defined level (Corporate/Unit)

Source of performance measures Purpose of

Army Corporate Command Level Customer Support Agreement

Measuring p

Corporate LCI BUS MNGT 8-1 Measuring p Corporate LCI BUS MNGT 8-1 Measuring p Corporate ALI POL 1-1 Performance

customer un Corporate FDC Monthly Statistics Reports Measuring p

Centres Corporate ADFLHS Annual Report Performance Corporate Distribution performance/freight

movement Statistics on

Corporate LCI BUS MNGT 8-1 Measuring p Corporate Proposed measures for dispatch and

deliver function Measuring tfunction

Unit South Queensland Logistic Group (SQLG) draft Service Level Agreement, Chapter 16 Warehousing and Inventory Management, Performance Measurement

Measuring punder devel

Unit Draft CSP contract for stores provision at SQLG

Measuring punder devel

Unit FAU SQLG monthly statistics collection Internal colleperformance

Unit FPA for clothing, pharmaceuticals, vehicle spares

Measuring p

Service



Air Corporate Logistic Performance Report (revised Measuring p

82

Force version - some not yet measured) squadrons (units (LMU)

Unit ALLMSQN Procedures Manual DI(AF) AAP 7007.003-1,Statistical Techniques and Performance Measurement

Proposed inperformancequality accre

Unit MOU between 82 Wing and 501 Wing for maintenance of F-111 aircraft

Internal meaWing in supcustomer

Unit ALLMSQN Performance Board Internal mea Unit Balanced Scorecard, Avionics

Maintenance Workshop, 501 Wing Internal meadata for use

Unit Performance measures identified by SRLMSQN, Army LMSQN, GSELMU, CSELMU, TALMSQN, and ALLMSQN in response to Command Execution Objective 4.3.1 1996-97 requiring logistic units to ‘identify five parameters that will provide a measure of logistic performance in supporting their operational customers’

Measuring pproviding logcustomers Note: Major

Unit Warehousing performance indicators developed by 304 ABW (developed at 304 ABW in June 1995; remainder of ABW directed to implement in March 1997)

Internal meaexternally re

Unit Monthly Activity Report, Procurement Section, Warehouse, 301 ABW

Internal mea

Unit Warehouse performance data collected by LSF, 301 ABW

Internal mea

Service



Navy Corporate Performance outcomes and measures, Bi-Annual Preparedness Report, Portfolio level reporting

Portfolio-levproviding logoperational

Corporate Performance outcomes and measures, Bi-Annual Preparedness Report, Sub-

Measuring p

83

Program level reporting Unit NSC Management Information Centre Electronic d

performance Unit Business Plan 1996-97, Superintendent

General Logistics Measuring pmanagers

Unit Supplier measurement by Aircraft Materiel Support Office, HMAS Albatross

Measuring trepairs cont

Unit HMAS Coonawarra performance indicators

Measures oSupport Com

Unit Quarterly Performance Report, HMAS Albatross

Measuring u

Unit Materiel Manager Weapons Systems internal performance indicators

Internal mea

Unit Draft SOR, Materiel Manager (General and Components) FFG Class Logistic Office, Monthly Performance Report

Measuring msupport of F

Unit DNSDC performance including Navy factors

Information Navy

Other DNSDC MOA, DNSDC - KPI (revised definitions Oct 1996)

Measuring poperation

DNSDC performance reported against MOA

Performanceperformance

Weekly reporting to HQ Army Log Command by DNSDC Oct 96 - Mar 97

Measuring p

Ad hoc reporting by DNSDC Providing in

84

Appendix 3 ANAO comparison of performance information availability in Defence with leading practice

In 1995 the Global Logistics Team, Michigan State University published the results of a study, 'World Class Logistics: The Challenge of Managing Continuous Change', in which the logistic performance of 3,693 firms from the United States, Europe, Asia and the Pacific was assessed (based on an indexed score). Columns 2 and 3 of the table below are from that study, and identify the percentage of firms in both the top and bottom thirds of the Michigan sample that used the performance measures listed in column 1 (measures relating to purely commercial enterprises have been omitted). Based on the sample of Defence performance measures analysed at Appendix 1, Columns 4-10 identifies measures for which the ANAO could identify some usage of commensurate measures in Defence. (Refer to paras 2.25 and 4.10-4.27 for discussion of the implications of this assessment)

Measurement Category Michigan study comparison of

information availability

ANAO assesuse at unit

Defen(Column 1) Upper 1/3

(Column 2)

Lower 1/3 (Column 3)

Navy (Col 4)

Army (Col 5)

Asset Management Inventory turns 96.9 94.7 Inventory levels, number of days’ supply

100 94.7

Obsolete inventory (See Note 1) 96.8 100 Inventory classification (A,B,C), (See Note 2)

93.6 94.7

Cost Total logistic/supply chain costs 93.8 95 Logistic/Supply Chain Cost per unit 83.9 83.3 Inbound freight 77.4 79 Outbound freight (See Note 3) 100 95 Administrative costs 93.8 84.2 Warehouse order processing 87.5 80

85

Direct labor 96.8 84.2 Comparison actual vs budget (See Note 4)

100 100

Cost trend analysis 93.8 90 Inventory carrying cost 86.7 73.7 Cost of returned goods 81.3 75 Cost of damage 87.5 80 Cost of service failures 40.6 47.4 Cost of back order 33.3 27.8





(Column 2)


Navy (Col 4)

Army (Col 5)

Customer Service Fill rate On time in Full (%) 96.9 61.01 Note 7Stock outs (%) 93.8 77.8 Shipping errors (%) 90.6 73.7 On-time delivery 93.6 89.5 Back orders 79.3 76.5 Order Cycle time (See Note 5) 90.6 72.2 Delivery consistency (See Note 6) 87.5 77.8 Response time to inquiries 41.9 29.4 Response accuracy 32.3 22.2 Complete orders 90 52.9 Customer complaints 71 79 Overall reliability 70 29.4 Overall satisfaction 58.1 55.6

Productivity Units shipped per employee 83.9 57.9 Units per labour dollar 67.7 57.9 Comparison to historical standard 87.5 73.7 Productivity index 78.1 57.9 Equipment downtime 65.6 27.8

86

Order entry productivity 65.5 52.9 Warehouse labour productivity 90.6 80.9 Transportation labour productivity 64.5 73.7





(Column 2)


Navy (Col 4)

Army (Col 5)

Quality Damage frequency 87.5 75 Note 8Order entry accuracy 80 45 Note 8Picking/shipping accuracy 90.6 75 Document/invoicing accuracy 84.4 35 Note 8Information availability 54.8 50 Information accuracy (Note 9) 45.2 33.3 Number of customer returns 96.8 88.9

Note 1: Identification of obsolete inventory at unit or inventory manager level conducted as one-off exercises rather than on-going performance indicator

Note 2: The Services, particularly Air Force started to profile inventory in late 1996. An instruction regarding A,B,C analysis issued in Army Log Comd. in February 1997.

Note 3: Measured as expenditure against Freight and Cartage budgets within Cash Limited Administrative Expenditure budgets; few examples of cost analysis

Note 4: In general, relates to monitoring of expenditure against approved budgets and forecast phasings, rather than analysis of costs.

87

Note 5: Navy & Air Force measure time taken to satisfy urgent demands affecting ship or aircraft capability, but do not measure the normal procurement cycle.

Note 6: Measured as % of demands delivered within AUSMIMPS priority time frames

Note 7: Proposed measure for Dispatch and Deliver function, but not yet implemented

Note 8: Proposed measures for inclusion in Service Level Agreement relating to CSP contract at one Army logistic unit

Note 9: Relates to stocktaking and stores adjustment reports

Appendix 4 Performance audits in the Department of Defence

Set out below are the titles of the ANAO’s main performance audit reports on the Department of Defence tabled in the Parliament in recent years.

Audit Report No. 22 1992-93 Audit Report No.31 1994-95 New Submarine Project Defence Contracting Audit Report No.5 1993-94 Audit Report No.8 1995-96 Explosive Ordnance Explosive Ordnance (Follow-up Audit) Audit Report No.11 1993-94 Audit Report No.11 1995-96 ANZAC Ship Project - Monitoring and Contracting Management Audit Audit Report No.19 1993-94 Audit Report No.17 1995-96 Defence Computer Environment Management of ADF Preparedness Supply Systems Redevelopment Project (Preliminary Study)

88

Audit Report No.27 1993-94 Audit Report No.26 1995-96 Report on Ministerial Portfolios, includes Defence Export Facilitation and Controls US Foreign Military Sales Program (Follow-up audit) Audit Report No.28 1995-96 Explosives Factory Maribyrnong Jindalee Operational Radar Network Project (JORN Project) Audit Report No.2 1994-95 Management of Army Training Areas Audit Report No.15 1996-97 (Follow-up audit) Management of Food Provisioning in the Acquisition of Additional F-111 Aircraft Australian Defence Force Audit Report No.13 1994-95 Audit Report No.17 1996-97 Australian Defence Force Housing Assistance Workforce Planning in the Australian Defence Force Audit Report No.25 1994-95 Australian Defence Force Living-in Audit Report No.27 1996-97 Accommodation Army Presence in the North Audit Report No.29 1994-95 Audit Report No.34 1996-97 Energy Management in Defence Australian Defence Force Health Services. ANZAC Ship Project Contract Amendments Overseas Visits by Defence Officers Audit Report No.xx 1997-98 Performance Management of Defence Inventory Defence Quality Assurance Organisation Preliminary Study

90

Series Titles

Titles published in the financial year 1997-98

[To be inserted by Reports and Publications Section]

performance management of defence inventory

Documents