Background to Reform

In August 2003 Mr Malcom Kinnaird, AO, reported to the Secretary of PM&C that significant reform was necessary to the processes by which defence capabilities were assessed, acquired and maintained.¹ His recommendations were contained in The Defence Procurement Review 2003 (the Kinnaird Review).

The review stated that ‘continuing delays in the delivery of major defence equipment mean that the ADF has failed to receive the capabilities it expects, according to the schedule required by Government.’ Kinnaird stated:

…that fundamental reform was necessary but there was no single remedy. As the body responsible for the management of major projects, the Defence Materiel Organisation (DMO) needs to become more business like and outcome driven. But reform must extend beyond the DMO. It is clear that change is needed at each stage of the cycle of acquisition and whole of life management of the equipment that comprises the core of defence capability.²

Defence acted quickly to implement the Kinnaird recommendations. A new Capability Development Group (CDG) was formed by amalgamating previously dispersed Defence Capability elements. The DMO re-structured and re-organised into three major areas to enhance its operation: domains, program managers, and operations. Further, the CEO DMO, restructured the reform process to reflect six key themes to ‘drive change and ensure the DMO achieves its vision of becoming the premier program management and engineering services organisation in Australia.’³ Within these themes a number of initiatives are being developed. These key themes are:

• Professionalise our workforce
• Reprioritise work so that effort is concentrated on the high-priority activities
• Standardise systems and work practices to ensure staff work efficiently and effectively
• Benchmark the DMO against the best Australian and international organisations of similar scale and scope
• Improve industry relations so they are more mature, share risks and avoid duplication of costs
• Lead reform in Defence by developing and implementing successful reforms in the DMO, which may facilitate change in Defence. ⁴

Prescribed Agency Status

Another milestone in the implementation of the Kinnaird Review recommendations was reached on 1 July 2005 when the DMO was accorded prescribed agency status. Prescription effectively de-mergered the DMO and Defence with the DMO becoming directly accountable to the Minister under the Financial Management and Accountability Act 1997, for the efficient, effective and ethical use of Commonwealth resources within the DMO. It is important to note that the DMO was not being created as a separate executive agency, ‘but will remain an integral part of the Defence Portfolio.’ ⁵

During the public hearing, the CEO of the DMO, Dr Stephen Gumley, reported on the according of Prescribed Agency Status by noting that although the 1 July 2005 date had been ambitious in terms of the supporting tasks required to be completed to ensure prescription, the DMO and Defence nonetheless believed ‘it was important to get on with the change agenda’. In an introductory assessment of this achievement, Dr Gumley said:

I think the prescribed agency status is the correct one for DMO at this stage of its development, and already we are seeing the benefits flow through to capability in that the organisation is lifting its productivity and becoming more efficient.⁶

The Committee requested information on the practical effect of Prescribed Agency Status and the de-merger of the DMO and Defence. Essentially, they sought to understand the degree of separation and how this would impact the management of the relationship between both entities, particularly as Defence had noted that prescription gave CEO DMO

the necessary independence to manage his organisation and control resources and staffing to deliver outputs ... [including] setting the DMO’s own financial management policy …⁷

CEO DMO noted that the relationship was now a full ‘customer-supplier’ relationship ensuring clear accountability and responsibility. This ‘customer-supplier’ relationship is underpinned by comprehensive agency agreements to cover both the acquisition of Defence equipment and their sustainment in-service, as well as activities normally expected of Defence in support of the organisation. These arrangements are the Materiel Acquisition Agreements and Materiel Sustainment Agreements.

Materiel Acquisition Agreements cover each capability project and each agreement specifies the project in terms of the scope to be delivered, the schedule for the delivery and the budget that is available.⁸

Materiel Sustainment Agreements formalise the DMO’s sustainment services to Defence and the price the DMO receives for these services. These agreements are based around the DMO sustainment products which have been designed around the key platforms, systems or fleets supported by the DMO.⁹

While this ‘customer-supplier’ relationship was considered effective for the major projects within the DMO, it was still to be implemented for the minor projects. Dr Gumley noted that getting the 100 or so minors within this structure was a key goal.¹⁰

One of the more complicated aspects of the prescription process was the separation of the financial statements of the DMO from Defence. This process necessitated, among other things, setting up a second chart of accounts within the Defence corporate information and financial management systems, a separate direct appropriation of funds to provide for policy advice and management service, and the transfer of civilian and military staff positions.¹¹ Indeed, a key feature of the Defence Portfolio Budget Statements 2005-06 was the separation of the DMO from the rest of Defence.

Project Management and Reporting

The Defence Annual Report 2004-05 provided details of approved major capital equipment projects ie those approved capital equipment programs costing generally in excess of $20m and which, following approval, are transferred from the Defence Capability Plan to the DMO for their acquisition. The top 30 major capital equipment projects were detailed and ranked on the basis of approved project cost, cumulative expenditure to 30 June 2005 and actual expenditure for 2004-05. ¹²

The Committee observed that the traditional practice of reporting projects by value alone presents only one means by which to analyse project management achievements. A further, and significant, other component that must be considered relates to risk. The Committee therefore sought further information on what methodologies had been developed for use by the DMO to assess the risk of a project beyond the dollar cost.

CEO DMO advised the Committee that the DMO reports to the Defence Committee on a monthly basis, generally updating on the top 60 or so projects defined by cost. However, he noted that in addition to this traditional method of reporting, he further reports on those projects which are regarded as being of ‘high interest to government’ or ‘having an unusual risk profile’¹³

The methodology by which DMO assesses the risk profiles for individual projects involves the development of, and assessment against, a ‘maturity score’. These maturity scores comprise several assessment criteria which are evaluated and scored to combine to produce a total out of 70. As CEO DMO noted during his testimony:

The job of every project manager is to deliver the project gradually and manage risks intensively until, at the end of the day, you deliver the capability to the war fighters with a score of 69 or 70 out of 70.¹⁴

Maturity Scores

The matter of managing DMO project risk by the use of the maturity score process was further pursued by the Committee. In order to facilitate an understanding of the process, within the bounds of maintaining commercial confidentiality in relation to the actual details of the process, CEO DMO broadly outlined the key stages and assessment criteria.

Essentially, a range of project attributes are assessed and scored, with 70 being the highest combined total achieveable. This total score enables DMO project managers to logically work their way through a project, assessing the risk at each stage. There will inevitably be a number of key stages and milestones for each project, however three in particular stand out: project commencement, first-pass approval and second-pass approval.

At project commencement CEO DMO advised the Committee that a maturity score of 10 or 15 out of 70 would not be unusual as there is generally a high risk associated with this early stage. The job of the DMO project managers is to de-risk the project to the point where a maturity score of 21 is attained to enable presentation to Government for first-pass approval¹⁵. This maturity score of 21 is considered the benchmark for first-pass approval. ¹⁶

Between first-pass and second-pass approval¹⁷ the Government funds DMO to further de-risk projects and to look at alternatives. A maturity score of around 35 out of 70 is necessary before a project can be presented to Government for second-pass approval.¹⁸

The development of a project is measured every month and the movement of maturity scores is monitored and evaluated as part of a constant and deliberate risk management process. CEO DMO advised that on a month to month basis, maturity scores do change and that backwards movement sometimes occurred. Such a backwards movement could be the result of an unexpected technical problem or a scheduling difficulty if equipment did not turn up. The regular measurement, monitoring and reporting of project progress enables tracking to occur so that once a score in the high 60s is reached, the project can be handed to the war fighter¹⁹.

The Committee observed that the maturity score process, and the actual final score out of 70, provided a sound and simple method by which to report the assessed level of risk for any given project and therefore facilitate a better understanding of the complex issue of project development in terms of risk management. The traditional ranking of Defence projects by dollar value alone does not provide such a clear and concise understanding of risk, progress or relativities across projects.

Indeed, the maturity score for each project was considered by the Committee to provide such an effective snapshot of project status and the progress over time of risk mitigation strategies that it was deemed highly desirable for inclusion in future Defence and DMO capital project reports.

CEO Defence Materiel Organisation commented that a regular report-back to the Committee on the status of project risk using the maturity score methodology was achieveable, he nonetheless noted that he would also desire to extend this reporting to include successful projects.²⁰

Selected Project Updates

The discussion in relation to the DMO assessment of project risk, and subsequent risk management by the application of the maturity score methodology, provided the foundation from which issues in relation to several key projects were pursued by the Committee. Of the capital projects reported in the Defence Annual Report 2004-05, the following were particularly raised by the Committee for closer examination.

• FFG Upgrade Implementation (SEA 1390 Ph 2). This project will improve the anti-ship missile defence and air surveillance capabilities of the ships to ensure they remain effective and supportable to the end of their life. The project was behind schedule at the time of the public hearing. CEO Defence Materiel Organisation advised the Committee that scheduling delays, such as have occurred with SEA 1390 Ph 2, are generally difficult to recover from, consequently, while no more time was lost in the reporting year, the scheduling gap remained.²¹
• Upgrade of M113 Armoured Vehicles (LAND 106). The project is upgrading 350 of the Army’s fleet which provide transport and fire support for the Army’s mechanised infantry forces to improve protection, firepower, mobility and habitability. The Committee was advised that scheduling difficulties that were being experienced in 2005 have been addressed and the project is again on schedule as a result of the contractor successfully remediating the outstanding technical problems. Any delays in the project are in the integrated logistic support package and are linked to the contractor having difficulty resourcing the necessary skilled people to undertake that work. Nonetheless, CEO Defence Materiel Organisation advised the Committee that the current forecast was for an in-service date of December 2006.²²
• Bushranger Infantry Mobility Vehicles (LAND 116). This project is acquiring 299 infantry mobility vehicles consisting of six variants: troop, command, assault pioneer, mortar, direct fire weapon and ambulance. The vehicles will provide two motorised infantry battalion groups to the Army and 12 troop variants to the Air Force Airfield Defence Guards. This particular project had been considered to be problematic, but has been effectively turned around. Indeed, CEO Defence Materiel Organisation noted the following:
• ADI did a fantastic job on that one in the first half of 2005. We were able to get them [Bushmasters] over to the overseas operations, they are highly regarded by the troops in the field and it has been a very big success story.²³
• Jindalee Operational Radar Network (JP 2025 Ph 3/4). The remaining work on this project relates to maintenance and support activities and engineering services that provide both facility sustainment and higher-end technical upkeep of the network’s assets. From a project management perspective, this particular project is considered very successful. Capability continues to meet contracted rate of effort and exceeds availability targets. CEO Defence Materiel Organisation specifically advised the Committee that ‘the contractor did a good job on this one.’²⁴
• Collins Replacement Combat System (SEA 1439 Ph 4A). This project will provide a replacement combat system for the Collins-class submarines. The systems integration in Australia is progressing to schedule with the project on target for the first installation, into the first submarine, Waller, in November 2006. CEO Defence Materiel Organisation advised the Committee that the upgrade to the entire fleet by the end of the decade was ‘entirely possible’. However, he advised that such an upgrade would be scheduled to occur during the normal maintenance/docking cycle of the submarine and that these docking cycles can be impacted by operational commitments.²⁵
• New Air Combat Capability (AIR 6000 SDD). This project aims to introduce a new air combat capability with the air dominance and strike functions currently provided by the F/A-18 Hornet and F-111 aircraft fleets. The Committee put a number of matters to Defence in relation to the Joint Strike Fighter and associated upgrade and expansion plans to support Australia’s air combat capability into the future. The growth of regional military capabilities and the implications for Australia’s future capability planning, especially with regard to retaining air power superiority was also considered. The Committee pursued this matter in more detail at separate public hearings in relation to Australia’s regional air superiority. Further information with regard to this inquiry can be obtained from the Committee’s website.²⁶

Conclusion

The Committee noted the Defence Materiel Organisation’s progress with regard to the reforms recommended by Kinnaird. In particular, the achievement of Prescribed Agency Status on 1 July 2005 was a key milestone in this reform process. The financial independence and accountability that prescription accorded the Defence Materiel Organisation should see further progress in the efficient delivery of equipment acquisition and fleet sustainment. The Committee recognised that the process of the de-merger of the broader Australian Defence Organisation and the Defence Materiel Organisation has been a complex and demanding exercise. The achievements to date are acknowledged and reflect the commitment of the Defence Materiel Organisation and Defence to achieving ongoing reform.

A vital part of the ongoing reform process is to mitigate, to the extent possible, the risks associated with any particular project. The process by which the Defence Materiel Organisation evaluates, monitors and reports on project development and risk management is the ‘maturity score’ methodology. The discussion and detail provided in relation to maturity scores was of such interest and utility to the Committee that it was considered worthy of regular reporting.

Progress in relation to the specific major acquisition projects that were explored by the Committee was mixed, however CEO Defence Materiel Organisation believes he has a firm grasp on where the problems lie and is working with his project management staff, contractors and Australian industry to address these issues.

Department of Defence, Defence Annual Report 2004-05, p. 263 Back

Department of Defence, Defence Annual Report 2004-05, p. 263 Back

Department of Defence, Defence Portfolio Budget Statements 2004-05, p. 242 Back

Department of Defence, Portfolio Budget Statements 2005-06, p 254 Back

Department of Defence, Portfolio Budget Statements 2005-06, p 270 Back