House of Representatives Committees

Chapter 2 Defence Materiel Organisation

Background to Reform
Prescribed Agency Status
Project Management and Reporting
Maturity Scores
Selected Project Updates
Conclusion

Background to Reform

2.1

In August 2003 Mr Malcom Kinnaird, AO, reported to the Secretary of PM&C that significant reform was necessary to the processes by which defence capabilities were assessed, acquired and maintained.1 His recommendations were contained in The Defence Procurement Review 2003 (the Kinnaird Review).

2.2

The review stated that ‘continuing delays in the delivery of major defence equipment mean that the ADF has failed to receive the capabilities it expects, according to the schedule required by Government.’ Kinnaird stated:

…that fundamental reform was necessary but there was no single remedy. As the body responsible for the management of major projects, the Defence Materiel Organisation (DMO) needs to become more business like and outcome driven. But reform must extend beyond the DMO. It is clear that change is needed at each stage of the cycle of acquisition and whole of life management of the equipment that comprises the core of defence capability.2

2.3

Defence acted quickly to implement the Kinnaird recommendations. A new Capability Development Group (CDG) was formed by amalgamating previously dispersed Defence Capability elements. The DMO re-structured and re-organised into three major areas to enhance its operation: domains, program managers, and operations. Further, the CEO DMO, restructured the reform process to reflect six key themes to ‘drive change and ensure the DMO achieves its vision of becoming the premier program management and engineering services organisation in Australia.’3 Within these themes a number of initiatives are being developed. These key themes are:

 

Prescribed Agency Status

2.4

Another milestone in the implementation of the Kinnaird Review recommendations was reached on 1 July 2005 when the DMO was accorded prescribed agency status. Prescription effectively de-mergered the DMO and Defence with the DMO becoming directly accountable to the Minister under the Financial Management and Accountability Act 1997, for the efficient, effective and ethical use of Commonwealth resources within the DMO. It is important to note that the DMO was not being created as a separate executive agency, ‘but will remain an integral part of the Defence Portfolio.’ 5

2.5

During the public hearing, the CEO of the DMO, Dr Stephen Gumley, reported on the according of Prescribed Agency Status by noting that although the 1 July 2005 date had been ambitious in terms of the supporting tasks required to be completed to ensure prescription, the DMO and Defence nonetheless believed ‘it was important to get on with the change agenda’. In an introductory assessment of this achievement, Dr Gumley said:

I think the prescribed agency status is the correct one for DMO at this stage of its development, and already we are seeing the benefits flow through to capability in that the organisation is lifting its productivity and becoming more efficient.6

2.6

The Committee requested information on the practical effect of Prescribed Agency Status and the de-merger of the DMO and Defence. Essentially, they sought to understand the degree of separation and how this would impact the management of the relationship between both entities, particularly as Defence had noted that prescription gave CEO DMO

the necessary independence to manage his organisation and control resources and staffing to deliver outputs ... [including] setting the DMO’s own financial management policy …7

2.7

CEO DMO noted that the relationship was now a full ‘customer-supplier’ relationship ensuring clear accountability and responsibility. This ‘customer-supplier’ relationship is underpinned by comprehensive agency agreements to cover both the acquisition of Defence equipment and their sustainment in-service, as well as activities normally expected of Defence in support of the organisation. These arrangements are the Materiel Acquisition Agreements and Materiel Sustainment Agreements.

Materiel Acquisition Agreements cover each capability project and each agreement specifies the project in terms of the scope to be delivered, the schedule for the delivery and the budget that is available.8

Materiel Sustainment Agreements formalise the DMO’s sustainment services to Defence and the price the DMO receives for these services. These agreements are based around the DMO sustainment products which have been designed around the key platforms, systems or fleets supported by the DMO.9

2.8

While this ‘customer-supplier’ relationship was considered effective for the major projects within the DMO, it was still to be implemented for the minor projects. Dr Gumley noted that getting the 100 or so minors within this structure was a key goal.10

2.9

One of the more complicated aspects of the prescription process was the separation of the financial statements of the DMO from Defence. This process necessitated, among other things, setting up a second chart of accounts within the Defence corporate information and financial management systems, a separate direct appropriation of funds to provide for policy advice and management service, and the transfer of civilian and military staff positions.11 Indeed, a key feature of the Defence Portfolio Budget Statements 2005-06 was the separation of the DMO from the rest of Defence.

 

Project Management and Reporting

2.10

The Defence Annual Report 2004-05 provided details of approved major capital equipment projects ie those approved capital equipment programs costing generally in excess of $20m and which, following approval, are transferred from the Defence Capability Plan to the DMO for their acquisition. The top 30 major capital equipment projects were detailed and ranked on the basis of approved project cost, cumulative expenditure to 30 June 2005 and actual expenditure for 2004-05. 12

2.11

The Committee observed that the traditional practice of reporting projects by value alone presents only one means by which to analyse project management achievements. A further, and significant, other component that must be considered relates to risk. The Committee therefore sought further information on what methodologies had been developed for use by the DMO to assess the risk of a project beyond the dollar cost.

2.12

CEO DMO advised the Committee that the DMO reports to the Defence Committee on a monthly basis, generally updating on the top 60 or so projects defined by cost. However, he noted that in addition to this traditional method of reporting, he further reports on those projects which are regarded as being of ‘high interest to government’ or ‘having an unusual risk profile’13

2.13

The methodology by which DMO assesses the risk profiles for individual projects involves the development of, and assessment against, a ‘maturity score’. These maturity scores comprise several assessment criteria which are evaluated and scored to combine to produce a total out of 70. As CEO DMO noted during his testimony:

The job of every project manager is to deliver the project gradually and manage risks intensively until, at the end of the day, you deliver the capability to the war fighters with a score of 69 or 70 out of 70.14

 

Maturity Scores

2.14

The matter of managing DMO project risk by the use of the maturity score process was further pursued by the Committee. In order to facilitate an understanding of the process, within the bounds of maintaining commercial confidentiality in relation to the actual details of the process, CEO DMO broadly outlined the key stages and assessment criteria.

2.15

Essentially, a range of project attributes are assessed and scored, with 70 being the highest combined total achieveable. This total score enables DMO project managers to logically work their way through a project, assessing the risk at each stage. There will inevitably be a number of key stages and milestones for each project, however three in particular stand out: project commencement, first-pass approval and second-pass approval.

2.16

At project commencement CEO DMO advised the Committee that a maturity score of 10 or 15 out of 70 would not be unusual as there is generally a high risk associated with this early stage. The job of the DMO project managers is to de-risk the project to the point where a maturity score of 21 is attained to enable presentation to Government for first-pass approval15. This maturity score of 21 is considered the benchmark for first-pass approval. 16

2.17

Between first-pass and second-pass approval17 the Government funds DMO to further de-risk projects and to look at alternatives. A maturity score of around 35 out of 70 is necessary before a project can be presented to Government for second-pass approval.18

2.18

The development of a project is measured every month and the movement of maturity scores is monitored and evaluated as part of a constant and deliberate risk management process. CEO DMO advised that on a month to month basis, maturity scores do change and that backwards movement sometimes occurred. Such a backwards movement could be the result of an unexpected technical problem or a scheduling difficulty if equipment did not turn up. The regular measurement, monitoring and reporting of project progress enables tracking to occur so that once a score in the high 60s is reached, the project can be handed to the war fighter19.

2.19

The Committee observed that the maturity score process, and the actual final score out of 70, provided a sound and simple method by which to report the assessed level of risk for any given project and therefore facilitate a better understanding of the complex issue of project development in terms of risk management. The traditional ranking of Defence projects by dollar value alone does not provide such a clear and concise understanding of risk, progress or relativities across projects.

2.20

Indeed, the maturity score for each project was considered by the Committee to provide such an effective snapshot of project status and the progress over time of risk mitigation strategies that it was deemed highly desirable for inclusion in future Defence and DMO capital project reports.

2.21

CEO Defence Materiel Organisation commented that a regular report-back to the Committee on the status of project risk using the maturity score methodology was achieveable, he nonetheless noted that he would also desire to extend this reporting to include successful projects.20

 

Recommendation 1

The Committee recommends that the Defence Materiel Organisation provide annual updates on the top ten high risk projects of the year using the Maturity Score methodology, noting that commercial-in-confidence imperatives will apply.

 

Selected Project Updates

2.22

The discussion in relation to the DMO assessment of project risk, and subsequent risk management by the application of the maturity score methodology, provided the foundation from which issues in relation to several key projects were pursued by the Committee. Of the capital projects reported in the Defence Annual Report 2004-05, the following were particularly raised by the Committee for closer examination.

 

Conclusion

2.23

The Committee noted the Defence Materiel Organisation’s progress with regard to the reforms recommended by Kinnaird. In particular, the achievement of Prescribed Agency Status on 1 July 2005 was a key milestone in this reform process. The financial independence and accountability that prescription accorded the Defence Materiel Organisation should see further progress in the efficient delivery of equipment acquisition and fleet sustainment. The Committee recognised that the process of the de-merger of the broader Australian Defence Organisation and the Defence Materiel Organisation has been a complex and demanding exercise. The achievements to date are acknowledged and reflect the commitment of the Defence Materiel Organisation and Defence to achieving ongoing reform.

2.24

A vital part of the ongoing reform process is to mitigate, to the extent possible, the risks associated with any particular project. The process by which the Defence Materiel Organisation evaluates, monitors and reports on project development and risk management is the ‘maturity score’ methodology. The discussion and detail provided in relation to maturity scores was of such interest and utility to the Committee that it was considered worthy of regular reporting.

2.25

Progress in relation to the specific major acquisition projects that were explored by the Committee was mixed, however CEO Defence Materiel Organisation believes he has a firm grasp on where the problems lie and is working with his project management staff, contractors and Australian industry to address these issues.



Footnotes

1 Department of Prime Minister and Cabinet, Defence Procurement Review 2003, 15 August 2003 , p. iii Back
2 Department of Prime Minister and Cabinet, Defence Procurement Review 2003, 15 August 2003 , p. iii Back
3

Department of Defence, Defence Annual Report 2004-05, p. 263 Back

4

Department of Defence, Defence Annual Report 2004-05, p. 263 Back

5

Department of Defence, Defence Portfolio Budget Statements 2004-05, p. 242 Back

6 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript p. 2 Back
7 Department of Defence, Defence Annual Report 2004-05, p.260 Back
8

Department of Defence, Portfolio Budget Statements 2005-06, p 254 Back

9

Department of Defence, Portfolio Budget Statements 2005-06, p 270 Back

10 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript p. 4 Back
11 Department of Defence, Portfolio Budget Statements 2005-06, p. 4 Back
12 Department of Defence, Defence Annual Report 2004-05, pp 267-283 Back
13 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript, p. 2 Back
14 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript pp 2-3 Back
15 First-pass approval refers to the process whereby Defence gives Government the opportunity to narrow the alternatives being examined by Defence to meet an agreed capability gap. First-pass approval allows a project to be included in the Defence Capability Plan and the Major Capital Investment Program. (Source: Defence Capability Development Manual 2005, p. 69) Back
16 Dr Stephen Gumley, CEO Defence Materiel Organisation, Department of Defence, Transcript p 2. Back
17 Second-pass approval refers to a key milestone in project management which requires more detailed, rigorous costing and assessment of each option submitted for consideration by Higher Defence Committees and Government than for first-pass. The project cannot proceed until this approval is obtained from Government, but it does not provide authority to spend public moneys. (Source: Defence Capability Development Manual 2005, p. 72) Back
18 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript, p. 2. Back
19 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript, p. 3. Back
20 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript, p. 10. Back
21 Dr Stephen Gumley, CEO Defence Materiel Organisation, Department of Defence, Transcript p.8. Back
22 Dr Stephen Gumley , CEO Defence Materiel Organisation, Department of Defence, Transcript pp 14-15. Back
23 Dr Stephen Gumley, CEO Defence Materiel Organisation, Department of Defence, Transcript p. 10. Back
24 Dr Stephen Gumley, CEO Defence Materiel Organisation, Department of Defence, Transcript p. 12. Back
25 Dr Stephen Gumley, CEO Defence Materiel Organisation, Department of Defence, Transcript p. 14. Back
26 http://www.aph.gov.au/committee/jfadt/index.htm

Print Chapter 2 (PDF 137KB) < - Report Home < - Chapter 1  : Chapter 3 - >

Back to top

We acknowledge the traditional owners and custodians of country throughout Australia and acknowledge their continuing connection to land, waters and community. We pay our respects to the people, the cultures and the elders past, present and emerging.

Aboriginal and Torres Strait Islander people are advised that this website may contain images and voices of deceased people.