Chapter 2 Major Projects Report 2011–12

Introduction

2.1                   This chapter provides an overview of the 2011–12 Major Projects Report (MPR) and a summary of the Australian National Audit Office (ANAO)’s findings in regard to cost performance; schedule performance; capability performance; and governance and business processes.

2.2                   The objective of the MPR is to provide:

• comprehensive information on the status of selected Major Projects, as reflected in Project Data Summary Sheets (PDSSs) prepared by the Defence Materiel Organisation (DMO);
  
  
• the Auditor-General’s formal review conclusion on the ANAO’s review of the preparation of the PDSSs by the DMO;
  
  
• ANAO analysis on the three key elements of the PDSSs—cost, schedule and capability—and in particular, longitudinal analysis across projects over time; and
  
  
• further insights and context by the DMO on issues highlighted during the year (not included within the scope of the review by the ANAO).[1]
  
  

2.3                   The MPR consists of three parts:

• Part 1: ANAO overview;
  
  
• Part 2: Commentary and overall analysis by the DMO; and
  
  
• Part 3: Auditor-General’s independent review report; statement by the Chief Executive Officer DMO; and the PDSSs.
  
  

2.4                   In appendices, the report also includes a copy of the endorsed guidance for the development of PDSSs (part of the MPR Guidelines); a copy of the DMO’s response to the Committee’s recommendations on its review of the 2010–11 report; and the ‘Lessons Learned’ for a project that was reported on for the last time in 2010–11 MPR (AIR 5376 Phase 3.2 – F/A-18 Hornet Upgrade Structural Refurbishment).

2.5                   The PDSSs prepared by the DMO have been refined over the years since the first MPR was developed in 2007–08. These changes have been made by the DMO and ANAO with the support of the Joint Committee of Public Accounts and Audit (JCPAA).

2.6                   The main changes to the PDSSs for the 2011–12 MPR were the addition of a project financial assurance statement in regard to cost performance; the removal of base date dollars from project budget and expenditure history, consistent with the move to reporting financial information in an ‘out-turned’ format; and the addition of a graph depicting project cost and schedule status.[2]

2.7                   In their current form, the PDSSs provide data in the following sections:

• Section 1 – project summary;
  
  
• Section 2 – financial performance;
  
  
• Section 3 – schedule performance;
  
  
• Section 4 – project cost and schedule status;
  
  
• Section 5 – materiel capability performance;
  
  
• Section 6 – major risks and issues;
  
  
• Section 7 – project maturity;
  
  
• Section 8 – lessons learned; and
  
  
• Section 9 – project line management.[3]

Major Projects included in 2011–12

2.8                   The 2011–12 MPR reports on 29 major projects, an increase of one project on the 2010–11 MPR. The total approved budget for the 29 projects, as at 30 June 2012, was $47.3 billion. The projects and their approved budgets appear in Table 2.1 below.

2.9                   The inclusion of projects in the MPR was based on the following criteria:

• projects only admitted one year after Year of Decision;
  
  
• a total approved project budget of more than $150 million;
  
  
• a project should have at least three years of asset delivery remaining;
  
  
• a project must have at least $50 million or 10 per cent (whichever is greater) of its budget remaining over the next two years; and
  
  
• a maximum of eight new projects in any one year.[4]
  
  

Table 2.1 2011–12 MPR projects and approved budgets at 30 June 2012

Source Australian National Audit Office, 2011–12 Major Projects Report, p. 15.

2.10               Twenty-seven of the projects were previously reported in the 2010-11 MPR (i.e. ‘repeat’ projects). The following two projects have been added:

• AIR 9000 Phase 8 – Future Naval Combat System (MH-60R Seahawk); and
  
  
• LAND 19 Phase 7A – Counter – Rocket, Artillery and Mortar (C-RAM).
  
  

2.11               The 2011–12 MPR Guidelines stipulated that projects which have achieved both Final Materiel Release (FMR) and Final Operational Capability (FOC) would be expected to be removed from future MPRs. One project was ‘exited’ from the 2011–12 after meeting this criteria:

• AIR 5376 Phase 3.2 – F/A-18 Hornet Upgrade Structural Refurbishment (Hornet Refurb).

Australian National Audit Office review

2.12               Although the ANAO conducts an assurance audit of the MPR, it cautions that the level of assurance is more limited than for an individual project performance audit. The ANAO does not provide any assurance in regards to PDSS data on the achievement of future dates or events, project financial assurance statements or major risks and issues. These items were excluded from the scope of the ANAO’s review.[5]

2.13               During its review of the 2010–11 MPR, the Committee identified the MPR as a ‘Priority Assurance Review’ under section 19A(5) of the Auditor-General Act 1997.[6] This designation has allowed the ANAO full access to the information gathering powers available under the Act, without necessitating the agreement of the DMO to perform its review.[7]

2.14               After reviewing the PDSS data, the ANAO’s conclusion was that:

... nothing has come to the attention of the ANAO that causes us to believe that the information in the PDSSs, within the scope of our review, has not been prepared, in all material respects, in accordance with the 2011–12 MPR Guidelines.[8]

2.15               The ANAO provided more detailed analysis on the following aspects of the MPR:

• Project cost, schedule and capability performance
  
  
• Governance and business processes.

Cost performance

2.16               In relation to cost, the ANAO concluded that within the review period, all projects continued to operate within their approved budget.[9]

2.17               At 30 June 2012, the total approved budgeted costs for the 29 projects was $47.3 billion, a net increase of $5.9 billion compared to their Second Pass Approval approved budgeted costs ($41.4 billion). The $5.9 billion comprised:

• price indexation increases of $7.5 billion;
  
  
• real variation increases of $2.7 billion; and
  
  
• foreign exchange rate decreases of $4.3 billion.[10]
  
  

2.18               While variations due to price indexation and exchange rates were ‘outside the direct control of project management’, real variations to budgeted costs:

… primarily reflect changes in the scope of projects, transfers between projects for approved equipment/capability, and budgetary adjustments such as administrative savings decisions.[11]

2.19               During the 2011–12 financial year, the approved budgeted cost of the 29 Major Projects decreased by $1.1 billion (or 2.4 per cent) due to foreign exchange decreases of $894.6 million and real decreases of $267.3 million.[12]

Schedule performance

2.20               According to the ANAO’s review, maintaining major projects on schedule remained the ‘most significant challenge for the DMO and its industry contractors’.[13]

2.21               Across the 29 major projects, 18 projects had experienced schedule slippage. Total schedule slippage to date was 859 months when compared to the initial predictions when the projects were first approved by government. This represented a 32 per cent increase on the expected schedule since the main investment decision was made (compared to 31 per cent in the 2010–11 MPR).[14]

2.22               In-year slippage for 2011–12 was a total of 99 months for the 27 projects that were also included in the 2010–11 MPR, representing a four per cent increase in the scheduled timeframe.[15]

2.23               According to the ANAO, ‘the reasons for schedule slippage vary but primarily reflect the underestimation of both the scope and complexity of work, particularly for Australianised Military Off-the-Shelf (AMOTS) and Developmental projects’.[16]

2.24               The ANAO noted that 87 per cent of the total schedule slippage across the Major Projects covered in the 2011–12 MPR was made up of projects approved prior to the DMO’s demerger from the Department of Defence in July 2005—projects which tended to be more developmental in nature.[17]

Capability performance

2.25               The ANAO noted that ‘… the DMO expects to deliver almost all capabilities associated with the Major Projects in this report’.

2.26               The capability of a project concerns its capacity or ability to achieve a particular operational effect.[18] Due to national security considerations, only the overall status from each project’s capability assessment is disclosed in the MPR.[19]

2.27               Expected capability delivery had decreased from 94 per cent for projects in the 2010–11 MPR to 91 per cent in 2011–12. Although outside the scope of its formal review conclusion, the ANAO indicated that the DMO’s assessment of capability to be delivered was ‘in some cases overly optimistic’.[20]

2.28               The ANAO added that

… the DMO’s key capability measures should be interpreted with some caution due to their lack of rigour as a data system and the high level of uncertainty in forecasting outcomes.[21]

and that there was

… not a clear underlying consistency in the identification and articulation of the [Materiel Release Milestones] and Completion Criteria’.[22]

Governance and business processes

2.29               The ANAO reviewed the following key governance aspects relating to major projects:

• Gate Reviews, in which DMO-appointed boards periodically assess projects at key milestones to provide assurance that the project is ready to progress to the next stage. Gate Reviews ‘have emerged as the DMO’s most prominent internal project assurance activity’.[23]
  
  
• The management of Projects of Concern, in which projects facing significant cost, schedule or capability delivery challenges are listed in order to focus the attention of senior management on remediating issues. Six MPR projects were Projects of Concern during 2011–12.[24]
  
  
• The Early Indicators and Warnings system, which is used to identify problems with projects early in order for appropriate action to take place to remediate them. In 2011–12, four MPR projects had Gate Reviews as a result of triggering Early Indicators and Warnings threshold criteria.[25]
  
  
• The use of Joint Project Directives (JPDs) for projects approved by government after 1 March 2010. JPDs are designed to provide a single authoritative source of project detail, and are the base document from which all Materiel Acquisition Agreements (MAAs) will be designed.[26]
  
  

2.30               Finally, the ANAO’s overview also made the following observations regarding the application of business systems and processes:

• The large range of corporate and project management applications being used by project offices continues to result in inconsistency between information produced by risk management, financial management and document management systems. This will be an area of continuing focus for the ANAO’s next review of the MPR.[27]
  
  
• The change in supplementation policy associated with out-turned budgeting has ‘emerged as a major risk or issue’ for some MPR projects, for which contingency funds may need to be drawn upon:
  
  
• The emergence of any indexation risk has, to some extent, changed the nature and use of the contingency budget from dealing with project risk management to broader price management …[28]