|
Parliamentary Joint Committee on Public Accounts and Audit
Navigation: Previous Page | Contents | Next Page
Chapter 2 Major Projects Report Work Program
Introduction
2.1
The Australian National Audit Office (ANAO) commented that the 2009-10
MPR includes ‘a greater level of information about each project’s performance’[1]
and progresses the data analysis undertaken on the 2008-09 Major Projects
Report (MPR). In this way, the 2009-10 MPR also ‘provides a basis for greater
longitudinal analysis on project performance in future years.’[2]
2.2
Issues surrounding cost and scheduling of major defence capital
acquisition projects (projects) is regularly reported in the public domain. In
this respect, the MPR also provides a level of public visibility of the issues
relevant to incorporated projects for initially approved budgets and schedules.
The Defence Materiel Organisation (DMO) acknowledged that schedule delay is an
area of improvement over the coming years.[3]
2.3
This chapter outlines the main elements of the MPR Work Plan and issues
raised in regard to the 2009-10 MPR and lists those projects proposed for incorporation
into the 2010-11 MPR. These include the criteria for selecting projects in
terms of their classification, and excluding projects in the 2010-11 MPR in
regard to the defined points of completion. The qualified audit opinion given
by the ANAO is also discussed.
Major Projects Report Work Plan
2.4
The MPR Work Plan has been provided to the committee annually (shortly
after the committee has held its public hearing for the review of the previous
year’s MPR) and forms part of the forward MPR Work Program. The committee has
an active role in considering and endorsing the MPR Work Plan including the
list of projects to be added to each MPR and the Guidelines for the Project
Data Summary Sheets (PDSS). The MPR Work Plan for 2009-10 included:
- The criteria for the
selection of major projects to be included in the MPR as well as a list of
projects selected for inclusion into the MPR.
- The roles and
responsibilities of the DMO in the production and review of the MPR.
- Guidelines for
producing the PDSS.
- The PDSS template.
- An indicative program
schedule for presentation of the MPR to the Parliament in November 2010.[4]
Guidelines for the Project Data Summary Sheets
Purpose of Guidelines
2.5
Major projects included in the MPR are reported on through individual
PDSS. These PDSS are collected and collated with an overview by the DMO,
together with the findings and audit opinion of the ANAO’s formal review.
2.6
The DMO in consultation with the ANAO has developed a set of Guidelines
to provide a framework for the production and provision of the PDSS and form a
basis for review. The Guidelines also require endorsement by the Joint
Committee of Public Accounts and Audit (JCPAA). A revised set of Guidelines is
usually made available to the committee for its consideration prior to drafting
of the upcoming MPR (in this case the 2010-11 MPR).
2.7
The PDSS contain information about each project contained in the MPR.
The basis of the ANAO’s review is to assess whether the information contained
in the PDSS meet the requirements included in the Guidelines.
Guidelines for the 2010-11 Major Projects Report
2.8
The DMO in conjunction with the ANAO has provided the committee with
revised Guidelines for the 2010-11 MPR. The changes refine the PDSS and are
designed to ‘provide more concise reporting and ... improve the flow of
information.’[5]
2.9
Proposed revisions to the Guidelines include:
- Project management
has been moved to Section 8 (previously section 1.1) and reduced to line
management within 2010-11 (e.g. no historical data).
- Project Maturity
Score and Benchmark is now Section 6.1 (previously section 1.6).
- Materiel Capability
Performance is now Section 4 (previously Section 3.5).
- Word limits have been
imposed to reduce the length of narrative.
- Duplication of
contract and other summary information has been removed.[6]
2.10
In its review of the 2007-08 MPR, the previous committee recommended
that no later than 31 August each year, the JCPAA be consulted on the projects
for inclusion and exclusion in the following year’s MPR.[7]
2.11
The committee received the DMO’s proposed list of projects and
accompanying project synopsis for inclusion in the 2010-11 at the start of the
43rd Parliament. The Guidelines for the 2010-11 MPR were included in
the DMO Work Plan which was forwarded to the committee in late March 2011.
Selection and exit criteria for projects
2.12
The 2009-10 MPR reported on 22 projects which is an increase of seven projects
from the previous MPR. The 2010-11 will include six additional projects to
bring the total number of projects reported on to 28.[8]
The MPR will eventually include summaries of a maximum of 30 projects.
2.13
The DMO stated that the maximum number of projects reported on in the
MPR has been determined taking into consideration the cost of reporting on
selected projects and the utility of this information to the committee. The DMO
explained:
It is a question of cost versus utility. Each of these
projects is very resource intensive to report in this way. There would have to
be a law of diminishing returns somewhere, and how far do you take it? We have
200 projects and it would be physically impossible to do all 200, so it really
does become a balance between cost and utility to the committee.[9]
2.14
The projects which are included in the MPR are subject to the following criteria:
- ‘projects [are] only
admitted one year after Year of Decision
- [projects must have]
a total approved project budget greater than $150 million
- a project should have
at least three years of asset delivery remaining
- a project must have
at least $50 million or 10 per cent (whichever is greater) of their budget
remaining over the next two years
- a maximum of eight new
projects in any one year
- all projects for
inclusion in the MPR will be proposed by the DMO in consultation with the ANAO,
based on the above criteria, and provided to the JCPAA by 31 August in the year
to which the MPR relates, for endorsement.’[10]
Major projects included in 2009-10
2.15
Taking into consideration the project inclusion criteria, seven new
projects were added to the 2009-10 MPR and joined the 15 repeat projects which
appeared in the 2008-09 MPR. These projects represented approximately $41 Billion
or just over half of DMO’s approved major capital investment program.[11]
These projects and their approved budgets appear in Table 2.1.
Table 2.1 2009-10 MPR Projects and their approved Budgets
at 30 June 2010
|
Project
|
DMO Abbreviation
|
Approved Budget $m
|
|
New projects included
in the MPR
|
|
|
|
Field vehicles and trailers
(LAND 121 Ph 3)
|
Overlander vehicles
|
2 879.2
|
|
Next Generation Satellite
Communications System (JP 2008 Ph 4)
|
Next Gen Satellite
|
894.1
|
|
Replacement Heavyweight
Torpedo (SEA 1429 Ph 2)
|
HW Torpedo
|
441.5
|
|
Follow-On Stand Off Weapon
(AIR 5418 Ph 1)
|
Stand Off Weapon
|
399.6
|
|
Anzac Anti-Ship Missile
Defence (SEA 1448 Ph 2A)
|
ANZAC ASMD 2A
|
377.1
|
|
Anzac Anti-Ship Missile
Defence (SEA 1448 Ph 2B)
|
ANZAC ASMD 2B
|
458.5
|
|
Collins Class Submarine
Reliability and Sustainability (SEA 1439 Ph 3)
|
Collins R&S
|
407.7
|
|
Repeat projects
updated since the 2008-09 MPR
|
|
|
|
Air Warfare Destroyer Build
(SEA 4000 Ph 3)
|
AWD Ships
|
7 740.1
|
|
Airborne Early Warning and
Control Aircraft (AIR 5077 Ph 3)
|
Wedgetail
|
3 883.5
|
|
Multi-Role Helicopter (AIR
9000 Phs 2/4/6)
|
MRH90 Helicopters
|
3 754.6
|
|
Bridging Air Combat
Capability (AIR 5349 Ph 1)
|
Super Hornet
|
3 629.1
|
|
Amphibious Deployment and
Support (JP 2048 Ph 4A/4B)
|
LHD Ships
|
3 160.8
|
|
Armed Reconnaissance
Helicopter (AIR 87 Ph 2)
|
ARH Tiger Helicopters
|
2 076.3
|
|
F/A-18 Hornet Upgrade (AIR
5376 Ph 2)
|
Hornet Upgrade
|
1 946.6
|
|
Air to Air Refuelling
Capability (AIR 5402)
|
Air to Air Refuel
|
1 889.4
|
|
C-17 Globemaster III Heavy
Airlifter (AIR 8000 Ph 3)
|
C-17 Heavy Airlift
|
1 834.6
|
|
Guided Missile Frigate
Upgrade Implementation (SEA 1390 Ph 2.1)
|
FFG Upgrade
|
1 529.6
|
|
F/A-18 Hornet Upgrade
Structural Refurbishment (AIR 5376 Ph 3.2)
|
Hornet Refurb
|
943.5
|
|
Bushmaster Protected
Mobility Vehicle (LAND 116 Ph 3)
|
Bushranger Vehicles
|
926.2
|
|
High Frequency Modernisation
(JP 2043 Ph 3A)
|
HF Modernisation
|
662.7
|
|
Armidale Class Patrol Boat
(SEA 1444 Ph 1)
|
Armidales
|
536.7
|
|
Collins Replacement Combat System
(SEA 1439 Ph 4A)
|
Collins RCS
|
458.0
|
|
TOTAL
|
|
40 829.4
|
Source Australian
National Audit Office, 2009-10 Major Projects Report, p. 14.
Major projects to be included in 2010-11
2.16
In addition to the 22 projects included in the 2009-10 MPR as listed in Table
2.1, the DMO has proposed that the 2010-11 MPR incorporate the following six
projects:
- New Air Combat
Capability – AIR 6000 Ph 2
- SM-1 Missile
Replacement – SEA 1390 Ph 4B
- Additional Chinook
Helicopter – AIR 9000 Ph 5C
- UHF SATCOM – JP 2008
Ph 5A
- Battle Management
System – LAND 75 Ph 3.4
- Artillery Replacement
155mm Howitzer – LAND 17 Ph 1A.[12]
Options for 2010-11 projects’ selection criteria
Projects of concern
2.17
On 26 November 2010, the Minister for Defence, the
Hon Stephen Smith MP and the Minister for Defence Materiel, the
Hon Jason Clare MP, jointly announced an updated list of those defence
capital acquisition projects which were considered to be Projects of Concern
(PoC).[13]
2.18
Projects are classified as PoC when they encounter ‘significant
challenges with scheduling, cost, capability delivery or project management.’[14]
Established in 2008 by the Minister for Defence, the PoC list focuses ‘the
attention of Defence and Industry senior management on remediating listed
projects.’[15]
2.19
Five of the 12 currently listed PoC are reported on in the 2009-10 MPR.[16]
The DMO stated that there is no direct link between the projects included in the
MPR and those that are listed as PoC.[17] Rather, the projects
included in the MPR are those that have met the requirements of the agreed
criteria (as outlined in paragraph 2.15).
2.20
The ANAO suggested that the committee could consider adding PoC to the
MPR. However, this would increase the cost of producing the MPR. In addition,
the ANAO stated there are issues surrounding the length of time a PoC would be
reported on in the MPR and the associated value of reporting on PoC’s that are
smaller projects in terms of their approved budget. The ANAO stated:
... we would suggest to the committee that you may consider
adding additional criteria ...specifically ... including any projects of
concern for the MPR. We are conscious that once they get in, there is the question
of how long they stay, and if they are not big spends but they are still a
worry, do we want to keep them in? It all adds to the cost of running this.[18]
2.21
The ANAO advised that the areas for consideration when assessing
possible inclusion of PoC into the MPR could include:
- ‘The impact on the
longitudinal analysis that is currently performed by both the DMO and the ANAO;
- Whether the project
is only reported for the length of time the project remains a PoC or for the
length of the project; and
- The potential number
of projects to be included in the MPR, should the PoC list exceed the 12
projects currently reported.’[19]
2.22
The DMO was concerned about adding PoC to the selection criteria for MPR
projects as PoC are determined on an ad hoc basis, whereas the projects
selected for inclusion into the MPR occurs at a particular point in the
calendar year. Another issue relates to the length of time a project remains a
PoC and if it enters the MPR based on its meeting the PoC criteria, when does
it exit the MPR if it is no longer classified a PoC.[20]
2.23
The DMO added that the PoC list is ‘subject to frequent reviews,
adjustment and Ministerial oversight, and is quite separate in its intent to
the MPR.’[21] Further, there could be
an adverse effect in adding PoC to the MPR in terms of trend analysis and
competition with reporting on other projects. The DMO explained:
To include PoC projects would have adverse impacts; in
particular it has the potential to compromise overall MPR trend analysis if PoC
projects are frequently moved on and off the MPR. It would also mean the
removal or deferment of other current/potential MPR projects (which represent
significant spend and capability) in order to not exceed the agreed cap of 30
projects.[22]
2.24
There was also the issue of whether a project should be included in the
MPR if it was classified in the pre-PoC stage of being a Project of Interest
(PoI). The DMO was not in favour of including PoIs in the MPR as such projects
did not yet require more management attention and could more readily be resolved.
The DMO stated:
We would probably suggest that not be one of the criteria. A
project of interest is a signal to Defence, to DMO and to industry that these
require a lot more management attention. They have not quite met the threshold
where you would declare them to be a very difficult project or a project of
concern. It is like an amber light when you come to the traffic lights—you have
just got to be cautious. I would prefer to keep that one as a management
prerogative and we put our efforts into it in the short term to try to get it
fixed.[23]
2.25
The ANAO further stated that even with the inclusion of specific PoC
reporting into the MPR, the focus of the MPR is different to that of what is
required to report specifically on PoC. The ANAO suggested that the committee could
request a list annually of those PoC which are not included in the MPR and seek
further information on these projects ‘on a shorter term basis than would be
the case for MPR projects.’ The ANAO also stated that ‘under this approach,
consideration would also need to be given ... to any possible commercial
sensitivity attached to such reporting.’[24]
2.26
Determining exit criteria that could be used for removing projects from
the MPR was also raised in relation to ascertaining when a project is completed.
Exit criteria
2.27
As a number of projects will be completed in the upcoming MPRs, the
issue of how long a project should continue to be reported on in the MPR and
when it should be removed from the MPR was raised.
2.28
In accordance with the 2010-11 MPR selection of projects’ criteria,
projects would be removed from the MPR once Final Materiel Release (FMR) and
Final Operational Capability (FOC) have been achieved.[25]
2.29
The ANAO provided an explanation of FMR and FOC and stated:
Final materiel release ... is basically when DMO consider
they have completed their contract. They hand the equipment over to the
Department of Defence and the Department of Defence then undertake training,
testing and all the rest of the things that they need to do to put it into use.
Final operational capability is that period of time when the capability manager
accepts that it is ready for use.[26]
2.30
The ANAO commented that the DMO in its relationship with the Department
of Defence ascertains completion of a project as the point where FMR is
achieved, and that the Department of Defence uses FOC.[27]
In addition, the ANAO stated:
By way of background, in February 2010 the Defence Committee
(Departmental) agreed to the adoption of the concepts of Initial Materiel
Release (IMR) and Final Materiel Release (FMR) to clearly and explicitly define
the mechanisms where the materiel element of capability is formally transferred
from the DMO to the Capability Manager. This change acknowledges that
achievement of FOC is the responsibility of the Capability Manager (and not the
DMO), and includes the addition of items such as training for pilots, and
completion of aircraft hangers, which are part of the Fundamental Inputs to
Capability.[28]
2.31
The DMO is currently transitioning from using Initial Operational
Capability (IOC) and FOC to Initial Materiel Release (IMR) and FMR. The DMO
stated that:
IMR and FMR will mark the DMO milestones for delivery and
release to the Capability Managers or materiel supplies to support the
Capability Manager’s achievement of IOC and FOC. The IOC and FOC are Defence
milestones that represent the estimated timeframe for when a capability system
... will achieve full capability. Consequently, the shift to IMR and FMR will
provide greater clarity of responsibilities between the DMO and Capability
Managers.[29]
2.32
The DMO acknowledged that there is a tension between initial and final
materiel release and achieving full operating capability.[30]
The DMO further stated that IMR and FMR, not FOC provide the best measures of
the DMO’s effectiveness as this is the point where DMO’s responsibility in
terms of project management ends and the project becomes the responsibility of
the Capability Manager (who is attached to the Department of Defence). The DMO
stated:
When the MPR was developed several years ago, Final Operating
Capability was considered to be the logical end point at which projects would
qualify for removal from the MPR. However, in February 2010 the Department’s
Defence Committee agreed to the concepts of Initial Materiel Release and Final
Materiel Release to explicitly define the points at which the materiel element
of capability is formally transferred from the DMO to the capability manager.
This change acknowledges that achievement of Final Operational Capability is the
responsibility of the Capability Manager and not the DMO. Hence, Initial
Materiel Release and Final Materiel Release, - not Final Operational Capability
– provide the best measurement of the effectiveness of DMO.[31]
2.33
The ANAO suggested that the exit criteria for an MPR project include
both the FMR and FOC stages as the MPR is an accountability document on DMO’s
performance, but importantly also reports on project visibility.[32]
2.34
The ANAO also stated that the DMO is undertaking an analysis of the
‘difference in scale, size and incidence of requirements to be completed
between the FMR and FOC.’ The Guidelines for the 2010-11 MPR include for both
FMR and FOC to be reported.[33]
2.35
In addition, the DMO suggested that following their completion, projects
could stay in the MPR for an additional year to report on lessons learned. The
DMO stated:
.. a project should probably stay on for another year after
it has finally met the criteria so that we can all do the lessons learned. It
also requires some reflection to say, ‘What did we get right on that project,
what did we get wrong and how can we apply it to future projects.’[34]
Concluding comments
Accountability and transparency of project and project management
information
2.36
The committee acknowledges and accepts the comments made by the
Australian National Audit Office (ANAO) that the 2009-10 Major Projects Report
(MPR) includes a greater level of information about each project’s performance
and that it provides a basis for greater longitudinal analysis on project
performance in future years.
2.37
The committee also notes the Defence Materiel Organisation’s (DMOs)
comments in relation to cost ‘blow-outs’ and schedule delay of projects as
reported in the media from time to time and believes that the DMO could do more
to ensure that systems are in place to prevent schedule delays. The committee
believes that, if not already occurring to a large extent, that it is
inevitable that schedule delays will, in the long term lead to increased costs
associated with managing and completion of projects, especially where those
delays are lengthy.
Guidelines for the project data summary sheets
2.38
The committee has noted the changes associated with the 2010-11 MPR
Guidelines and exit criteria for projects and endorses them as the framework
for the Project Data Summary Sheets (PDSS).
2.39
The committee received the DMO’s proposed list of projects for inclusion
at the commencement of the 43rd Parliament which afforded adequate
time for the consideration and endorsement of the list.
2.40
The 2010-11 Guidelines were requested at the public hearing for the
review of the 2009-10 MPR on 28 February 2011 and made available to the committee
in late March 2011.
2.41
Given the short time frame for the production of each annual MPR (which
includes the ANAO’s review), the committee could undertake its role more
effectively if it were provided with the Guidelines for the following year’s
MPR at the same time that it receives the list of possible projects for
inclusion. This would provide more time for the committee’s annual review of
the MPR. This would also avoid possible inefficiencies for the DMO in meeting
the annual MPR work schedule.
Projects of concern
2.42
Projects are classified as Projects of Concern (PoC) when they encounter
significant challenges with scheduling, cost, capability delivery or project
management. The list of POC was created in 2008 by the Minister for Defence and
is separate to the annual MPR process of the reporting of selected major
capital acquisition projects.
2.43
The committee notes that five of the 12 currently listed PoC are
reported on in the 2009-10 MPR. All projects that are included in the MPR have
met the requirements of the agreed criteria as specified in the 2009-10 MPR
Guidelines.
2.44
The option for DMO to report to the committee specifically on PoC within
the MPR was considered in regard to two main issues. These are: the possible
increase in the cost of producing the MPR, and the possible increase in the length
of time a project continues to be reported on in the MPR.
2.45
The committee believes that projects classified as PoC should not
specifically be reported on simply because they are PoC projects, but rather
because they meet the criteria of being major projects as included in the
Guidelines for the PDSS.
Exit criteria
2.46
For the first time since review of the MPRs commenced, projects that
have been completed may be exited from the MPR. In this regard, the committee
was asked to consider the criteria for the exit of projects from the MPR.
2.47
The point at which projects are considered complete and so able to be
taken out of the MPR was presented to the committee as the points at which
either Final Materiel Release (FMR) or Final Operational Capability (FOC) is
achieved.
2.48
The committee understands that FMR is when the DMO considers it has
completed its contract for a particular capital acquisition project and FOC is
the point at which the Department of Defence accepts that the capital or
equipment is ready for use.
2.49
The DMO put forward the view that FMR should be the point at which a
project is considered complete for the purpose of the MPR and hence should no
longer be reported on in the MPR. At the point of FMR, the DMO’s responsibility
in relation to project management ends and the project becomes the
responsibility of the Department of Defence.
2.50
The ANAO suggested that the exit criteria for projects from the MPR
include both FMR and FOC as the MPR is an important accountability document on
the DMO’s performance and allows for project visibility.
2.51
The committee notes both the viewpoints presented by the DMO and the
ANAO, and also notes that the 2010-2011 MPR Guidelines include both FMR and FOC
as the point of exit for projects for the MPR.
2.52
The committee acknowledges that the MPR document is an accountability
document and understands the difference in the role of the DMO and the
Department of Defence in regard to projects included in the MPR. However, the
committee also believes there is value in terms of increased transparency and
improved accountability in including reporting on both FMR and FOC before
projects are removed from the MPR.
2.53
The committee also acknowledges and supports DMO’s efforts in its
current analysis of the difference in scale, size and incidence of requirements
to be completed between FMR and FOC. The committee awaits the DMO’s report on
this analysis. The
committee will consider seeking further comment on this point from each of the
three services, the Department of Defence, the ANAO and industry generally.
2.54
The DMO also suggested that completed projects remain on the MPR for an
additional year so that lessons learned may be reported on in the consecutive
MPR. The committee believes there is merit in reporting on lessons learned from
projects and understands this is a strategic approach to planning which should
already be part of the DMO’s internal practice of monitoring and review of
projects.
2.55
In the interest of visibility of projects and in the spirit of the MPR,
the committee agrees that lessons learned on projects that will be retired from
the MPR should be reported on in regard to whole-of-organisation best practice
improvement. The committee does not see a reason for these projects to delay
the listing of future possible projects. However, the listing of completed
projects (for example in an appendix) would seem to be a reasonable compromise.
Recommendation 1 |
2.56 |
The committee recommends that the Major Projects Report
(MPR) Work Plan (which contains the MPR Guidelines) be provided to the Joint
Committee of Public Accounts and Audit concurrently with the list of proposed
projects for inclusion and exclusion in the following year’s MPR, no later
than 31 August each year. |
Recommendation 2 |
2.57 |
The committee recommends that Projects of Concern (PoC) not
be specifically included in the selection criteria for projects to be
reported on in the Major Projects Report (MPR), but where projects reported
on in the MPR are also PoC, that they continue to be identified as such. |
Recommendation 3 |
2.58 |
The committee recommends that the exit criteria for projects
reported on in the Major Projects Report be the point at which both Final
Materiel Release and Final Operational Capability (as currently defined by
the Defence Materiel Organisation and Department of Defence respectively) is
achieved. |
Recommendation 4 |
2.59 |
The committee recommends that in determining whether the exit
criteria is appropriate for future Major Projects Reports (MPRs), that the
Defence Materiel Organisation’s assessment of the difference in scale, size
and incidence of requirements to be completed between Final Materiel Release
and Final Operational Capability be provided to the Joint Committee of Public
Accounts and Audit as soon as possible to allow for the implementation of any
changes to occur for the 2011-12 MPR. In conducting its analysis, the DMO
should consult with the three services, the Department of Defence, the
Australian National Audit Office and industry representatives. |
Recommendation 5 |
2.60 |
The committee recommends that once projects have met the
exit criteria, they be removed from the Major Projects Report (MPR) and for
each project which has been removed, the lessons learned at both the project
level and the whole-of-organisation level are included as a separate section
in the following MPR. |
Navigation: Previous Page | Contents | Next Page
Back to top