Chapter 15

Conclusion

...it takes many things for an acquisition to succeed, while only one source of unmanaged risk can cause a poor outcome.[1]

15.1 This chapter draws together the evidence presented in the previous chapters in order to answer one of the key questions driving the committee's inquiry—whether entrenched structural impediments to efficient and effective leadership within Defence are at the source of Defence's procurement woes. In this report, the committee has sought to establish whether a reallocation and redefinition of roles, functions and responsibilities is required. Indeed, whether the current management matrix needs to be overhauled or even dismantled.

Challenges and Defence's responses

15.2 The committee notes and supports evidence that highlights the competence, dedication and hard work of people at all levels of Defence's procurement process.

15.3 Defence projects for acquiring major capital equipment face an array of internal and external forces and influences that create significant difficulties for Defence. In fact, such projects are of a scale and complexity that they present 'formidable and ever-increasing challenges'.[2] The problems identified in defence procurement, however, are largely a function of the Defence organisation's own making—unintentionally self-inflicted. They include: inadequate planning and scoping of project; poor risk management from beginning to end of project; failure to appreciate the developmental nature of the project or complexity with integration; poor project management; underestimation of defence industry capacity; lack of skilled workforce; inadequate contracting arrangements; insufficient consideration of through-life support; and a breakdown in the relationship between the relevant service, DMO and contractors.

15.4 The committee finds that the current management structure in Defence has produced an organisation that lacks a robust risk regime: an organisation where its personnel are insensitive or unresponsive to risk, where no one owns risk and is incapable of learning lessons from past mistakes. In brief, Defence is currently an organisation that cannot anticipate, understand or manage risk—a fundamental flaw in an organisation that undertakes large-scale and complex projects that are in essence engineering operations. Importantly, this failure to own risk and to learn lessons is not a process problem—it is clearly a weakness deep within the organisation that permeates outwards and effectively precludes people from taking responsibility and being accountable.

15.5 The inability of Defence to learn from past mishaps is a particularly salient point. Defence may well argue that the failures noted in this report are drawn from history: but if it cannot or will not apply lessons from previous projects to current and future ones then it is destined to repeat them. Learning lessons is not only about keeping a risk register or a data base—that is simply a process—it about those lessons becoming part of the corporate knowledge.

Process versus genuine reform

15.6 Defence's responses to the evident failings in their procurement projects have tended to focus on process. Even its most recent initiatives to clarify responsibilities and strengthen accountability look to process for solutions—re-badging and improving the Initiation and Project Review Boards; emphasising the role of joint project directives, enforcing Materiel Acquisition Agreements (MAAs); and introducing project charters (Mortimer recommendation). If implemented and properly adhered to, such measures should go some way to reduce the opportunities for a project to slip off the tracks. The committee is concerned, however, that such measures merely promote form over substance and it remains to be convinced that in practice they are effective.

15.7 The committee has highlighted prevailing practices and circumstances in Defence that sabotage its endeavours to realise the objectives of these initiatives. They include non-compliance with policy and guidelines, an environment that has generated multiple and confusing layers of bureaucracy, poor linkages between key agencies, and a lack of, or mismatch of, appropriate skills. Thus, despite Defence's confidence in its initiatives, the committee envisages that, with the passage of time, the damaging behaviours, which have simply been papered over, will again surface to perpetuate the pattern of poor performance. For example, the committee fears the potential for the Project Initiation and Review Board to turn out to be a simple re-packaging of the Options Review Board and hence replicate the same shortcomings—an unwieldy committee made up of a number of groups lacking authority and whose members do not have the required experience or specific competence for the task. The committee has heard nothing to indicate that, despite current enthusiasm for the boards, they will not revert to form.

15.8 Indeed, the weight of evidence indicates that not only has Defence's preoccupation with process been misguided but it has been counterproductive. In response to identified problems, Defence has created a procurement process that is convoluted and overburdened by administration. The committee is of the view that the entire organisational structure of Defence must be simplified and streamlined. Only by reducing the number of stakeholders (groups) involved in the process can the excessive administrative burdens and committees be reduced.

15.9 Defence is also convinced that it has a robust quality assurance framework with the revamped gate reviews and the independence of the DSTO, DMO and CIR Div. It is of the view that these independent bodies provide the necessary contestability to ensure that decision-makers are provided with a range of well-considered, impartial and specialist advice.

15.10 In this regard, the committee acknowledges that Defence has a quality assurance framework that is designed to provide internal contestability and external scrutiny. But again, the committee finds that the overall impression of Defence as an organisation with a healthy and open approach to independent review and diversity of views fails to match facts.

15.11 All sectors, including industry and defence analysts, supported the work of the gate review boards. A very worrying development noted in the ANAO's audit of gate review boards, however, was the growing tendency for a manager with a direct connection to the project to be appointed chair of the review. According to the ANAO this trend has increased in recent times:

During the first year DMO conducted Gate Reviews (July 2009–June 2010), 33 per cent of Gate Reviews of ACAT I and II projects were chaired by a manager with some responsibility or accountability for the project under review. During the second year (July 2010–June 2011) this increased to 42 per cent. During the first six months of IPPO's management of all Gate Reviews this increased further to 50 per cent.[3]

15.12 Throughout this report, the committee has referred to numerous instances of non-compliance with policy or guidelines. The gate review examples cited by the ANAO throw into sharp relief, how genuine, sound reforms can be rendered useless by a management structure that cannot or will not exert authority. This latest clear disregard of policy whereby the independence of gate review chairs was compromised underlines the committee's scepticism about the effectiveness of other recent initiatives such as project charters, MAAs, and the project initiation and review board.

15.13 In the committee's view, Defence have been tinkering at the margins of the problem, giving the impression that by improving process, the desired change in behaviour will follow. The committee believes that such an approach only serves to mask fundamental weakness in the overall management structure of Defence and its major acquisition programs. Thus, despite a raft of reforms and reliance on Defence quality assurance frameworks, the persistent pattern of poor project performance continues. Problems such as mistaking a developmental project for a genuine off-the-shelf product indicates that this internal filter and the gate reviews have not worked as well as they should. Indeed, the reforms have done nothing to prevent highly developmental projects being submitted to the Chief of the Defence Force and the Secretary of the Department and ultimately to government as off-the-shelf products. Clearly, the answer is not more process.

15.14 With regard to the independent advice provided by agencies such as DSTO and DMO, and advice obtained (or not) from domain experts and industry, the committee again finds Defence's depiction of their effectiveness at odds with reality. The committee cited observations about DSTO's advice not receiving the respect it deserves, of Defence agencies generally undervaluing technical advice and, in some major projects, downplaying, misinterpreting or even completely disregarding specialist domain advice such as pre-contract T&E reports.

15.15 The committee also noted Defence's response to Mortimer's recommendation that the CEO DMO 'should provide independent advice to Government on the cost, schedule, risk and commercial aspects of all major capital equipment acquisitions'. According to Mortimer, the CEO DMO should also be a permanently invited adviser to government committees considering defence procurement.[4] Defence made clear that the views of organisations including CDG, DSTO, and capability managers 'must be properly reflected in the cabinet submissions'.[5] It stated further, however, that 'it would not be appropriate for DMO to make coordinating comments on Defence cabinet submissions because, for procurement matters, DMO is intimately involved in preparing these submissions'. Defence's approach contradicts the Mortimer principle and effectively negates one element of contestability which relies on independence for its effectiveness.

15.16 It is also important for agencies' advice and recommendations to be clearly discernible so that they can be held accountable for them. But Defence's 'one view', mantra effectively removes diversity of opinion provided by specialist agencies, experts and senior Defence leaders by presenting just one position. A number of witnesses acknowledged that it was appropriate for Defence to speak with one voice 'provided that what that one voice was saying had been arrived at via a process of thorough contestability and lots of frank and fearless advice, carefully listened to within Defence'.[6] Unfortunately such is not always the case in Defence.

Disenfranchised capability managers

15.17 The most glaring consequence of Defence's failure to effect meaningful reforms has been the disenfranchisement of the capability managers. Capability managers ultimately operate the equipment or platform being acquired and are responsible for ensuring that the acquisition is fit-for-purpose In 2003, Mr Kinnaird argued that:

Capability managers, the most prominent being the Service Chiefs, should be made responsible and accountable for monitoring and reporting to government on all aspects of approved defence capabilities.[7]

15.18 This responsibility would be for 'the whole of capability from the point where government approves a particular capability option, that is at second pass approval, through to the time that the capability is retired from service'.[8] Mortimer also made a number of recommendations that, if implemented properly, would make capability managers an integral and engaged part of the acquisition process. This would include capability managers reporting regularly to government 'on the status of the capability development initiatives for which they are accountable'.[9]

15.19 Yet evidence before the committee is unequivocal—capability managers have been left out of the acquisition loop. For example, Mr King accepted that there was a time post Kinnaird where the centralisation of the capability development under CDG and the DMO operating as the acquisition organisation 'appeared to disenfranchise the capability managers in the process'.[10] He stated that the situation led to 'a period where, despite having the two pass process in place, the CM, CDG and DMO were not interacting, coordinating and integrating as well as they might'. This breakdown in communication was particularly evident in the maritime space. Mr King explained in simplified terms what he thought had happened:

...the customer base―the capability manager―had developed a feeling that DMO would just pass something or throw something over the fence at them and they would have to take it. I think they had fallen into a mode of 'Well, I'll see if I like it when I get it.'[11]

15.20 It is clear that capability managers have much ground to recover and must regain authority over key areas of capability development, particularly the responsibility for determining the technical specifications they require for acceptance into service. Most notably, this applies to the Chief of Navy. Capability managers must also have adequate and appropriate resources, including a core of trained professional engineers, in order to exercise their responsibilities. They must also be held to account for the way in which they exercise that authority, which means that their decisions must be traceable back to them.

15.21 Defence believes that capability managers are now 'upfront' and cite the fact that they currently sign the MAA as evidence of that engagement. The committee has already expressed its doubts about the effectiveness of this measure. It should be noted that MAAs have been in place since 2005, yet they have failed to do their job. An important issue for the committee is how to prevent a situation developing that effectively disenfranchises the capability manager from the acquisition process.

15.22 In this report, the committee has shown repeatedly that while Defence has correct practices and procedures on paper, it fails to implement them properly. More tinkering of manuals and guidelines and policy statements and adding more process to an already overburdened one will not work. If not accompanied by genuine changes in management, such initiatives will simply compound Defence's problems. In this regard, Defence needs to pay close attention to creating an environment, especially through its management structure, that is inclusive, counters the tendency for groups to work in silos and allows those with responsibility to exercise their authority. In doing so, Defence should also be intent on removing layers of administration not adding to them.

15.23 The committee is recommending a restructuring of Defence that would ensure capability managers have a central role in the acquisition and sustainment of their major capital equipment. The intention would be to institute direct contractual agreements after second pass between clients (capability managers) and contracted providers with no third party involvement. Without such a standard commercial approach, there will be no change, only more process and more red tape clogging up the system.

Continuing struggle for skilled people

15.24 The committee also believes that by focusing on process to solve procurement problems, attention may be diverted from the more important matter of finding the right people for the right position so they can drive necessary change or implement process more effectively and efficiently. One industry representative observed:

...Organisational structures only go part way towards solving performance issues...I could have any organisation structure I like that aids communication and interaction. If [we] do not have the right people with the right competencies and the right way of behaviours, then the organisational structure is worth nothing.[12]

15.25 Throughout the report, the committee has underlined the futility of Defence resorting to more information, more process, more people and committees or paperwork rather than having 'good-quality, appropriately qualified and current staff in correct positions'.[13] If Defence wants to be a smart customer then it needs knowledgeable people with a deep understanding of what it intends to buy and highly skilled experts, who understand commercial realities and are able to negotiate with contractors to deliver value for money for the Commonwealth. For example, a number of witnesses referred to inexperienced and inadequately skilled project managers in DMO. In this regard, a representative from one of the prime contractors observed that a few of the project managers with whom he had worked were good people but 'way out of their depth trying to manage a project of which they had little experience'—'a recipe for disaster'.[14]

15.26 Too often uniformed people, with operational experience and technical knowhow, are engaged as desk officers in general project management, costing of proposals and administrative tasks. They feel undertrained and ill-suited for the tasks at hand and moreover they tend to be on short term postings of less than three years. Clearly, it is important when seconding military people to CDG and DMO, where relevant, that they are placed where their skills and experience can be best utilised. A three year posting, or less, in a managerial position for uniformed personnel is an inefficient use of otherwise very skilled and experienced people. The emphasis must be on finding the right people and placing them in the right position.

15.27 The critical shortage of engineers and allied technical skills is a matter that requires immediate and serious attention. While there are many external forces undermining Defence's efforts to attract and retain skilled engineers and technicians, the committee is of the view that it is imperative for Defence to grow its engineering and allied skills base. Indeed, many witnesses indicated the skills shortage in Defence must be addressed as a priority: that 'the work on retaining and attracting key personnel cannot wait until tomorrow'.[15] In the committee's view, Defence requires a far more targeted and concerted effort to build up a critical core of skills within its major acquisition groups and agencies. This also requires the creation of opportunities to gain and maintain relevant experience.

15.28 The committee is also concerned about competition between the groups in Defence involved in procurement and sustainment for skilled personnel, particularly in the engineering and technical areas, where the supply is already under pressure from demands from the private sector. The committee proposes a model for the consolidation of technical skills into each of the Services, which should address this waste. In this new organisational arrangement, capability managers would be responsible for the primary technical input to all capability proposals, test and evaluation in line with central policy, and all operational and sustainment management. This applies especially in respect to large and complex single service capability, most notably in Air Force and Navy. In this way, the committee believes it should be possible to:

minimise the inefficiency caused by intra-organisational postings and duplication;
enable capability managers to rebuild their design engineering, logistics and technical skill base capable of understanding the most sophisticated levels of modern defence technology and effectively balancing operational and sustainment considerations;
provide meaningful and rewarding skill paths for technically skilled personnel whether they be uniform or civilian, noting that stability and continuity of skill may be more achievable from the latter;
provide complementarity of skills rather than the current internal competition;
retain skilled staff on long term projects from conceptual development through to sustainment and disposal from within one organisation, fully and singly accountable;
provide a stronger technical counter to industry in contract negotiations and management, and
establish greater permanence to Defence's capacity to follow rapidly escalating technical complexity of defence capabilities around the world.

15.29 Equally, DMO with its reduced role should be better able to concentrate on building the high level skills needed for tendering, contracting and project management (whether through recruitment or contracting from industry). These skills are critical to support the capability managers who will now be responsible for the acquisition of capability. DMO as the centre of excellence would also have an independent role in assuring the quality of information going to government for initial purchase decisions.

15.30 Accepting that the Services role will be expanded and DMO's role changed, the committee recognises the implications of this model for capability planning in Defence Strategy Group and CDG. With regard to contestability, the planning and reporting arrangements will need to ensure that the independent voices of DMO, DSTO and expert agencies such as T&E can be heard by key decision-makers without fear or favour.

Underperforming organisation

15.31 Overall, the committee found that Defence is an organisation that has:

a growing disconnect between strategic guidance and capability development with the current foundation document—the 2009 White Paper—setting an unrealistic and unachievable acquisition program for the ADF's future capability;
a culture of non-compliance with policy guidelines and practice manuals; where personnel get 'bogged down' with too much paper work, produce a 'certain amount of nugatory work' and 'miss the important things going on';[16]
confused and uncertain lines of responsibility and accountability that are too diffuse to be effective—the organisation is unable or unwilling to hold people to account;
a poor alignment of responsibility due to the excessive number of groups and agency functions, which gives rise to unhealthy management and organisational relationships—for example capability managers removed from active participation in an acquisition;
weak compliance assurance with and between agencies or groups creating an environment where, for example, DSTO assessments or technical advice from domain experts can be undervalued, or even discarded without checks and balances to make sure dissenting voices are heard by decision-makers; and
a poor understanding of the commercial world and as a consequence Defence is yet to engage actively with industry as a collaborative partner in capability development and acquisition.

15.32 The challenge for Defence is to change an organisational structure with entrenched attitudes that despite repeated reforms:

cannot learn lessons from past mistakes;
still resorts to changes to process rather than implementing genuine organisational reforms designed to clarify responsibilities and make individuals accountable for their decisions and performance;
has effectively disenfranchised the end users—capability managers—who have been left on the sidelines, without authority over key areas of capability, procurement and sustainment;
disempowers project managers, most of whom are diligent and hardworking, but, without clear lines of delegated authority, are unable or unwilling to make decisions and take responsibility;
fails to understand and appreciate the importance of contestability and simply cannot, or refuses to, comprehend the meaning of 'independent advice';
despite having a 'One Defence' view does not operate as an integrated enterprise but rather remains an organisation composed of separate groups working to their own agendas;
struggles to attract and retain people with the required level of skill and experience to support acquisition activities including negotiating with business, tendering and contracting, and particularly engineering, having over the past 15 years or more allowed its mainstay skills to atrophy especially the hollowing out of technical skills in Navy;
undervalues technical advice and has serious shortcomings in technical analysis, critical to engineering based projects; particularly its downgrading of the importance of T&E; and
is yet to achieve the status of smart or intelligent customer and, for some major projects, has a troubled history with its suppliers—poor tendering and contracting practices and overall lack of business acumen.

Need for structural reform

15.33 The recommendations in this report take account of Defence's attempts to remedy shortcomings. They also recognise that Defence has made these efforts while simultaneously attempting to comply with multiple reform agendas arising from a string of government reviews and directives. The central overriding recommendations, however, underscore the importance of Defence becoming a self critical, self evaluating and self correcting organisation. To do so, the committee believes that leadership, accountability and the correct alignment of responsibilities is required. This means that roles and responsibilities should be clearly defined so that a single point of accountability is attached to single senior positions, not organisational structures. These roles and responsibilities should also be complementary.

15.34 The headline recommendations deal with much needed organisational change in Defence in order achieve the correct realignment of responsibilities and functions of relevant agencies, and provide them with the skills and resources they need to fulfil their obligations. In this context, the emphasis is on bringing increased clarity and transparency to responsibility and who owns it—to reduce the diffusion of responsibility and decision-making. More specifically, the recommendations are intended to:

return responsibility to capability managers and make them accountable for decision-making and performance under their areas of authority;
make DMO a streamlined and specialist acquisition agency;
inject real contestability into decision-making and guarantee that the government is provided with independent advice from key agencies—DSTO, DMO and technical experts; and
ensure that Defence's focus is on obtaining the right people with the right skills and experience and matching their skills with the right job: that Defence also manages its skill base in such a way that agencies complement their skill requirements and do not compete for skills from the same pool of specialists.

Recommendations—proposed model

15.35 The committee proposes a model that, after second pass decision, allocates one single point of accountability for every project to the relevant capability manager, supported by financial delegation and budget control. It reduces CDG’s role with savings, and limits DMO's functions—thus eliminating much overlap. It also reinforces the Kinnaird/Mortimer concept for internal independence for the purposes of genuine contestability, and it reduces the waste of skill through inappropriate placement, duplication and misalignment of skills. The committee's proposal also introduces a direct client/provider model without any intermediaries, with precise accountability. Under this model, the DMO would become a contract and project management specialist supporting the capability manager at relevant points in the acquisition and sustainment cycles.

15.36 This model would remove the unnecessary layers of current vested interests and streamline the process through a single point of accountability. In short, it is a greatly simplified model aided by significant streamlining. It builds on the strengths of accountability in the services (as identified by the Black Review) and seeks to harness the learning and potential for alignment across the three services envisaged with the creation of DAO and DMO.[17]

Strategic Policy Division

Recommendation paragraph 8.63

15.37 The committee recommends that all matters concerning strategic planning, capability planning, industry policy, costing and all matters for the coordination of contestability from DMO, DSTO and industry should remain with the current Strategic Policy Group and CDG in combination.

Recommendation paragraph 11.93

15.38 The committee recommends that Strategic Policy Group and CDG should have more strategic analytical skills to test rigorously and independently the capability managers’ development of the Defence White Paper capability elements, restoring the creative tension but free of competition for skills.

Capability managers

15.39 The committee recognises that capability managers have been sidelined with CDG and DMO assuming key positions during the acquisition phase. To ensure that they have the power and capacity to discharge their duties, capability managers require the authority that now resides with the CDG as departmental coordinator and centre of power. In the committee's view, the priority should be on giving the capability manager appropriate control over the acquisition ensuring all the while that the responsibilities of CDG, DMO and the capability managers are complementary.

Recommendation paragraph 8.64

15.40 The committee recommends that accountability for all service specific procurement items should be exclusively transferred with budgets to service chiefs, who should be responsible for all procurement and sustainment of their materiel. This transfer of responsibility occurs after proposals have been thoroughly tested internally and externally and after government decisions are made at second pass.

Recommendation paragraph 8.65

15.41 The committee recommends that the capability manager should have expanded responsibility and importantly financial responsibility after second pass. Under the committee's recommended model, for all acquisition projects, the capability manager would be the sole client with the contracted suppliers; DMO's role being limited to tendering, contracting and project management specialities, strictly according to the terms of the second pass decision. All specification changes should be monitored by CDG and put to government for agreement, as currently the practice, with the capability manager to be fully accountable.

15.42 The committee is of the view that in considering the restructuring of the organisation, Defence must look closely at the skills required by the respective agencies and while maintaining strong contestability, ensure that specialists are located where they are most needed and not unnecessarily duplicated or spread too thinly throughout the organisation.

15.43 Clearly, capability managers need to have the technical experts within their service able to provide high level specialist advice on a project proposal from its inception through acquisition, delivery and sustainment. They should have the responsibility for growing, developing and retaining that skills base. This is particularly so, given that under the preferred model they are to be largely responsible for technical input before and after contract—that is at the heart of the new accountability the committee seeks to achieve.

Recommendation paragraph 11.94

15.44 The committee recommends that, after second pass, capability managers have sole responsibility for acquisition projects, supported by staff seconded through the DMO, as well as maintaining relationships with contractor and sub contractors.

Capability Development Group

Recommendation paragraph 8.66

15.45 The committee recommends that all matters of coordination, overall budget management monitoring and reporting after second pass should remain in the current CDG, but without budgetary control.

Defence Materiel Organisation

15.46 If capability managers are to be empowered; if they are to exert greater control over the acquisition of a capability they will use, then DMO's role must change as well. To be effective, DMO must be responsible for the standards to be applied to tendering, contracting and project management and have independent access to the minister.

15.47 Responsible for setting the standard for contracting and project management for the acquisition of the majority of capital equipment, DMO is a key stakeholder in the capability life cycle. It brings a particular perspective to capability development and its voice should be included in advice to CDF and Secretary, and to minister and cabinet. For the purposes of genuine contestability, organisations such as DMO and DSTO should be truly independent of Defence with accountability direct to the minister pursuant to ministerial directive. The Ministerial Directive was a key accountability document for defence capital projects between 2005 and 2008. It established the CEO DMO's direct obligations to the minister, his overarching responsibilities and his management priorities in relation to DMO's business outcomes.

Recommendation paragraph 10.82

15.48 The committee recommends that the minister review, update and reinstate the Ministerial Directive to CEO DMO. The directive is intended to set boundaries and expectations and establish clear accountability for achievement of Defence capital acquisition programs. It should include the requirement that CEO DMO provides independent advice to the minister in DMO's specialist area of major capital projects.

Recommendation paragraph 10.83

15.49 The committee recommends that the government should again look carefully at making DMO a statutorily independent agency, as previously recommended by Kinnaird and Mortimer, but rejected by Defence and government. The CEO’s salary should be set by the Remuneration Tribunal and, as stipulated in the previous recommendation, direct access to the minister should be restored pursuant to a re-instatement of a ministerial directive which has fallen into disuse. The intention behind this recommendation is to find a better way to: guarantee DMO's independence and assist it to provide frank advice to government, have its functions and responsibilities spelt out in legislation and allow it more latitude to employ specialist personnel.

Specialist acquisition organisation

15.50 For almost a decade, DMO has been actively endeavouring to make itself a more business-like organisation with the required skills to function as a high performing acquisition agency. The committee recognises the work the organisation is doing to achieve that objective but notes that it is falling short. The committee notes in particular that the skills of uniformed personnel seconded to DMO may not match the tasks they are required to undertake, which is further complicated by their short term tenures. The committee recognises that the DMO needs highly skilled project managers and also tendering and contracting specialists.

Recommendation paragraph 11.95

15.51 The committee recommends that the government ensure that the DMO has the funds, means and government support necessary to consolidate and build on the efforts already underway to develop its multidiscipline skills base with the ultimate goal of achieving a world-class acquisition community.

Recommendation paragraph 11.96

15.52 The committee recommends most strongly that the organisational changes specified in the recommendations dealing with skills be adopted, and that the streamlining and consolidation of skills identified be the primary focus and outcome in securing that change.

Defence Science and Technology Organisation

Recommendation paragraph 10.84

15.53 The committee recommends that the minister consider how best to ensure that DSTO's specialist advice on technical risk associated with Defence's major capability developments are conveyed to government in a clear and accurate way. The Ministerial Directive to CEO DMO may serve as a model.

Recommendation paragraph 10.85

15.54 The committee recommends that the Technical Risk Assessments and Technical Risk Certifications (currently presented to the Defence Capability Committee and the Defence Capability and Investment Committee) should be a joint activity overseen by the relevant Service T&E agency head and the Chief Defence Scientist. In light of past underestimation of technical risk, the intention would be to review past experiences and current documentation to determine how risk assessments could be better presented to non-technical experts to minimise the opportunity for risk assessments to be misinterpreted.[18] The reporting structure also needs to be transparent such that assessments cannot be ignored without justification to the key decision-makers (e.g. minister).

Other recommendations

15.55 In addition to changing the organisational structure, the committee also recommends action be taken on matters such as the proposed purchase of the 12 new submarines, Air 8000 Ph 2, the 2013 White Paper, the DCP, MAAs and gate reviews and finally on T&E.

Future submarines SEA 1000—applying lessons

15.56 The committee is very concerned about the current unease expressed by a number of defence analysts regarding decisions already taken on the 12 new submarines. Recent announcements in relation to studies to consider procurement options for the future submarines together with studies in relation to an industry skills plan are encouraging. Nonetheless, early decisions reflect troubling signs that one of the centrepiece projects listed in the 2009 White Paper is yet to undergo thorough analysis and consideration.[19]

Recommendation paragraph 3.20

15.57 Because this project is still at an early stage, and based on the RAND study, the Coles Report, independent defence analysts and the past performance of major Defence acquisition projects, the committee recommends that government and Defence start work immediately to:

ensure that the program is directly managed by Chief of Navy supported by the ASC and DMO where relevant, the scientific community and the public—support must be both external to the program and internal within the navy and submarine community;[20]
avoid early lock-in through premature weapons systems choices;
ensure that the capability sought is available and minimises developmental risks;
take drastic action to address the serious skill shortages identified by RAND before a decision on assembly in Australia is made, regardless of type and design;
ensure that the program is open and transparent—full disclosure throughout the program is necessary to obtain government, industry and public support;
involve experienced people in key management positions—this requires a strategy to grow people so they are experienced in various disciplines—a top-level strategic lesson must be implemented far in advance of any specific program; and
listen to technical community concerns about risk—the technical community, supplemented by outside expertise from industry and allied technology partners as necessary, should understand the state of technology and the degree to which a new design extends that technology.[21]

Recommendation paragraph 3.22

15.58 The committee recommends that government and Defence respond publicly to the committee's criticisms made in this report with respect to lessons not learnt, and outline the detailed process and all the options on which current planning on submarines is taking place.

15.59 The new White Paper presents an opportunity for the government and Defence to start to provide assurances that the decisions relating to SEA 1000 are based in sound, robust and fully considered analysis.

AIR 8000 Ph 2 (Battlefield Airlift—Caribou replacement)

15.60 Intended to enhance the ADF's intra-theatre and regional airlift capability, the Air 8000 Phase 2 project focuses on the provision of an intra-theatre airlift solution with some inter-theatre application.[22] According to the Capability Plan, Phase 2 'will provide appropriate training support, which could include the provision of a Full Flight Simulator'.[23]

15.61 The project has been accelerated in order to benefit from the advantageous pricing through an FMS case. It is assumed by Defence to be an OTS acquisition and therefore low risk.[24] The committee's attention is drawn, however, to the incomplete state of the US Air Force military certification activities. It appears that to date, Defence has not tasked people qualified and experienced in risk identification to complete a detailed evaluation of the gaps in capability and certification nor of the suitability of proposed training simulators to meet training needs. The pattern appears ominously familiar to the committee.

Recommendation

15.62 The committee recommends that the Chief of Air Force as the relevant capability manager require a report by the relevant T&E agency against the approved Statement of Operational Requirement to provide early identification of potential issues that could delay introduction into service.

The 2013 White Paper—clarity on future capability

15.63 The committee's concerns in relation to the accuracy of Defence costings are reflected throughout this report in terms both of the overall budget and individual projects. In relation to transparency, the committee emphasises that greater detail needs to be provided in the Defence White Paper, portfolio budget statements and Defence Annual Reports. The committee urges the government to ensure that the 2013 White Paper provides clarity on future capability including funding commitments and scheduling underpinned by comprehensive analysis. The primary step toward better alignment between strategy and capability development would be to ensure that the White Paper—'the corner stone document'—sets out a realistic and achievable program for capability development.

Recommendation paragraph 3.65

15.64 The committee recommends that the 2013 White Paper is prepared in such a way that all procurement proposals are costed and scheduled realistically and that Defence undertake comprehensive consultation with industry before decisions on inclusion are made, or alternately, a green paper is issued in advance for broader and open public consultation.

Recommendation paragraph 3.66

15.65 The committee recommends that, commencing next financial year, Defence publishes as an addendum to its portfolio budget statements, all the current financial detail of planned capability from the time of inclusion in the DCP, right through to contract completion and provision for sustainment, for all projects over $30 million for total procurement and lifelong sustainment.

Improving gate reviews

15.66 The committee has registered its lack of confidence in the effectiveness of the measures that Defence has taken to improve its procurement processes—project initiation and review boards, project directives, and gate review boards. This is not to say that the improvements to these mechanisms should not go ahead. As noted earlier, the committee is concerned about adding to an already bureaucratically overburdened process and any such measures to improve or introduce boards, directives, charters and agreements must always be guided by the principle of simplification wherever possible. With this principle in mind, the committee supports for example the strengthened gate reviews and believes that within the right organisational structure they hold promise.

15.67 A number of witnesses strongly supported Defence's revamped gate reviews which are an improvement on their predecessors especially the inclusion of two independent experts. The committee, however, does not want to see the contribution of gate reviews rendered ineffective because of a fundamentally flawed management structure. The committee underlines the importance of Defence ensuring that the members of the gate review boards have the relevant skills, knowledge and competencies to scrutinise the proposals before them effectively. The committee would like to see the independence of the external members guaranteed and their ability to provide genuine contestability assured.

15.68 In this regard, the committee believes that the gate reviews should be overseen by a body that can exert its independence and authority to ensure that gate reviews remain at arm's length from the influence of those with a vested interest in the project under consideration. The contraventions identified by ANAO require Defence to look carefully at ways to safeguard the integrity of these reviews. The committee would also like to see concrete measures taken to ensure that the implementation of recommendations made by the review boards are monitored, recorded and reported to the relevant capability manager, CCDG and CEO DMO.

Recommendation paragraph 10.77

15.69 The committee notes concern about the gate reviews losing their potency and simply becoming part of the process if overused. The committee believes an annual gate review for major projects would add value but recognises that the format and/or structure may need to be scaled to suit project scope/cost. The committee recommends that full gate reviews be:

mandatory for major projects at the following specified milestones—DCP entry; project initiation and review board consideration; first pass approval; second pass approval; contract solicitation and contract negotiation; and
mandatory when a project starts to diverge from original cost or schedule or when significant changes to scope are proposed.

Recommendation paragraph 10.78

15.70 In light of revelations about breaches of policy such as chairs of boards having line management responsibility and of misunderstandings stemming from the documentation provided to the gate review boards, the committee recommends further that the Independent Project Performance Office (IPPO):

exert stronger compliance checks to guarantee the independence and impartiality of the gate review board particularly enforcing the requirement that the chair of the board must not have line management responsibility for the project under review; and
exercise greater scrutiny of the documentation provided to the review board to ensure that it is relevant and complete including reports on technical risk.

To ensure that the IPPO has the authority and resources to discharge its functions, the committee further recommends that Defence consider carefully whether the functions of the Office should be located in CDG or another agency.

Recommendation paragraph 10.79

15.71 With regard to ensuring that the recommendations of the review boards are implemented, the committee endorses the ANAO's recommendation that 'Defence ensures that a control mechanism be deployed to monitor the status and completion of actions recommended by Gate Review Assurance Boards and agreed by the relevant executive'.[25]

Test and evaluation—building capability

15.72 In its report on materiel acquisition and management in Defence, tabled in March 2003, the committee expressed a lack of confidence in Defence's 'capacity or will to address T&E concerns seriously'. Five years later, in its 2008 T&E Roadmap, Defence highlighted a raft of shortcomings in Defence's T&E pointing to a need for greater funding, improved training and attracting and retaining skilled and experienced personnel. Now, Defence is still talking about producing a manual—that is about process.

Recommendation paragraph 12.51

15.73 The committee recommends that the government make a long-term commitment to building technical competence in the ADF by requiring Defence to create the opportunities for the development of relevant experience.

Recommendation paragraph 12.52

15.74 The committee recommends that capability managers should require their developmental T&E practitioners to be an equal stakeholder with DSTO in the pre-first pass risk analysis and specifically to conduct the pre-contract evaluation so they are aware of risks before committing to the project.

15.75 Given that the capability to conduct this T&E and analysis needs to be extant prior to the commencement of any given project, the committee is concerned that cost pressures will lead individual services and projects to degrade this capability over time.

Recommendation paragraph 12.54

15.76 The committee recommends:

the immediate finalisation of central defence policy on T&E to be implemented by capability managers in line with the committee’s recommended shift of full accountability for capability managers for all technical assessment of capability procurement and sustainment (independently assessed in conjunction with DSTO);
full responsibility for the implementation of prescribed T&E processes be assigned to capability managers for all procurement activity from inception through to acquisition and sustainment; and
each capability manager should ensure adequate skilled resources to oversee all T&E activity in line with central policy, as part of all acquisitions, including MOTS, as part of the capability managers’ total responsibility for procurement, but prior to as well as after second pass.

Recommendation paragraph 12.55

15.77 The committee recommends that Defence build on the capability already extant in aerospace to identify training and experience requirements for operators and engineers in the land and maritime domains and apply these to ADTEO. Capability managers will need to invest in a comparable level of training to enable their personnel to conduct (or at least participate in) developmental testing. The intention is to provide a base of expertise from which Defence can draw on as a smart customer during the first pass stage and to assist in the acceptance testing of capability.

Recommendation paragraph 12.56

15.78 The committee recommends that Defence mandate a default position of engaging specialist T&E personnel pre-first pass during the project and on acceptance in order to stay abreast of potential or realised risk and subsequent management. This requirement to apply also to MOTS/COTS acquisition.

Industry—planning for investment and early engagement with Defence

15.79 Defence's approach to its dealings with industry—planning, acquisition and sustainment for defence projects—is essential for the successful delivery of Force 2030. Industry's ability to plan for, and invest in, people and facilities to deliver future defence projects is significantly dependent on the information Defence provides about its intentions. The DCP and Defence White Papers are the main public information tools and key planning documents for industry.[26] Clearly, from industry's perspective, they fall short in providing the level of certainty and confidence that industry requires to be an effective partner in capability development. Industry identified the following problems:

access to information—the White Paper and CDP deemed to be inadequate and unreliable;
workflows—feast and famine which severely affects industry's ability to retain staff—undermines business' confidence and willingness to invest in future projects;
slow down rate in approvals—affects both confidence and also industry's ability to hold on to skilled workers;
industry not engaged early enough in the capability development process;
strained relationship between DMO and industry due to DMO's lack of business acumen, poor contracting practices;
contracting arrangements a disincentive for industry to value add; and
inappropriate risk sharing, fixed price.[27]

Recommendation paragraph 13.55

15.80 The committee recommends that Defence make their DCP a document that provides industry with greater certainty about its plans and intentions for future capability development to enable industry to invest with confidence in capability development. In particular, the committee recommends that the next DCP include:

a schedule that provides anticipated timelines for the construction and delivery of all DCP items, with continuity the key feature;
a detailed explanation on this acquisition schedule indicating the reasoning and analysis behind it and how Defence has taken into account demand flows; and
reliable cost estimates.

15.81 Increased industry involvement earlier in the capability development process is clearly an important factor in the successful delivery of defence capability. In this regard, the committee welcomes the reinvigoration of the Capability Development Advisory Forum and the environmental working groups. They provide an ideal opportunity to involve senior defence industry representatives early in the capability development phase without compromising the integrity of an acquisition process.

Recommendation paragraph 14.28

15.82 The committee recommends that Defence:

continue to collaborate with industry to reinvigorate the Capability Development Advisory Forum and the associated environmental working groups as a means of engaging industry early in the capability development process. The committee recommends further that Defence ensure that such engagement with industry is a genuine two-way exchange of ideas and of information; and
continue to support training programs such as Skilling Australia's Defence Industry (SADI).

Recommendation paragraph 14.29

15.83 Given the reach back capacity of primes and their ability to tap into research and development of US and European headquarters, the committee recommends that industry consultation start at the earliest Defence White Paper and Defence Capability Plan stage.

Senator Alan Eggleston

Chair

Navigation: Previous Page | Contents | Next Page