Coalition Senators' dissenting report
1.1Coalition Senators do not support the findings of the Government Senators’ report[1] into the establishment of the Defence Capability Assurance Agency (DCAA) and related oversight bodies.
1.2Coalition Senators recognise that there are many people within the AustralianDefence Force (ADF) and Capability Acquisition and Sustainment Group (CASG) that work diligently to enable capability acquisition, capability upgrades and successful deployment of capabilities to meet integrated, multidomain requirements and integrated force missions. The evidence provided to the Committee demonstrates, however, that this outcome is not consistent and that in spite of Defence policy, decisions are too often being made in the absence of timely and accurate assessments of risk. This has been shown to have real consequences for safety, mission capability and the effective use of taxpayers’ dollars.
1.3The Defence submission stated that:
T&E [Test and Evaluation] is a structured process to obtain objective, reliable information on whether a piece of equipment/capability can perform its intended role and is safe to operate. …. The results of T&E therefore provide impartial evidence to inform decision makers on the existence, and degree of, capability risks associated with the equipment under consideration.[2]
1.4Despite this policy position, evidence to the Committee highlighted a significant variance in culture across ADF leadership and CASG project managers in respect of T&E. This has had a flow on effect to priorities for funding, training and engagement with a T&E workforce that is often undervalued and, in some cases, actively disparaged. The Committee heard evidence of assessments of risk which were deliberately set aside or not reported to senior decision makers because they were perceived to be a threat to the achievement of a project schedule or fielding of a capability.
1.5The Government Senators’ report acknowledges that all witnesses—including Defence—recognised the need for reform and supported the four principles underpinning the bill (independence, task-specific competence, transparency and accountability).The Report, however, does not reflect the extent of concern raised by witnesses about the current state of Defence T&E capability and practice, nor the doubtful prospects for effective and sustained internal reform—given previous cycles of Defence led reform that have resulted in more process, rather than better outcomes. Two “exemplar” incidents¬—exemplar in the negative sense—were brought to the Committee’s attention (discussed below) which illustrate the cumulative effect of these principles not being enforced.
1.6Finally, the Government Senators’ report downplayed the significance of the strong alignment in the views of witnesses other than Defence (paragraph 2.61), citing the small number of submissions received.Coalition Senators note that the majority of witnesses are—or have been—directly involved with DefenceT&E, including some at very senior levels. Representatives of professional organisations indicated that they had engaged broadly with their membership base which included professionals working within government, academia and industry – both prime contractors and SMEs. The Committee received evidence that was expert, current, and informed by a breadth of practice across relevant stakeholders in the Australian context. All stakeholders[3] who appeared as witnesses at the public and in-camera hearings (bar Defence) supported the passage of the bill (with amendments).
1.7Coalition Senators recommend the bill and Explanatory Memorandum (EM) be amended to:
-remove the establishment of a Parliamentary Joint Committee on Defence, and require the Government to include oversight of the DCAA by the Joint Statutory Committee on Defence which was announced in response to the JSCFADT report on the inquiry into international armed conflict decision making (Aug 23);
-require the DCAA to benchmark T&E qualification and practice standards with relevant professional associations and peer organisations (such as ITEA and DOT&E) to ensure that they reflect international best practice and do not advantage any particular industry stakeholder;
-clarify that the DCAA as a Commonwealth Agency will be responsible for the award of contracts for T&E services, and require the DCAA Board to engage with industry stakeholders during the establishment of the DCAA, and at least annually thereafter, to ensure that there is an effective mechanism to provide open and fair competition; and
-the amended bill be passed.
1.8There was unanimous support for the four principles underpinning the bill. Coalition Senators will therefore use these principles as a framework to highlight why the evidence demands a response that is more effective than the past attempts by Defence at internal reform. The four principles in summary are:
Independence. The risk identification function must be independent so that assessment is made without bias or influence (intended or unintended). Independence also ensures that the assessor of risk has a voice (NB not a veto) that is heard at each decision-making level of the capability life cycle.
Task-specific competence. Government must ensure staff with the right skills are employed to identify and manage risk. Competence is a matrix of qualifications and experience that are directly relevant to the task at hand.
Transparency. Previous inquiries highlight that risk assessors working within Defence face various barriers (individual or organisational) that influence whether decision-makers actually consider their assessments. Given the costs and national security implications, the taxpayer deserves to know that decisions are being made on the basis of accurate understanding of risk.
Accountability. The DCAA will be underpinned by an audited and enforceable requirement that Defence engages the Agency to evaluate risk across the capability life cycle. DCAA reports are to be specifically included in briefs provided to project managers, assurance bodies, Defence Investment Committee and the National Security Committee of Cabinet.
1.9The Committee heard evidence of the detrimental impact of the lack of independence of T&E in Defence which has had an impact on safety, mission achievement, retention of highly trained T&E personnel. At the most basic level, independence requires that risk assessment is made without bias or influence. Representatives of the International Test and Evaluation Association’s (ITEA) Southern Cross Chapter gave evidence that:
…every test and evaluation practitioner who has had a long career and has been involved in test programs of significance has been subjected to those types of pressures. It's a sad fact that, when the community comes together at association dinners and the like, the stories always turn around to 'when I tested something and somebody didn't like the answer, and the consequences that I experienced on a personal basis.[4]
1.10Other witnesses gave similar evidence including personal experiences of inappropriate pressure and influence after reporting of risk which disrupted the intended schedule and direction of a project.
1.11Independence also means greater consistency in the application of T&E, particularly in the early stages of a project. Mr Mark Schweikert, with extensive professional expertise working with Defence, pointed to a lack of T&E input in the requirement-setting stage of Defence’s T&E process.[5] The Committee heard evidence from other witness of how this lack of early engagement is often the result of the discretion afforded to capability sponsors and project managers. In the case one of the aforementioned exemplar projects, the lack of early engagement set the project on a track to deliver a capability that would not be able to perform the majority of “essential” missions and would have risks to certification affecting safety.[6]
1.12Witnesses appearing in public and in-camera stated that the aspect of legislated independence was one of the key strengths of the bill. Dr Joiner stated:
I think the bill fulfils a really important function, and I hit that in the submission, in that it gives independent authority to capability assurance, and I think that's desperately needed. Our AUKUS partners have already implemented this.[7]
1.13The ITEA Southern Cross Chapter stated that:
We are, however, firmly of the opinion that there should be adequate independence between people and agencies performing test and evaluation and those who are directly responsible for the cost schedule and acceptance of acquisitions. We believe this is necessary to protect against undue pressure and influence.[8]
1.14Mr Schweikert stated that:
An independent agency, I think, would be fantastic to have and would give us the culture and efficiencies that we require in delivering capability.[9]
1.15Coalition Senators believe that risk assessment requires statutory independence.
1.16The Committee heard evidence of a great disparity between levels of professional qualifications, standard of follow-up on-the-job training and levels of supervisory experience retained across the land, air, maritime, space and cyber domains. Defence gave evidence that full financial year 2022‒23 reporting showed that, of the 466 staff employed in T&E duties, 58per cent were fully competent and an additional 24 per cent were on track to be qualified following the completion of on-the-job training. Other witnesses highlighted concerns with the assessment of underpinning this reporting.
1.17Dr Joiner gave evidence that:
At the moment, the leading university has delivered 1,200 people in the last seven years to do the introduction to tests and evaluation as part of the Masters of Systems Engineering. That constitutes, if you took it as a dedicated course, four weeks of training. That's it. We offer one elective, which is in test design, which is one of the competencies that the UnitedStates implemented 13 years ago, and I have had 120 students do that in the last five years. It's a bare trickle. That would extend you to twomonths.[10]
1.18Mr Grabert representing the ITEA Southern Cross Chapter stated that:
Yes, we do believe there are gaps [in availability] to Australian practitioners in regards to T&E training. There are offerings at the very introductory level that have been around for some time, and Defence has recently added to that, which we welcome. As you say, at the high end there are the aerospace long courses. I'm a product of that system myself. But there is a big gap in the middle, particularly for people who will be testing complex adaptive systems. There's very little available to Australian practitioners to be trained and educated here in Australia.[11]
1.19Dr Joiner identified further aspects of the gap in training saying:
All of those aspects that I spoke about—software testing, cybersecurity testing, agile testing, modelling and simulation and the ability to test using model-based systems engineering—would fill out the remaining nine to 10months of a one-year qualification and lead to, with those rules, some serious risk exploration and testing in some of our capabilities, in my view. So that's why I think it's inadequate. Systems have developed and technology has advanced, and our training hasn't matched that.[12]
1.20Witnesses saw the creation of the DCAA as a way to remedy training shortfalls through a structured gap analysis, adequate long-term funding for training and incentive for personnel to choose this pathway.
1.21Mr Grabert stated that:
…the ITEA Southern Cross Chapter sees this bill as an opportunity for professionalisation of the T&E workforce, improved career opportunities, longevity for people wishing to specialise in T&E, improved access to T&E training for Australian professionals, and the writing into law of the requirement for independence and protection from undue influence for people tasked with performing the test and evaluation of weapon systems.[13]
1.22Coalition Senators are concerned by evidence that Defence does not invest in adequate training for personnel conducting risk assessment via T&E, and has been willing to progress capability procurement or upgrades without qualified staff completing competent assessments of technical risk.[14] Dr Joiner highlighted the shortfall in the reliance on a DSTG assessment of technology risk stating:
The OneSKY project was an example where they came forward and said 90per cent of the capability is off-the-shelf mature. The Defence scientists said that the technology there was not advanced or challenging, and so they gave it low technical risk. Myself, sitting there on the Defence capability gate review board for four years, said: 'Yes, but it's all independent pieces that have to be integrated. I see that as quite challenging, and I would like to see a pilot, whether that's modelled or simulation or actually being used somewhere with low air traffic space density. Just build it and show us that it's going to work before we commit to rolling it out across the whole of Australia.' That was rejected as it would cause delay, and we were going to have a capability within three years, for a Defence bill of XXX million dollars. Here we are, seven years later, two hundred per cent over budget, or even three times, on the Defence bill side, and we've seen it before.[15]
1.23A high attrition rate of experienced T&E practitioners means that many positions, even notionally supervisory roles, are now filled with qualified but inexperienced practitioners. The reported mitigation approach of reducing tasking to allow for safe levels of supervision while personnel gain experience may be necessary under the current construct, but it undermines the stated need for Defence to rapidly acquire new capability in response to the heightened strategic threat environment. The DCAA would provide a framework to boost capacity for tasking, training and supervision restoring Defence’s ability to respond to strategic threats while restoring the resilience of its T&E workforce.
1.24The Committee received evidence of multiple projects where assessments of risk that were seen as potentially disrupting a schedule or preferred direction for a project were not transparently reported to senior decision makers.
1.25When Mr Grabert from ITEA was asked about transparency he stated:
The answer would be yes, there are members of the chapter that have that experience. The questions of transparency and independence are, I believe, almost equal in importance. They also go hand in hand. Being independent allows you to be transparent, so the two aren't mutually exclusive. So the answer is: yes, our members have experienced times where their conclusions and recommendations have been interpreted differently from how they were intended to be interpreted. Yes, we see independence and transparency—obviously, transparency within the context of national security—as critical to testing and evaluation.[16]
1.26Coalition Senators are concerned at two Defence statements with regard to transparency.
1.27Firstly, that transparency will be addressed by the creation of a new process/tool known as the “capability risk assessment mechanism”, an independent channel to report T&E outcomes to the investment committee. Evidence to the Committee makes it clear that staff at various levels within Defence have the discretion to decide the extent to they will engage professional T&E and what they will do with any assessments of risk that are provided. A reporting tool is only as effective as the quality of the information entered into it.
1.28Secondly, Brigadier D'Arcy gave evidence to the Committee that:
The role of the capability manager to balance the risks identified from a number of sources, including test and evaluation, as part of forming the recommendation to government through the Investment Committee.[17]
1.29In principle this is consistent with the intent of the bill which is to ensure that capability managers have a timely and accurate assessment of risk on which to base their decisions and advice to government. This assumes, however, that in “balancing”, they engage with the identified risk to: eliminate it; mitigate it; or accept it transparently report the likely consequences of the risk being realised. Evidence to the Committee demonstrates that a culture has developed within some parts of Defence, that it is within the remit of a capability/project manager to choose to set aside an objective assessment of risk—which they find inconvenient—and substitute it with a subjective opinion of lower risk from personnel who have no T&E qualifications or authority to make such an assessment. This practice, if allowed to continue, will render the “capability risk assessment mechanism” worse than ineffective—potentially it will aid deception by adding a veneer of transparency to a system that appears incapable of ensuring that decision makers at all levels have visibility of legitimate assessments of risk. Defence is not alone in seeing this drift from sound capability management. The following well-documented failures of engineering and capability management are pertinent.
1.30The recent Boeing Max tragedy, where a Congressional inquiry found that:
The investigative report outlines the intense pressure Boeing employees were under to keep costs down and the plane's development on schedule, as the company rushed development of the 737 Max to compete with the new Airbus A320neo”. Democratic Rep. Rick Larsen of Washington, said “misguided priorities of senior management drove a number of troubling decisions that put profits ahead of safety.”[18]
1.31Another troubling finding is what the report calls the “culture of concealment.” It documents several critical instances in which Boeing withheld crucial information from the FAA, its customers and 737 Max pilots.
1.32The US Columbia Accident Investigation Board (CAIB)[19] which found that:
…pressure to stay on an existing launch schedule and inefficient resources to do so are considered to be contributing factors of the STS-107 accident. CAIB also found that NASA’s safety program falls short of achieving the level of safety necessary for the shuttle program. As a result, CAIB recommended that the safety system at NASA be restructured to include an “independent Technical Engineering Authority that is responsible for technical requirements and waivers to them and will build a disciplined, systematic approach to identifying, analyzing, and controlling hazards throughout the life cycle of the Shuttle System.CAIB emphasized that the independent authority “should have no connection to or responsibility for schedule or program cost.”[20]
1.33The Committee received evidence that there is little to no accountability for actions of capability/project managers who choose to not engage T&E early (if at all in a project) or fail to respond transparently and effectively to assessments of risk by an authorised T&E entity, even if risks subsequently are realised in terms of safety, mission accomplishment or cost.
1.34Brigadier D'Arcy stated during the hearing that:
Our position is that every project that goes through the internal Defence decision-making process has scrutiny placed on its test and evaluation master plan. That is a critical element for the respective committees to determine that the project [inaudible] has considered it and that the project has a robust plan in place, depending on the type and level of T&E that needs to be required for the particular project's circumstances. We acknowledge that, on very rare occasions, there might be some exceptions to that rule, but, in terms of the policy itself, the delivery heads are responsible for managing, coordinating and enabling the following on behalf of the capability manager. The first is sufficient preview, test and evaluation to support requirement development; then it's sufficient preview, test and evaluation to ensure informed, risk-appropriate solicitation decisions; sufficient developmental test and evaluations and acceptance test and evaluation to verify agreed functions, limits, requirements and certification requirements; and then, finally, sufficient support to operational test and evaluation to finalise the program and project product validation of capabilities to the specified operational concept. That is the position in our test and evaluation manual, and that is what we apply when we judge submissions that pass through the internal process. There's also a direct auditing function, which is led by the directorate into a number of critical projects where that risk might be higher and which are subject to a dedicated audit, if you like, of their T&E planning.[21]
1.35This summary appears like a robust process but Coalition Senators remain concerned by the disconnect between this stated policy, which relies on internal scrutiny, and actual practice highlighted by evidence of recent projects. The Inspector-General of Defence Capability Assurance (IGDCA) proposed by the bill would provide a greater level of assurance to the Parliament and the taxpayer that decisions are in fact being informed by accurate and timely assessments of risk.
1.36There were several other issues raised in submissions, most of which were canvassed during hearings. These include:
1.37Concerns were raised that the DCAA would create workforce pressures by duplicating functions currently undertaken by Defence personnel. Evidence provided during the hearings allayed these concerns. MrLaFranchi stated:
Again, this is where you're looking at aggregating an existing workforce of limited proportions and parameters. By aggregating them you certainly don't set up an arrangement of duplication. The legislation in fact talks of putting industry, as well as embedding operational Defence personnel, into that structure so that you have coordination” and “Again I would see that you would transfer functions that currently exist within Defence across to the agency. The equivalent here is the regulatory function in the AUKUS program where you have a separate regulator supporting the AustralianSubmarine Agency. There's no discussion or hint there that there would be duplication, rather it's focusing and placing appropriate people in appropriate places for appropriate purposes.[22]
1.38Mr Grabert ITEA likewise stated:
In regard to would the new agencies take people away from their current roles, we're of the opinion that if the bill went through there would be a form of harmonisation against the existing workforce. We note that VCDF group is already pursuing an agenda of increased T&E governance, including standing up new roles in that organisation. So we would be of the opinion that if the bill went through there would be some sort of harmonisation between the agencies that are being set up at the moment by VCDF and what would be brought by the bill.[23]
1.39In fact, the bill is quite specific that there would not be duplication. The DCAA would take over coordination and management roles for T&E while strengthening and growing the internal Defence T&E workforce. Where new functions are required, Major-General King stated:
On the way in which the governance model is implemented, my view would be that the workforce required to do that, compared between a DCAA model and an internal Defence model, would be similar.[24]
1.40The stated concern by Defence that the bill’s proposed industry partner was “based on the United Kingdom’s experience with their Long Term Partnering Arrangement (LTPA)”[25] was highlighted during the hearing to be an inaccurate assertion, and that the DCAA in fact was a Commonwealth entity which would engage multiple industry stakeholders to provide T&E where ADF/Defence capability or capacity were insufficient.
1.41The long-term industry partner would primarily be an input to the governance function of DCAA, not T&E delivery. Brigadier D'Arcy stated:
I certainly acknowledge that explanation. The concern around governance across a commercial entity setting those standards et cetera was where that concern was born from originally. But I acknowledge the importance here of Commonwealth oversight of that in order to prevent the issues we've just described in our response and as provided by Group Captain Young.[26]
1.42Mr Grabert ITEA stated:
No. We had no reason to object to that, to an industry partner providing that governance function. We quite deliberately, as a not for profit, decided not to make any recommendations that might benefit any particular organisation or not. But having a commercial partner operating within the agency and having that commercial partner subject to the to the board and the director with those functions, making sure that the agency was operating in the best interest of the Commonwealth, we had no reason to object to that.[27]
1.43Coalition Senators recognise that there has been concern expressed about the potential for a monopoly situation and support amendment to the bill to clarify the role of the DCAA as a Commonwealth Agency to control contracting and ensure that standards set by the DCAA regulator reflect international best practice and do not advantage or disadvantage any particular industry stakeholder.
1.44Subsequent to the tabling of the bill, the government’s response to the JSCFADT report on the inquiry into international armed conflict decision making, released in August of this year, included the agreement to establish a Joint Statutory Committee on Defence (JSCD). This proposed committee will fulfil a similar range of oversight to the bill’s proposed PJCD. On this basis, Coalition Senator’s see no reason to proceed with legislation to establish the PJCD but support an amendment to the bill to ensure that the proposed JSCD will have oversight of the DCAA.
1.45The stated concern that the DCAA “applies only to acquisition-related T&E and would not directly contribute to the conduct of OT&E and FLOT&E”[28] was acknowledged during the hearing by Defence to be inaccurate. The EM is specific that the DCAA is engaged in T&E across the capability life cycle.
1.46The stated concern that “implementation of the bill may undermine accountable officers’ ability to assure that acquired systems and capabilities meet the integrated, multidomain requirements and integrated force missions in accordance with the intent of DSR and Australia’s strategic circumstances”[29] contradicts Defence’s own submission which acknowledges that “[t]he results of T&E therefore provide impartial evidence to inform decision makers on the existence, and degree of, capability risks associated with the equipment under consideration” and that Defence’s approach to T&E “has sometimes failed to live up to that commitment.”[30]
1.47The DCAA provides the war-fighter, government and taxpayer assurance that these accountable officers do in fact consistently receive the “impartial evidence” they require to make their decisions. Evidence from a range of witnesses, contrary to the position taken by Defence, made it clear to CoalitionSenators that far from “undermining” accountable officers, the DCAA would in fact enable these officers to fulfil their obligations in a more timely and effective manner.
1.48Witnesses were asked about the potential for Defence to drive effective and sustained reform in T&E. In response to a question as to whether Defence has actually accepted recommendations from previous Australian National Audit Office (ANAO) and Parliamentary or is just repeating the cycle of process reform as opposed to outcomes, Mr Schweikert stated “I would suggest that we're just repeating that.”[31]
1.49Dr Joiner stated that:
Consistently, Defence, over the last 20 years, has failed to implement real reform in test and evaluation. The Australian National Audit Office called for an equivalent to the US for independent assurance of test and evaluation back in 2001.[32]
1.50Dr Joiner went on to describe the areas responsible for T&E in Australia as “chronically underfunded’ and with minimal policy and test infrastructure’ despite two decades of “reform”[33] following significant ANAO reports (2002and 2015), Defence’s 2008 Roadmap for Test and Evaluation, and a 2012 Senate Inquiry into Defence Procurement which made specific recommendations pertaining to T&E. Of note, paragraphs 2.6 and 2.7 from chapter 2 of the Senate Foreign Affairs, Defence and Trade References Committee, Commonwealth of Australia, Procurement procedures for Defence capital projects states:
Despite early warnings from subject matters experts, the project proceeded without any effective risk management. In early 2008, briefs prepared for senior Defence personnel outlined a series of inadequacies in the Super Seasprite capability, some of which had been identified as early as 1998. These matters had also been covered in the 2005 Deficiency Review which, according to an ANAO audit report, had 'effectively recommended that the Project be cancelled'. The ANAO concluded that the Project was:
…high risk from the outset and the scale of these risks escalated rapidly in the early stages and remained high prior to the Government's decision to cancel the Project. The issues encountered were fundamental to the Project's success and were not overcome during the 12-year life of the Project. From an accountability perspective, this leads to a question regarding how the Project was allowed to continue for so long…Factors contributing to this outcome include a degree of optimism surrounding the ability to achieve outcomes, a reluctance to make firm decisions based on the information available; and a lack of visibility of information to decision makers.
This failed project provides a raft of lessons for any future project. It especially drives home the need not only for the adequate resourcing of early T&E activities but to ensure that the advice from subject matter experts is communicated to key decision-makers, who are able to comprehend and heed such advice and take decisive action—that is take responsibility.[34]
1.51The very clear articulation of problems in these paragraphs and others, resulted in a reworking of Defence T&E policy and process, but as this inquiry has found, the change was neither effective or sustained. In-camera evidence and answers to Question on Notice reinforced the assessment that the capacity for internal reform is low. Coalition Senators have little confidence that internal reform of T&E will result in substantive change to independence, task-specific competence, transparency or accountability, and therefore strongly support the passage of the bill to establish the DCAA and IGDCA.
1.52The Committee received evidence regarding two projects that exemplify the potential for disastrous outcomes when the basic principles of effective risk assessment are not implemented and enforced. The key points of concern included:
1.53Evidence to the Committee indicated that the view of capability and project staff was that “both of the equipment options available to the project would be suitable”. Consequently, capability management staff developed requirements for a project without engaging the relevant competent T&E organisation to ensure that they would actually meet the operational needs of the war fighter and the safety considerations of certification. After tenders were received, the relevant T&E organisation was finally engaged to provided an assessment of the performance of the proposed equipment solution. The T&E personnel provided assessments against both the “approved” requirements and against the requirements that would actually meet the mission and certification requirements. Their professional assessment, peer reviewed within an authorised test organisation using internationally recognised analysis techniques, found that the proposed equipment solution failed to meet the performance requirements for the majority of “essential” mission profiles and had safety risks.
1.54Rather than report this assessment to superiors and then respond to the identified risks, project and capability staff sought to substitute the professional, objective assessment of risk by seeking the subjective assessment of operational personnel who were not T&E qualified. Evidence to the Committee was that this “independent” analysis used a flawed method and non-standard assumptions regarding mission and risk factors to reach an assessment that the proposed solution would meet operational requirements. It is an indictment on the attitude of Defence regarding the value of competent T&E that the project passed the next stage of review. Evidence to the Committee was that documentation to the reviewers acknowledged there were two assessments of risk and that the project had opted to accept the informal assessment in preference to one coming from an authorised test organisation.
1.55A similar pattern of behaviour was reported to the Committee in a separate, in-service upgrade to a system. The professional assessment of an UNACCEPTABLE risk to safety arising from a technical defect was reported by the relevant T&E organisation. This assessment took into account the full range of environmental and operational factors the mission could entail. The capability management staff responded by seeking an alternate assessment. The relevant test organisation was asked to review the planning for this “operational” assessment. The plan included environmental and operational limitations as risk mitigation for the people conducting the assessment. The operational assessment was therefore conducted in a benign environment which masked the impact of the technical deficiency.The upgraded system was subsequently released to operational service but with none of the operational and environmental mitigations carried forward, exposing personnel to a safety risk which had been assessed as UNACCEPTABLE when operating in the challenging conditions known to be required by some missions. Evidence to the Committee was that capability management staff believed they had mitigated any residual risk by proposing the use of a secondary system as a backup, despite the fact that it was not certified by the manufacturer nor tested by the ADF for that purpose. No action was apparently taken in response to concerns subsequently raised by staff at the relevant T&E organisation. It was not clear to the Committee how the capability management process approved the release to unrestricted service in the light of the original peer reviewed assessment conducted by an approved, regulated test organisation.
1.56Coalition Senators believe that the above paragraphs 1.28, and 1.32 in particular, are likely pertinent in driving a culture where consideration of other factors is seen as a valid reason to dismiss an assessed risk. This stands in contrast with the long-accepted obligation for a capability manager to: eliminate the cause of risk (normally by rectifying the deficiency); if that is not feasible then treat the risk with valid measures to minimise the probability of occurrence or mitigate the consequences; if the operational need dictates, accept the risk but transparently acknowledge it and ensure that both senior decision makers and operational staff understand the likely consequences if the risk is realised.
1.57The Committee heard evidence that project staff in the first case, subsequently conspired to exclude T&E staff from other technical project assessments and the officer in charge of the T&E organisation came under pressure from superior officers. Coalition Senators find this conduct reprehensible and directly in conflict with the Defence evidence to the Committee.
1.58Coalition Senators note that these were recent projects, yet the failures in respect to the effective use of qualified T&E professionals to identify and respond to technical risk are almost identical to the 2002 and 2015 ANAO reports into defence T&E, despite as noted by Dr Joiner, two decades of internally led and resourced “reform”.
1.59Noting the deteriorating strategic circumstances detailed by the DSU 2020 and DSR 2023 which require a rapid and effective investment in new and complex military capability, Coalition Senators find the evidence of expert and experienced T&E practitioners compelling regarding the need for an effective T&E capability. We also note evidence of good intentions for reform but a track record of poor delivery and no sustained improvement at an organisational level. Coalition Senators support the passage of the bill as amended by the recommendations of this dissenting report.
Senator Claire ChandlerSenator the Hon David Fawcett
Deputy Chair
Footnotes
[1]Convention would see this report titled the Majority Report however in this case the Chair used his casting vote to table a report which was not supported by half of the voting members of the Committee, being the Coalition and cross-bench Senators. The Report only reflects the views of Government Senators and is referred to in this manner for clarity.
[2]Department of Defence, Submission 10, p. 1.
[3]In the case of ITEA, they phrased their position on a number of topics as having “no reason to object” including in respect to the establishment of the DCAA as proposed by the bill.
[4]Mr Timothy Grabert, Committee Member, Southern Cross Chapter, International Test and Evaluation Association, Committee Hansard, Adelaide, 28 September 2023, p. 9.
[5]Mr Schweikert lodged a submission to the inquiry, Number 2, and appeared at the Thursday, 14 September 2023 public hearing in Canberra.
[6]This project did not proceed.
[7]Dr Keith Joiner, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 1.
[8]Mr Timothy Grabert, Committee Hansard, Adelaide, 28 September 2023, p. 8.
[9]Mr Mark Schweikert, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 10.
[10]Dr Keith Joiner, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 2.
[11]Mr Timothy Grabert, Committee Hansard, Adelaide, 28 September 2023, p. 10.
[12]Dr Keith Joiner, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 2.
[13]Mr Timothy Grabert, Committee Hansard, Adelaide, 28 September 2023, p. 8.
[14]Technical risk goes to not just the technology, but the integration of a system of systems, the ability to meet operational and certification requirements. It is quite distinct to an assessment of technology in isolation.
[15]Dr Keith Joiner, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 4.
[16]Mr Timothy Grabert, Committee Hansard, Adelaide, 28 September 2023, p. 10.
[17]Brigadier Steve D’Arcy, Acting Head, Force Integration Division, Vice Chief of the Defence Force Group, Department of Defence, Committee Hansard, Adelaide, 28 September 2023, p. 16.
[18]‘Congressional Inquiry Faults Boeing And FAA Failures For Deadly 737 Max Plane Crashed’, NPR webpage, 16September 2020, https://www.npr.org/2020/09/16/913426448/congressional-inquiry-faults-boeing-and-faa-failures-for-deadly-737-max-plane-cr, (accessed 27 November 2023).
[19]‘Columbia Accident Investigation Board (CAIB) Synopsis’, Jennifer Troxell, NASA History Office, NASA webpage, https://history.nasa.gov/columbia/Troxell/Columbia%20Web%20Site/CAIB/CAIB%20Synopsis.htm, (accessed 27 November 2023).
[20]‘Columbia Accident Investigation Board (CAIB) Synopsis’, Jennifer Troxell, NASA History Office, NASA webpage, (accessed 27 November 2023).
[21]Brigadier Steve D’Arcy, Committee Hansard, Adelaide, 28 September 2023, pp. 17 – 18.
[22]Mr Peter La Franchi, Executive General Manager, Future Business, Nova Systems Australia and New Zealand, Committee Hansard, Adelaide, 28 September 2023, p. 4.
[23]Mr Timothy Grabert, Committee Hansard, Adelaide, 28 September 2023, p. 11.
[24]Major General Jeremy King, Head, Joint Aviation Systems Division, Capability Acquisition and Sustainment Group, Department of Defence, Committee Hansard, Adelaide, 28 September 2023, p. 18.
[25]Department of Defence, Submission 10, p., 5.
[26]Brigadier Steve D’Arcy, Committee Hansard, Adelaide, 28 September 2023, p. 20.
[27]Mr Timothy Grabert, Committee Hansard, Adelaide, 28 September 2023, p. 12.
[28]Department of Defence, Submission 10, p., 5.
[29]Department of Defence, Submission 10, p., 5.
[30]Department of Defence, Submission 10, p., 2.
[31]Mr Mark Schweikert, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 6.
[32]Dr Keith Joiner, private capacity, Committee Hansard, Canberra, 14 September 2023, p. 2.
[33]Dr Keith Joiner, Submission 1, p. 2
[34]Senate Foreign Affairs, Defence and Trade References Committee, Commonwealth of Australia,Procurement procedures for Defence capital projects,August 2012, pp. 16 – 17.
Inquiry into Defence Capability Assurance and Oversight Bill 2023.
Senate
House of Representatives
Get informed
Bills
Committees
Get involved
Visit Parliament
Website features
Parliamentary Departments