Performance of aircraft in testing

Introduction

3.1        This chapter discusses the concerns raised in evidence regarding the performance of the F-35 in testing, including the aircraft's:

• manoeuvrability and flight capabilities;
• stealth capabilities;
• mission systems;
• mission data loads and Autonomic Logistics Information System; and
• escape system.

3.2        The chapter also considers concerns regarding the performance and accuracy of the Verification Simulator (VSim).

Classified data

3.3        It is important to note that some submissions emphasised that it is impossible to accurately understand and critique the capabilities of the F-35 without access to detailed classified performance data.[1] The Australian Strategic Policy Institute (ASPI) told the committee that it does not draw on classified information for its public discussion papers and as a result 'have found it difficult to draw confident conclusions one way or the other from publicly available information'.[2] Mr James Hicks cautioned that 'the F-35's effectiveness in air to air combat, air to ground and ship killing roles cannot be evaluated accurately without access to classified information'. Mr Hicks also warned the committee that 'comparisons made of the F-35 with other aircraft are frequently nonsensical'.[3]

Testing and evaluation

3.4        Defence informed the committee that 'the significant capability of the F-35 means the complexity of the test and evaluation process cannot be underestimated'. Defence advised that the test and evaluation (T&E) program is currently employing a 'fly-fix-fly' approach, but warned that while this methodology is appropriate it has 'introduced some schedule risk to the program':

The F-35 test and evaluation program is currently employing a “fly-fix-fly” approach. While this methodology is appropriate to the complexity of the F-35 software, it has introduced some schedule risk to the program. The US Department of Defense acknowledged this risk in 2014 and curtailed and rationalised the F-35 test and evaluation program to better focus resources on the testing of the final software to be delivered under the System Development and Demonstration phase in 2017. Notably, in 2015, the program achieved all planned test points, some 1,374 test flights and 9,582 test points.

Given the complexity of the F-35 software it would be unrealistic to assume that problems will not be encountered during test but the measure of a mature program is the ability to effectively prioritise and resolve these issues. The Joint Program Office continues to demonstrate that the safety of the F-35 Program and the delivery of critical warfighting capability to be delivered in the initial software by the end of the System Development and Demonstration phase will not be compromised. Similarly, it is unrealistic to expect complex programs to achieve 100 percent of requirements, but it can reasonably be expected that some lower priority functionality may be deferred to a later phase.[4]

2015 Operational Test and Evaluation Report

3.5        The United States Office of the Secretary of Defense, Director of Operational Test and Evaluation (DOT&E), released the 2015 DOT&E report on the F-35 to Congress on 2 February 2016. Defence advised the committee that the challenges raised in the report 'are well known and being managed by the Joint Program Office, Prime Contractors, and Partner nations' and cautioned that 'the report must be interpreted for the Australian program', noting that:

The Joint Program Office has acknowledged the schedule risk associated with problems identified during test and evaluation. As reflected in the Director of Operational Test and Evaluation report, most remaining development risk is in software. The Joint Program Office believes that schedule margin exists to develop further software releases, if required, without compromising capability requirements.

Defence believes that this schedule risk is clearly evident and that a delay of between six-12 months to the completion of the System Development and Demonstration phase is likely. Due to the significant gap between the Australian initial operational capability milestone being four years later than the US Air Force initial operational capability (2016) milestone and two years later than the US Navy initial operational capability (2018), Defence assesses that the Australian Program remains on track to achieve initial operational capability in 2020, fully cognisant of the issues raised in the report.[5]

3.6        Defence assured the committee that it is also focused on monitoring other strategic risks, such as 'risks associated with integration of the capability into the Australian environment including the global support solution, maintenance, and pilot training, information systems and the ability to develop mission data files'.[6]

Manoeuvrability and flight capabilities

3.7        A number of submissions raised concerns regarding the F-35's manoeuvrability and flight capabilities.[7] Mr Peter Goon, Head of Test and Evaluation for Air Power Australia, told the committee that the F-35's flight capabilities do not exceed those of the F-16 and F/A-18 and questioned whether this would adequately serve Australia's future needs:

...the fundamental problem with the JSF goes to the question: why would you specify your new air combat capability to be comparable with what you have already got? That is what the JSFs have been from day one. You look at the specification, you look at the language of the marketing and the representations made in the formative years of this program; they kept comparing it to a F16 and a F-18. You ask why? That is what you have already got. That is not what is going to be out there in 10, 20, 30 years' time. People were driving along by looking in the rear-vision mirror rather than looking ahead and looking into the future and seeing what the reference threats are going to be—this is the net assessment process referred to earlier—and what you need to do to balance, what you need to do to counter those reference threats so you do maintain balance in global power or, in our case, balance in regional power.[8]

3.8        Mr David Archibald was critical of the F-35's aerodynamic performance, asserting that it is a 'subsonic aircraft in both air intercept and ground attack missions' which cannot achieve supercruise. He advised the committee that the F-35 has very low instantaneous and sustained turn rates, low acceleration, and limited combat endurance:

The F-35A has combat weight of 18.3 tonnes, a wing loading of 428 kg/m2, thrust-to-weight ratio of 1.07 and span loading of 1.75 tonnes/m. Wing sweep is 34°, and the engine has a power-to-frontal area ratio of 17.9 N/cm2. As a result, the F-35 has very low instantaneous and sustained turn rates (less than half of the F-22's sustained turn rate, or ~11° per second) as well as low acceleration, while its weight harms the transient performance. The F-35's inefficient aerodynamics and inefficient power plant also limits combat endurance despite an excellent fuel fraction of 0.38. The F-35 has a specific fuel consumption of 0.9 lb/lb/hour versus 0.75 for other advanced combat jet engines.[9]

3.9        However, Mr Hicks cautioned the committee against focusing on manoeuvrability or flight performance when considering the merits of the F-35, noting that the usefulness of manoeuvrability and speed has been drastically reduced since the 1960s. Mr Hicks explained that modern air-to-air missiles are now highly intelligent and have significant advantages over manned fighter aircraft when it comes to speed and manoeuvrability, limiting the ability of any manned aircraft to win a tight manoeuvring 'dog fight':

Since the 1990s, air to air missiles have become more intelligent and even better equipped. Meanwhile, fighter aircraft have not exceeded their 9-10G maximum emergency manoeuvre – and can't, as they retain a human pilot who would be knocked unconscious by a tighter turn. Obviously missiles have no such limitation.

As a result, the need to out manoeuvre one's opponent has all but vanished from air to air combat, based on statistical analysis of actual air to air kills. Both heat seeking and radar guided missiles now have a kill ratio of over 50% – generally speaking if someone launches a missile at you, most likely you will be shot down by it. Furthermore, the missiles themselves have gotten a whole lot more intelligent, being able to more or less identify "that's a flare, that's chaff, and THIS is my target", as well as plot an intelligent intercept on a manoeuvring target.[10]

3.10      The Sir Richard Williams Foundation (SRWF) argued that the F-35 is highly survivable and lethal when confronting advanced threats. SRWF emphasised that the F-35 was designed as a multi-role aircraft and that its manoeuvrability and flight performance is appropriate for this purpose. It should not be compared to the manoeuvrability and flight performance of high altitude air-to-air combat aircraft such as the F-22:

The F-35's unequalled situational awareness, combined with advanced weapons and countermeasures, makes the F-35 highly survivable and lethal when confronting advanced threats in the air, land and sea battle space. It is not designed to perform like the F-22, a high altitude air-to-air combat aircraft. It is a multi-role aircraft designed to avoid Within Visual Range operations with acceptable turn performance comparable to the F-15E and F/A-18.[11]

3.11      Defence assured the committee that it is confident in the F-35's manoeuvrability and flight performance. Defence explained that media reports regarding the manoeuvring performance of the F-35 highlight an event which occurred during the conduct of the F-35 test and evaluation during one test flight and have been taken out of context from the larger test and evaluation program.[12] Defence explained that it understands the design parameters for the F-35A and the combined effect of its fighter characteristics in the battlespace and is confident that it will meet Australia's needs:

The F-35 design itself is a product of war fighter requirements which considered the relative importance of specific fighter characteristics in the execution of the intended missions. The importance of stealth, payload, range and combat manoeuvrability, obtained through weapons, fuel and sensors being carried internally, outweighed other potential design choices.

Networked with advanced datalinks and sensors, a combat configured F-35 has the manoeuvrability, stealth and superior situational awareness to enable the engagement of air and surface targets while delaying and defeating the adversary's attack. The above characteristics of lethality, survivability, affordability and supportability define the F-35 as a fifth-generation fighter. Defence understands the design parameters for the
F-35A and the combined effect of its fighter characteristics in the battlespace, and is confident that this variant of the F-35 design will meet Australia's war fighting needs.[13]

3.12      Defence noted that Squadron Leader Andrew Jackson, an experienced pilot who has flown both the F/A-18 and the F-35A, found the F-35 to be superior to the F/A-18, stating that 'in [his] experience flying more than 140 hours in the F-35 so far, it is better in performance and manoeuvrability than a representatively configured F/A-18 while remaining easy to fly'.[14]

Stealth capabilities

3.13      A number of submissions raised concerns regarding the F-35's stealth capabilities.[15] Mr Goon questioned the F-35's stealth performance, asserting that the survivability of the F-35 was 'defined around the ability to survive in a battlefield interdiction environment where the aircraft would be confronted by medium range and short range SAMs [Surface-to-Air-Missiles], and AAA [Anti-Aircraft Artillery] systems, assuming that hostile fighters, long range SAMs and supporting radars will have been already destroyed by the F-22 fleet'. Mr Goon asserted that the F-35's stealth performance was 'optimised around this model' but that the evolution of both radars and SAMs have differed from what was anticipated when the Joint Strike Fighter program was launched:

The JSF's stealth performance, reflected in shaping, was optimised around this model, with independent technical analyses showing that the aircraft will have viable stealth in the front sector, but much weaker stealth performance in the beam and aft sectors. The evolving market for radars and surface to air missiles has, however, taken a different turn to that anticipated when the JSF program was launched. Highly mobile long range SAMs, supported by high-power aperture radars, have been far more popular in the market than the short and medium range weapons which the JSF was defined to and built to defeat.[16]

3.14      Mr Daniel Nowlan asserted that the F-35 is not 'all aspect stealth' and that as a consequence, 'any stealth advantage the F-35 enjoys is temporary at best and already compromised'. Mr Nowlan explained that the F-35's inferior stealth capabilities are a consequence of shaping and cannot be changed:

What is even more critical is that the F-35 is not all aspect stealth. It is only optimised in the forward aspect in the X and Ku Bands. This is a critical oversight because you have no guarantees of where the radar will be in war time. In total contrast the F-22 and B2 were designed with all aspect stealth (the ability to be low observable regardless of the radar location) from the beginning which makes them much more effective because they are much harder to detect. Again this has been known in defence circles for years. Also this is something that can't be fixed. It is a consequence of the shaping of the F-35 and cannot be changed.

What all this means in plain English is that any stealth advantage the F-35 enjoys is temporary at best and is already compromised. A very stark example of this is that Russia has ordered 100 55Zh6ME Nebo M radars. This is a mobile radar that combines VHF, L-band and S-band components with data fusion for counter-stealth. It is also highly likely this radar system will be available for export.[17]

3.15      However, SRWF challenged these assertions, advising the committee that stealth has evolved beyond using radical shaping and exotic materials to give a low radar cross section:

Stealth is much more than just the traditional view of using radical shaping and exotic materials to give a low radar cross section. True low observability (LO) is designed in from the ground up in every signature of the platform, including IR, RF and the visual spectrums. LO technology also means minimising the probability of intercept of its electronic emissions while at the same time enhancing networking capabilities and situational awareness to give a pilot decision superiority.

Stealth is not about preventing detection; it's about ensuring access. True stealth means that the pilot is able to choose where to operate, when to engage or disengage, and when to be seen or not be seen. It means reducing an adversary's situational awareness to almost zero, thereby providing improved mission success and increased survivability.[18]

3.16      The Deputy Chairman of SRWF, Mr Geoffrey Brown, asserted that the stealth characteristics of the F-35 and the F-22 are 'not much different', with the only difference being the F-35 stealth coatings are 'far more maintainable than they are on the F-22'.[19] Furthermore, Mr Brown refuted assertions that the F-35 cannot support ground forces effectively, arguing that the F-35's stealth characteristics have been shown to provide effective close air support in contested environments:

...it was an interesting discussion I had with the Deputy Commandant of Marines. He talked about his legacy aeroplanes which were F-18s, and some of the scenarios they have just run in the last couple of months where they have actually run the F-35 against a high-end surface-to-air missile system—similar to what Senator Fawcett was talking about. They had two F-35s clean and two with weapons, and they were able to do close air support in a very contested environment, which is something they could not do with their legacy aeroplanes. So as far as they are concerned, the capability of the aeroplane is much better for fighting high-end conflict than the F-18 was.

...

Because the stealth characteristics of the F-35 meant that it could actually deal with the high-end SAM, and then they could go in and do close air support with the F-35.

...

That is the normal procedure. Take them out or avoid them, one of the two, and when you are doing close air support you cannot avoid it.[20]

3.17      Dr Andrew Davies, Director of the Defence and Strategy Program at the Australian Strategic Policy Institute (ASPI), advised the committee that the F-35 offers better stealth and electronic warfare capabilities than any other available aircraft.[21] Furthermore, ASPI advised that the F-35 appears to be meeting its stealth design targets:

...actual testing of the F-35 is indicating that the aircraft is meeting its stealth design targets. Program head Admiral Venlet, in his brief to US media on 21 April, when questioned about the GAO's [United States Government Accountability Office]  reference to these issues, said 'in regard to aircraft signature... we have delivered aircraft off the production line and have flown them over ranges and we have very, very good results. We don't have any worries currently that we have detected'.[22]

3.18      Lockheed Martin described the F-35's stealth capabilities as 'unprecedented in tactical fighter aviation' and stated that its 'integrated airframe design, advanced materials and other features make the F-35 virtually undetectable to enemy radar'. Lockheed Martin advised that 'extensive analysis and flight test of the survivability of the F-35 with its combination of stealth, advanced sensors, data fusion, sophisticated countermeasures, and electronic attack demonstrate conclusively its superior advantages over legacy aircraft'.[23] Mr Gary North, Vice President of Customer Requirements, Aeronautics at Lockheed Martin, assured the committee that they are very confident in the F-35's stealth capabilities:

I was a user of tactical fast jet airplanes for over 36 years. Lockheed Martin and other companies have been developing stealth for over 40 years. Very low observable stealth is one component of a platform in battle space and the ability to manoeuvre inside the battle space. We are very good at what we do. We know our airplane better than anyone. We model the threats. We are very capable in producing airplanes that are very survivable and very lethal. There have been large assertions of adversary airplanes. They are behind us in capability. Obviously, they believe stealth is very important or they would not be trying so hard to develop it. So we are very confident in the capabilities of the platform.[24]

Mission systems

3.19      The term 'mission systems' refers to the operating software, avionics, integrated electronic sensors, displays, and communications systems that collect and share data with the pilot and other friendly aircraft. Lockheed Martin asserted that the 'F-35 has the most robust communications suite of any fighter aircraft built to date. It will also be the first fighter to possess a satellite-linked communications capability that integrates beyond line-of-sight communications throughout the spectrum of missions it is tasked to perform'.[25] Lockheed Martin advised that the key elements of the mission systems software include:

• Sensor Fusion: which enables pilots to draw on information from all of their on-board sensors to create a single integrated picture of the battlefield and automatically share this information with other pilots and command and control operating centres on their network;
• Active Electronically Scanned Array (AESA) Radar: which enables pilots to engage air and ground targets at long range, while also providing situational awareness for enhanced survivability;
• Distributed Aperture System (DAS): which provides pilots with 360-degree spherical situational awareness, sending high resolution real-time imagery to the pilot's helmet from six infrared cameras mounted around the aircraft. This allows pilots to see the environment around them and to detect and track approaching aircraft from any angle. The DAS is integrated with other sensors within the aircraft, so if the F-35's radar detects something of interest, DAS's software will analyse it and make the pilot aware of potential threats. When there are multiple threats, the DAS is able to identify the highest value targets and recommend the order in which to deal with each threat. It provides missile detection and tracking; launch point detection; situational awareness infra-red search and track (IRST) and cueing; weapons support; day/night navigation; fire control capability; and precision tracking of friendly aircraft for tactical manoeuvring;
• Electro-Optical Targeting System (EOTS): which combines forward-looking infrared (FLIR) and infrared search and track (IRST) functionality to enhance the pilot's situational awareness and provide precision air-to-air and air-to-surface targeting capabilities;
• Helmet Mounted Display System: which provides pilots with 'unprecedented situational awareness' as all the information pilots need to complete their missions—airspeed, heading, altitude, targeting information and warnings—is projected on the helmet's visor, rather than on a traditional Heads-up Display on the canopy widescreen. The DAS streams real-time imagery to the helmet from six infrared cameras mounted around the aircraft, allowing pilots to "look through" the airplane and see the entire environment surrounding them. The helmet also provides pilots with infrared night vision through the use of an integrated camera, making images in total darkness look exactly like they would in daylight; and
• Communications, Navigation and Identification (CNI) Avionics System: which provides pilots with the capability of more than 27 avionics functions. The CNI uses software-defined radio technology to allow for simultaneous operation of multiple critical functions, such as identification of friend or foe, precision navigation, and various voice and data communications.[26]

3.20      A number of submissions raised concerns regarding the performance of the F-35's mission systems.[27] Mr Mills noted that the F-35's air combat capabilities are dependent on the proper functioning of the mission systems software and that the mission systems software development 'is said to be the largest project of its type in the world'. This engenders significant risks of failure:

In software development, increasing the size of the code-base presents an exponentially increasing risk of failure. For example, 'regression testing' must prove that a sub-program for one operational function does not have adverse or unforeseen consequences to other operational subprograms. Synchronising 'real time' computations across a complex multifunction platform such as the JSF aircraft is another substantial risk.[28]

3.21      United States Lieutenant General Christopher Bogdan, the Program Executive Officer for the F-35, acknowledged that there are ongoing issues with the stability of the mission systems software, but assured the committee that the issues are being identified and resolved:

Our mission systems software--the software that controls the sensors and weapons, and also provides the pilot with battlespace awareness--is a complex, sometimes tricky, and often frustrating part of the program. The current initial Block 3 software is not nearly as stable as it needs to be to support our warfighters. What I mean by this is that about once every four to four-and-a-half hours of flight time the radar or one of the other sensors has to be reset. Our goal is to reduce this phenomenon to less than once every eight hours. In order to ensure that the software demonstrates the needed levels of stability, the government has launched an in-depth look at the software stability—called a Red Team—to help understand the causes and solutions to this problem. I believe that by the May-June time frame of this year we will have this issue resolved.[29]

3.22      Defence advised the committee that there is schedule risk associated with the completion of the test and evaluation program and incorporation of fixes to meet the scheduled completion of the System and Development and Demonstration phase by the end of 2017:

The F-35 Program is executing the System Development and Demonstration phase through a developmental test and evaluation program, which evaluates the aircraft and supporting systems, leading to acceptance of these systems. There is schedule risk associated with the completion of the test and evaluation program and incorporation of fixes to meet the scheduled completion of the System Development and Demonstration phase by the end of 2017. The completion of test and evaluation is a critical precursor to the conduct of operational test and evaluation, which aims to test these systems in an operationally representative environment.[30]

3.23      Furthermore, the United States Director of Operational Test and Evaluation (DOT&E) 2015 annual report found that 'full block 3F mission systems development and testing cannot be completed by May 2017, the date reflected in the most recent Program Office schedule', estimating that the program is not likely to finish Block 3F development and flight testing prior to January 2018.[31]

Helmet Mounted Display System

3.24      Some submissions raised concerns regarding the performance of the Helmet Mounted Display System.[32] Defence acknowledged that there have been issues during the development of the system, including:

• Green Glow: in which the minimum brightness of the optic driver in no-light or low-light conditions was too bright for some carrier operations;
• Jitter: in which the displayed symbols are difficult to read because of the movements of the pilot's head due to the buffet effect;
• Aided Night Vision Acuity: in which there was insufficient contrast in low light conditions; and
• Alignment/Optical Targeting Accuracy: in which symbols' alignment accuracy met requirements for Block 2 capabilities but needed to be improved to meet alignment accuracy requirements for Block 3 and up, which are driven by gun strafe capabilities requirements.[33] 

3.25      Lieutenant General Bogdan assured the committee that the Gen III helmet has addressed these issues:

We originally had problems with our helmet; as you recall they were issues knowns as green glow, jitter, swimming, latency, and poor visual acuity. Twelve months after these discoveries, we fielded our new Gen III helmet...which pilots are using today...with no problems.[34]

3.26      Defence agreed, advising the committee that, while the Helmet Mounted Display System is still assessed as a high technical risk, 'current flight test results are indicating that the Helmet Mounted Display System issues are being addressed by manufacturer, Rockwell Collins, and prime contractor, Lockheed Martin'.[35]

3.27      The DOT&E 2015 report also noted that, after developmental testing of the Gen III helmet, 'developmental test pilots reported less jitter, proper alignment, improved ability to set symbology intensity, less latency in imagery projections, and improved performance of the night vision camera'. However, it warned that 'operational testing in realistic conditions and mission task levels, including gun employment, is required to determine if further adjustments are needed'.[36]

Mission data loads and Autonomic Logistics Information System

3.28      The F-35 relies on mission data loads, which comprise compilations of the mission data files needed for the operation of the sensors and other mission systems components. The mission data loads work in conjunction with the system software data load to drive sensor search parameters and to identify and correlate sensor detections of threat radar signals. The loads are produced by the US Reprogramming Laboratory.[37] The Autonomic Logistics Information System (ALIS) transmits the
F-35's health and maintenance action information to the appropriate users on a globally-distributed network to technicians worldwide. ALIS receives Health Reporting Codes via a radio frequency downlink while the F-35 is still in flight, which will enable the pre-positioning of parts and qualified maintainers so that when the aircraft lands, downtime is minimized and efficiency is increased.[38]

3.29      Some submissions raised concerns regarding the F-35's reliance on data exchanges.[39] Mr Archibald warned that disruptions to data exchanges could significantly compromise the F-35's effectiveness and noted that aircraft turn-around will be directly linked to the speed with which the necessary data can be downloaded and uploaded:

All F-35 software laboratories are located within the United States. This has introduced vulnerabilities in the operation and sustainment of the global F-35 fleet that are only beginning to emerge. The biggest risk is that, since the F-35 cannot operate effectively without permanent data exchanges with its software labs and logistic support computers in the United States, any disruption in the two-way flow of information would compromise its effectiveness.

All F-35 aircraft operating across the world will have to update their mission data files and their ALIS profiles before and after every sortie, to ensure that on-board systems are programmed with the latest available operational data and that ALIS is kept permanently informed of each aircraft's technical status and maintenance requirements. ALIS can, and has, prevented aircraft taking off because of an incomplete data file. Currently, downloading the data file from a 1.5 hour flight of the F-35 takes 1.5 hours. It is hoped to get that down to 15 minutes. By comparison, the Gripen E can be re-armed and refuelled after an air-to-air mission in 10 minutes.

The volume of data that must travel to and from the United States is gigantic, and any disruption in Internet traffic could cripple air forces as the F-35 cannot operate unless it is logged into, and cleared by, ALIS. Updating and uploading mission data loads depends on a functioning Internet. That such a major weapon system would rely upon a separate and delicate system is the height of stupidity.[40]

3.30      Mr Robert Charette asserted that the F-35 cannot function without all of its software operating at an extremely high level, describing the F-35's reliance on ALIS as a 'major source of operational risk':

Software is the heart and soul of the F-35: without it functioning at an extremely high level of reliability, availability and maintainability, the F-35 is no more than a nice aircraft museum piece. This includes both the embedded software found on the aircraft itself, as well as the ALIS system which is tightly coupled in an unprecedented manner to the F-35...If ALIS indicates that an F-35 isn't ready to fly, it takes significant manual effort to override its decision. Further, it is a major source of operational risk: if ALIS doesn't work correctly, it is no exaggeration to state that the aircraft doesn't work, either. From a systems view, the reliability of the F-35 is a combination of both the embedded flight systems' software and the ALIS system, a fact that the F-35 program understandably does not wish to highlight.[41]

3.31      Mr Charette warned that partner countries' reliance on the United States for mission data files is 'an unquantified economic and operational risk':

One other known risk issue concerns not only all F-35 software is up-to-date among the US and all its partners, but also that all of the operational mission data files will be available in a timely manner. The F-35 program executive again acknowledges this is proving to be a risk that it underestimated. The impact of latency in delivery of F-35 software or data likely will be more severe than currently being estimated. The lack of control of the F-35 updates by its partners is also an unquantified economic and operational risk.[42]

3.32      Defence acknowledged that 'both the aircraft and support system maturity is still developing', but assured the committee that 'these systems continue to mature and improvements are becoming increasingly evident at operational units'.[43] Defence also advised that it is focused on mitigating strategic risks 'including the global support solution, maintenance and pilot training, information systems and the ability to develop mission data files'.[44] Air Vice-Marshal Deeble advised the committee that Defence is working to mitigate these risks by developing mission data reprograming capability:

I am talking about the ability to reprogram it with the information that is required to conduct each mission. We are setting up a collaborative facility with the Canadians and the UK at Eglin Air Force Base...That will be our reprogramming capability for the aircraft. I think it is true to say that reprogramming across the JSF enterprise is evolving as we speak. The ability to get a mission data file into the aircraft that will do everything we need it to do when the aircraft first arrives back here in Australia is a concern. We are working through those risks as we speak—what we are going to get, how good it is going to be and whether it is going to be capable of supporting our operational testing and evaluation. We are currently looking with the US at a range of risk mitigation approaches to address that specific risk.[45]

3.33      Lockheed Martin advised that the development of ALIS 2.0.2 is underway to support the USAF F-35A Initial Operating Capability in 2016 and that full ALIS capability (the ALIS 3.0 series) is scheduled for delivery in 2017:

The F-35 Lightning II is the first tactical aircraft system with sustainment tools designed in concert with the air vehicle to optimise operations. Initially fielded in 2009, the Autonomic Logistics Information System (ALIS) is maturing along with aircraft capability. The next generation of ALIS, ALIS 2.0, completed installation at all current F-35 operating locations in March 2015. As a result, it has equipped the F-35 enterprise with improvements across all of its fleet-management reporting tools. A subsequent release, ALIS 2.0.1, completed its roll out to all F-35 locations in September 2015. This upgrade included a deployable hardware suite called the Standard Operating Unit version 2. This suite supports operations on carriers, amphibious craft and remote locations. ALIS 2.0.1 supported the USMC Initial Operational Capability (IOC) in July 2015.

Development of ALIS 2.0.2 is underway to support the USAF F-35A Initial Operating Capability in 2016. It includes four major enhancements: an electronic deployment planning tool, a networking feature, parts life-tracking and the integration of engine propulsion management with air vehicle data.

The full ALIS capability - the ALIS 3.0 series - is scheduled for delivery in 2017, in line with the conclusion of the SOD program phase, to support U.S. and partner-nation operations. Currently, a representative simulation of an ALIS squadron kit is located at Lockheed Martin Centennial House in Canberra. This ALIS squadron kit demonstrates the ALIS functions of maintenance, supply, training and preventive maintenance activities that are typically performed at the aircraft squadron.[46]

Cybersecurity

3.34      Mr Charette advised the committee that there are growing concerns regarding the cybersecurity requirements for the F-35 and its systems. He noted that the scope of the requirements for cybersecurity were originally underestimated and that much of the security is now being implemented into the systems 'after the fact' rather than being designed 'from the beginning':

...there is now a requirement for greatly enhanced cyber security above that which was planned for when the F-35 and ALIS were first being developed. As the F-35 contractor program team has admitted, this was an 'unforeseen requirement', at least in its scope. While debatable, the fact remains that to implement security into a system after the fact, rather than designing it in from the beginning, is a well-known 'original sin' of any software system development. With the extensive use of commercial-off-the-shelf (COTS) software in ALIS, much without robust cyber security built in from the beginning, is an especially worrying concern.[47]

3.35      Dr Joiner warned the committee that the F-35 and its systems remain vulnerable:

Finally, there is the cybersecurity testing. The cooperative vulnerability and penetration testing for the actual aircraft—never mind the logistics system—has not been commenced, and there is no assurance that Australia will be part of that testing. Until it does commence, that software, to my mind, remains vulnerable.[48]

3.36      The DOT&E 2015 report noted that the limited cybersecurity testing that was permitted 'revealed significant deficiencies that must be corrected and highlighted the requirement to complete all planned cybersecurity testing'.[49]

Escape system

3.37      Some submissions raised concerns regarding reports of the failure of the escape system to successfully eject a manikin without exceeding the load/stress limits on the manikin.[50] Defence advised the committee that an increased risk of neck injury to light weight pilots during low speed ejection had been identified and subsequently addressed. In August 2015, the US Services restricted F-35 pilots weighing less than 136 pounds (62 kilograms) from operating the aircraft. Defence advised that, currently, no F-35 pilots, including Australian pilots, are impacted by this restriction. Safe escape risks are also being reduced by installing a switch on the seat for lightweight pilots that will slightly delay parachute deployment and lessen parachute opening forces; designing a lighter helmet; and mounting a head support panel, which is a fabric panel sewn between the parachute risers which will protect the pilot's head from moving backwards during the parachute opening.[51]

Verification Simulator (VSim)

3.38      A number of submissions questioned the accuracy of the Verification Simulator (VSim) used to test the performance of the F-35.[52] Mr Michael Price asserted that VSim, the main simulation built by Lockheed Martin and used by the
F-35 project, does not adequately reflect the capabilities of the F-35. Mr Price advised the committee that 'all the JSF project simulation results gathered over the last 10 years or so have no validity at all'.[53] He also questioned the wisdom of allowing Lockheed Martin to construct both the F-35 and the simulator intended to test its performance.[54] Mr Price described the VSim as the world's most expensive multi-player, interactive videogame, asserting that:

[The simulation results] only represent parts of a virtual F-35 in a virtual world (Lockheed Martin Land) where the laws of physics, advanced threats and systems are ignored and the virtual F-35 has capabilities that do not exist outside of the simulation.

Right now [the simulation] is not only incomplete in terms of contemporary and future threats as well as models for the combat scenarios but also inaccurate for its intended purpose.[55]

3.39      Mr Chris Mills asserted that the VSim has 'not passed essential Verification and Validation tests'[56] and is therefore incapable of accurately assessing the F-35's combat effectiveness in contested environments:

Without a functional VSim capability, the JSF will not be able to be evaluated for 'combat effectiveness' in contested environments featuring 'Anti-Access/Area Denial' systems, and highly capable and lethal purpose-built air combat fighters such as the Su-35S, the Su-50, Chengdu J-20 and the Shenyang J-31.[57]

3.40      Dr Keith Joiner raised concerns regarding the small percentage of the F-35's capability in the simulation that has been proven:

...the validation of the simulation model for the JSF aircraft is badly incomplete, lacks leadership and there has been a small percentage of testing. If you look at [the DOT&E report, it] says 10 per cent of the capability of the JSF in that simulation model has actually been proven. That is 90 per cent that has not been proven, so that is a real worry.[58]

3.41      Many submissions and witnesses pointed to the comments made by the United States Director of Operational Test and Evaluation (DOT&E) in its 2013 report, which stated that it was tracking formal risks with regard to VSim, including among other things:

• risks associated with the timeliness of VSim software delivery, completeness with regard to modelled capabilities, and discrepancies between VSim and aircraft software;
• risks associated with the timeliness, completeness and production-representativeness of data from flight testing and other testing used to verify and validate VSim; and
• fundamental risks regarding the ability of VSim to faithfully replicate all aspects of F-35 and threat systems performance.[59]

3.42      Defence explained that 'simulation is key to replicating the threat environments that the F-35 is expected to be employed in', noting that 'the development of a highly accurate simulation environment that can effectively be used to test the F-35 has been challenging and the path forward is currently being determined by the US Department of Defense'.[60] The former manager of the Joint Strike Fighter program, Air Vice-Marshal Chris Deeble (Retd), advised the committee that VSim remains an area of concern:

The area of VSim is another area of concern. VSim is not just about testing and evaluation. It is also about high-fidelity training in the full-mission aircraft simulators we have. Lockheed Martin have not lost all of the contract. They are still responsible for refining the aircraft model that is fundamental to VSim. The work that has gone to the US Navy is the work associated with the joint synthetic environment—effectively where the threats get represented appropriately. That work is going to the US Navy, but the two have to come back together. We hope to leverage that. While the simulator has high fidelity for the conduct of ab initio training, you need to train your pilot well in a simulator before you take them into the aircraft. We are concerned about being able to do mission rehearsal, which requires much higher fidelity. We hope to leverage the work that is being done in that joint synthetic environment and the work being done at Lockheed Martin to refine their aircraft model—to get that back into the full mission simulator to improve the level and nature of training we can conduct.[61]

3.43      The committee asked Lockheed Martin to explain the basis for its claims regarding the F-35's operational capability if its modelling has not been verified or certified, particularly with regards to threats for ground-based integrated air defence systems, emerging multimode systems, and the emerging Russian and Chinese threats in a stealth-on-stealth situation. Mr Jeff Babione, Executive Vice President and General Manager of the F-35 Lightning II Program at Lockheed Martin, advised that it is currently relying on data from the flight test community:

Most of that is based on the data from the flight test programs, so not the operational users testing it—although we do have an OT community that has been doing operational tests and evaluation on their own as they get the capability. The scenarios that I am talking about are ones that we have set up as part of the flight test program in the development phase, where we may set up a scenario where we are going against ground forces and we have to detect, track and destroy that target—for example, complete the kill chain, avoid a SAM site as we are approaching or to measure the actual detection range for the SAM site or the ability of the F-35 to avoid it. All that is information or data that is being gathered as we do the flight test program.

The verification and validation that we talk about in the virtual simulation is an extension of that that has yet to actually occur, but we are looking at the data that we are currently seeing from the flight test community as to the capabilities of the aeroplane. The US Air Force and the US Marine Corps have seen that same data and is making decisions as to whether or not the capabilities are sufficient for their IOCs.[62]

3.44      At the time of the 2015 DOT&E Report, these risks do not appear to have been adequately addressed, with the report noting that 'the program has failed to develop and deliver a Verification Simulator (VSim) for use either by the developmental test team or the JSF Operational Test Team'. The report explained that the Joint Strike Fighter Operational Test Team (JOTT) now plan to conduct a significant number of additional open-air flights during Initial Operational Test and Evaluation (IOT&E) to partially compensate for the lack of a simulator test venue:

The Program Office's sudden decision in August 2015 to move the VSim to a Naval Air Systems Command (NAVAIR)-proposed, government-led Joint Simulation Environment (JSE), will not result in a simulation with the required capabilities and fidelity in time for F-35 IOT&E. Without a high-fidelity simulation, the F-35 IOT&E will not be able to test the F-35's full capabilities against the full range of required threats and scenarios. Nonetheless, because aircraft continue to be produced in substantial quantities (all of which will require some level of modifications and retrofits before being used in combat), the IOT&E must be conducted without further delay to evaluate F-35 combat effectiveness under the most realistic conditions that can be obtained. Therefore, to partially compensate for the lack of a simulator test venue, the JOTT will now plan to conduct a significant number of additional open-air flights during IOT&E relative to the previous test designs. In the unlikely event a simulator test venue is available, the additional flights would not be flown.[63]

Navigation: Previous Page | Contents | Next Page