The National Airworthiness Council artificial intelligence working group (NACAIWG) summit proceedings 2022

Abstract

In 2021, the United States Department of Defense (DoD) airworthiness community joined together to form the National Airworthiness Council Artificial Intelligence Working Group (NACAIWG). In June 2021, the NACAIWG held their first summit and examined the use case of an uncrewed aircraft (UA), operating under the guidance of a United States Navy (USN) permanent flight clearance (PFC), performing automated air-to-air refueling (A3R), a mission standardized by the North Atlantic Treaty Organization (NATO) 3 months post the 2021 summit,1 as the probed receiver of a drogue configured aircraft.2 In June of 2022, a second summit was held to examine potential artifacts collected from academia to support a technical assessment, leveraging defined standards, criteria, and methods of compliance within specific relevant technical domains, by airworthiness authorities for a learning enabled component (LEC) of a learning enabled system (LES) to perform the object detection portion of the A3R task covered in the 2021 summit. This communication article summarizes the findings of the 2022 summit. During the 2021 summit, a baseline overview of literature pertaining to the field of computer vision (CV) was provided. For the 2022 summit, a sample of literature published since the 2021 summit was provided to introduce participants to updated work in the field of test and evaluation (T&E), CV, algorithmic assurance, and frameworks to provide assurance of algorithms. All papers were collected from the Purdue University Library3 and Dissertations and Theses Database4 and cited in references.5-14 Building off the standards survey conducted for the 2021 summit, a follow-on survey of standards for all modes of transportation (e.g., rail, sea, road, etc.) was conducted in support of the 2022 summit to determine if any notable improvements or advancements had occurred. While evolutionary gradual improvements have occurred within the field, a standard that governs ML applications that could be considered final currently does not exist at the time of submission of this paper. Key documents developed over the past few years not covered at the 2021 Summit were the Safety Assurance Objectives for Autonomous Systems Version 3.015 and SAE AS-6983.16 The greatest potential addition to the airworthiness certification community for the certification of LES since the 2021 Summit was the draft document AS-6983 from SAE G-34.16 AS-6983 is targeted at filling the void, identified in AFE-87 and AIR-6988, in the existing civilian aviation standards for the certification of traditional systems, but not a LES. Many of the members of the G-34 are from European Aviation Safety Agency (EASA), and thus contributed to the construction of the EASA Roadmap and EASA Level 1 guidance.17 Universally, the 2022 survey found that key areas associated with flight safety critical applications remain unaddressed to include the areas of ML tool qualification, ML hardware concerns for graphic processing units (GPU) and Tensor Processing Units (TPU), ML object-oriented software language concerns, ML reuse concerns, and reinforcement learning. The overall certification of the human/air system combination is divided into two parts: certifying the human and certifying the air system. The naval airworthiness process to certify a system is a technical assessment process, leveraging defined standards, criteria, and methods of compliance within specific relevant technical domains. An airworthiness assessment identifies areas of technical compliance and potentially, non-compliance. Areas of non-compliance are examined to identify and characterize resultant hazards, possible mitigations, or even areas potentially requiring re-design. Resultant residual risks are adjudicated through an appropriate risk acceptance process. Once the risk has been identified and mitigated to an acceptable level, the Airworthiness and Cybersafe Office (ACO) certifies the air system to be operated by a human operator, eithe