Guaranteeing Safety for Neural Network-Based Aircraft Collision Avoidance Systems

Abstract

The decision logic for the ACAS X family of aircraft collision avoidance systems is represented as a large numeric table. Due to storage constraints of certified avionics hardware, neural networks have been suggested as a way to significantly compress the data while still preserving performance in terms of safety. However, neural networks are complex continuous functions with outputs that are difficult to predict. Because simulations evaluate only a finite number of encounters, simulations are not sufficient to guarantee that the neural network will perform correctly in all possible situations. We propose a method to provide safety guarantees when using a neural network collision avoidance system. The neural network outputs are bounded using neural network verification tools like Reluplex and Reluval, and a reachability method determines all possible ways aircraft encounters will resolve using neural network advisories and assuming bounded aircraft dynamics. Experiments with systems inspired by ACAS X show that neural networks giving either horizontal or vertical maneuvers can be proven safe. We explore how relaxing the bounds on aircraft dynamics can lead to potentially unsafe encounters and demonstrate how neural network controllers can be modified to guarantee safety through online costs or lowering alerting cost. The reachability method is flexible and can incorporate uncertainties such as pilot delay and sensor error. These results suggest a method for certifying neural network collision avoidance systems for use in real aircraft.

arXiv comment10 pages, 11 figures, presented at the 2019 AIAA Digital Avionics Systems Conference (DASC)

References in corpus (4)

• Formal Security Analysis of Neural Networks using Symbolic Intervals
• Deep Neural Network Compression for Aircraft Collision Avoidance Systems
• Reachability Analysis and Safety Verification for Neural Network Control Systems
• Verifying Aircraft Collision Avoidance Neural Networks Through Linear Approximations of Safe Regions

Cited by in corpus (14)

• On the Opportunities and Risks of Foundation Models
• How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
• Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions
• Neural Network Compression of ACAS Xu Early Prototype is Unsafe: Closed-Loop Verification through Quantized State Backreachability
• Deep Model Predictive Control with Stability Guarantees
• From Requirements to Autonomous Flight: An Overview of the Monitoring ICAROUS Project
• Generating Probabilistic Safety Guarantees for Neural Network Controllers
• Discovering Closed-Loop Failures of Vision-Based Controllers via Reachability Analysis
• Sound and Complete Neural Network Repair with Minimality and Locality Guarantees
• Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networks
• Verification of Neural Network Control Systems in Continuous Time
• Probabilistic Safety for Bayesian Neural Networks
• Scaling up Memory-Efficient Formal Verification Tools for Tree Ensembles
• Neural Network Repair with Reachability Analysis