A Mapping of Assurance Techniques for Learning Enabled Autonomous Systems to the Systems Engineering Lifecycle

Abstract

Learning enabled autonomous systems provide increased capabilities compared to traditional systems. However, the complexity of and probabilistic nature in the underlying methods enabling such capabilities present challenges for current systems engineering processes for assurance, and test, evaluation, verification, and validation (TEVV). This paper provides a preliminary attempt to map recently developed technical approaches in the assurance and TEVV of learning enabled autonomous systems (LEAS) literature to a traditional systems engineering v-model. This mapping categorizes such techniques into three main approaches: development, acquisition, and sustainment. We review the latest techniques to develop safe, reliable, and resilient learning enabled autonomous systems, without recommending radical and impractical changes to existing systems engineering processes. By performing this mapping, we seek to assist acquisition professionals by (i) informing comprehensive test and evaluation planning, and (ii) objectively communicating risk to leaders.

References in corpus (20)

• A Brief Survey of Deep Reinforcement Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• End to End Learning for Self-Driving Cars
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• A Survey of Autonomous Driving: Common Practices and Emerging Technologies
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Object Detection in 20 Years: A Survey
• An Algorithmic Perspective on Imitation Learning
• A Survey of End-to-End Driving: Architectures and Training Methods
• Formal Specification and Verification of Autonomous Robotic Systems: A Survey
• A Survey on AI Assurance
• Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
• A Survey of Runtime Monitoring Instrumentation Techniques
• Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences
• Measuring the Reliability of Reinforcement Learning Algorithms
• Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
• Modeling Generalization in Machine Learning: A Methodological and Computational Study
• Synthesis of Static Test Environments for Observing Sequence-like Behaviors in Autonomous Systems