Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges

Abstract

Machine learning has evolved into an enabling technology for a wide range of highly successful applications. The potential for this success to continue and accelerate has placed machine learning (ML) at the top of research, economic and political agendas. Such unprecedented interest is fuelled by a vision of ML applicability extending to healthcare, transportation, defence and other domains of great societal importance. Achieving this vision requires the use of ML in safety-critical applications that demand levels of assurance beyond those needed for current ML applications. Our paper provides a comprehensive survey of the state-of-the-art in the assurance of ML, i.e. in the generation of evidence that ML is sufficiently safe for its intended use. The survey covers the methods capable of providing such evidence at different stages of the machine learning lifecycle, i.e. of the complex, iterative process that starts with the collection of the data used to train an ML component for a system, and ends with the deployment of that component within the system. The paper begins with a systematic presentation of the ML lifecycle and its stages. We then define assurance desiderata for each stage, review existing methods that contribute to achieving these desiderata, and identify open challenges that require further research.

References in corpus (28)

• Explaining and Harnessing Adversarial Examples
• Improving neural networks by preventing co-adaptation of feature detectors
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Intriguing properties of neural networks
• Towards A Rigorous Science of Interpretable Machine Learning
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Data Augmentation Generative Adversarial Networks
• Random Erasing Data Augmentation
• Tunability: Importance of Hyperparameters of Machine Learning Algorithms
• Linear Dimensionality Reduction: Survey, Insights, and Generalizations
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• Improving Deep Learning using Generic Data Augmentation
• A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
• Learn on Source, Refine on Target:A Model Transfer Learning Framework with Random Forests
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Verification for Machine Learning, Autonomy, and Neural Networks Survey
• CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
• DeepRoad: GAN-based Metamorphic Autonomous Driving System Testing
• A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
• Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262
• Feature Engineering for Predictive Modeling using Reinforcement Learning
• Big Holes in Big Data: A Monte Carlo Algorithm for Detecting Large Hyper-rectangles in High Dimensional Data
• Controlling Over-generalization and its Effect on Adversarial Examples Generation and Detection
• Rethinking Numerical Representations for Deep Neural Networks
• LEAFAGE: Example-based and Feature importance-based Explanationsfor Black-box ML models

Cited by in corpus (11)

• Challenges in Deploying Machine Learning: a Survey of Case Studies
• A Safety Framework for Critical Systems Utilising Deep Neural Networks
• Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS)
• Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure
• Choosing the Right Path for AI Integration in Engineering Companies: A Strategic Guide
• Certification of embedded systems based on Machine Learning: A survey
• Machine Learning practices and infrastructures
• A Framework for Assurance of Medication Safety using Machine Learning
• Optimization's Neglected Normative Commitments
• Benchmark of Data Preprocessing Methods for Imbalanced Classification
• Learn Faster and Forget Slower via Fast and Stable Task Adaptation