Quality Management of Machine Learning Systems

Abstract

In the past decade, Artificial Intelligence (AI) has become a part of our daily lives due to major advances in Machine Learning (ML) techniques. In spite of an explosive growth in the raw AI technology and in consumer facing applications on the internet, its adoption in business applications has conspicuously lagged behind. For business/mission-critical systems, serious concerns about reliability and maintainability of AI applications remain. Due to the statistical nature of the output, software 'defects' are not well defined. Consequently, many traditional quality management techniques such as program debugging, static code analysis, functional testing, etc. have to be reevaluated. Beyond the correctness of an AI model, many other new quality attributes, such as fairness, robustness, explainability, transparency, etc. become important in delivering an AI system. The purpose of this paper is to present a view of a holistic quality management framework for ML applications based on the current advances and identify new areas of software engineering research to achieve a more trustworthy AI.

arXiv commentAAAI-20 Workshop on Engineering Dependable and Secure Machine Learning Systems- February 7, 2020

References in corpus (9)

• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
• Testing Deep Neural Networks
• Machine Learning Testing: Survey, Landscapes and Horizons
• Combinatorial Testing for Deep Learning Systems
• Debugging Machine Learning Tasks
• Characterizing machine learning process: A maturity framework
• Towards Automating the AI Operations Lifecycle
• Engineering Reliable Deep Learning Systems