Software Engineering for AI-Based Systems: A Survey
arXiv:2105.01984 · doi:10.1145/3487043
Abstract
AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.
arXiv commentAccepted in ACM Transactions on Software Engineering and Methodology (TOSEM). For its published version refer to the Journal of ACM TOSEM
References in corpus (48)
- Robust Physical-World Attacks on Deep Learning Models
- On a Formal Model of Safe and Scalable Self-driving Cars
- Formal Security Analysis of Neural Networks using Symbolic Intervals
- Software Engineering Challenges of Deep Learning
- Automated Directed Fairness Testing
- On the Performance of Hybrid Search Strategies for Systematic Literature Reviews in Software Engineering
- Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing
- Efficient Formal Safety Analysis of Neural Networks
- TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
- Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
- An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software
- Empirical Standards for Software Engineering Research
- Adoption and Effects of Software Engineering Best Practices in Machine Learning
- Applying Deep Learning To Airbnb Search
- Towards Verified Artificial Intelligence
- Clipper: A Low-Latency Online Prediction Serving System
- Machine Teaching: A New Paradigm for Building Machine Learning Systems
- Management of quality requirements in agile and rapid software development: A systematic mapping study
- DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks
- Combinatorial Testing for Deep Learning Systems
- Experimental Resilience Assessment of An Open-Source Driving Agent
- Software engineering for artificial intelligence and machine learning software: A systematic literature review
- Detecting Adversarial Samples for Deep Neural Networks through Mutation Testing
- DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems
- Defining Explainable AI for Requirements Analysis
- Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262
- Concolic Testing for Deep Neural Networks
- Machine Learning Software Engineering in Practice: An Industrial Case Study
- Paracosm: A Language and Tool for Testing Autonomous Driving Systems
- Debugging Machine Learning Tasks
- Software Engineering Practice in the Development of Deep Learning Applications
- MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks
- Infrastructure for Usable Machine Learning: The Stanford DAWN Project
- Synergy between Machine/Deep Learning and Software Engineering: How Far Are We?
- Engineering AI Systems: A Research Agenda
- Beyond the technical challenges for deploying Machine Learning solutions in a software company
- Characterizing machine learning process: A maturity framework
- Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review
- How Tertiary Studies perform Quality Assessment of Secondary Studies in Software Engineering
- Trustworthy AI in the Age of Pervasive Computing and Big Data
- What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow
- Engineering Reliable Deep Learning Systems
- Towards Testing of Deep Learning Systems with Training Set Reduction
- Improving Certified Robustness via Statistical Learning with Logical Reasoning
- Gray-box Adversarial Testing for Control Systems with Machine Learning Component
- DeepFault: Fault Localization for Deep Neural Networks
- DeepSaucer: Unified Environment for Verifying Deep Neural Networks
- Packaging and Sharing Machine Learning Models via the Acumos AI Open Platform