Towards Quality Assurance of Software Product Lines with Adversarial Configurations
arXiv:1909.07283
Abstract
Software product line (SPL) engineers put a lot of effort to ensure that, through the setting of a large number of possible configuration options, products are acceptable and well-tailored to customers' needs. Unfortunately, options and their mutual interactions create a huge configuration space which is intractable to exhaustively explore. Instead of testing all products, machine learning techniques are increasingly employed to approximate the set of acceptable products out of a small training sample of configurations. Machine learning (ML) techniques can refine a software product line through learned constraints and a priori prevent non-acceptable products to be derived. In this paper, we use adversarial ML techniques to generate adversarial configurations fooling ML classifiers and pinpoint incorrect classifications of products (videos) derived from an industrial video generator. Our attacks yield (up to) a 100% misclassification rate and a drop in accuracy of 5%. We discuss the implications these results have on SPL quality assurance.
arXiv commentThis is a preview version of a paper accepted and presented at SPLC'19 that took place in Paris from 9th to 13th of September 2019. Some minor changes might appear compared to the one from the proceedings of the conference
References in corpus (11)
- SMOTE: Synthetic Minority Over-sampling Technique
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- DeepXplore: Automated Whitebox Testing of Deep Learning Systems
- Poisoning Attacks against Support Vector Machines
- Robust Physical-World Attacks on Deep Learning Models
- Countering Adversarial Images using Input Transformations
- Stochastic Activation Pruning for Robust Adversarial Defense
- Using Bad Learners to find Good Configurations
- Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack
- DeepRoad: GAN-based Metamorphic Autonomous Driving System Testing