Evading classifiers in discrete domains with provable optimality guarantees

Abstract

Machine-learning models for security-critical applications such as bot, malware, or spam detection, operate in constrained discrete domains. These applications would benefit from having provable guarantees against adversarial examples. The existing literature on provable adversarial robustness of models, however, exclusively focuses on robustness to gradient-based attacks in domains such as images. These attacks model the adversarial cost, e.g., amount of distortion applied to an image, as a $p$-norm. We argue that this approach is not well-suited to model adversarial costs in constrained domains where not all examples are feasible. We introduce a graphical framework that (1) generalizes existing attacks in discrete domains, (2) can accommodate complex cost functions beyond $p$-norms, including financial cost incurred when attacking a classifier, and (3) efficiently produces valid adversarial examples with guarantees of minimal adversarial cost. These guarantees directly translate into a notion of adversarial robustness that takes into account domain constraints and the adversary's capabilities. We show how our framework can be used to evaluate security by crafting adversarial examples that evade a Twitter-bot detection classifier with provably minimal number of changes; and to build privacy defenses by crafting adversarial examples that evade a privacy-invasive website-fingerprinting classifier.

NeurIPS 2018 Workshop on Security in Machine Learning

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Deep Text Classification Can be Fooled
• Adversarial Perturbations Against Deep Neural Networks for Malware Classification
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Deceiving Google's Perspective API Built for Detecting Toxic Comments
• Robustness of classifiers: from adversarial to random noise
• A Game-Based Approximate Verification of Deep Neural Networks with Provable Guarantees
• Provably Minimally-Distorted Adversarial Examples
• AttriGuard: A Practical Defense Against Attribute Inference Attacks via Adversarial Machine Learning
• Cost-Sensitive Robustness against Adversarial Examples

Cited by in corpus (4)

• TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
• Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
• Cost-Aware Robust Tree Ensembles for Security Applications
• Defense of Word-level Adversarial Attacks via Random Substitution Encoding