Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation

Abstract

Neural networks are part of many contemporary NLP systems, yet their empirical successes come at the price of vulnerability to adversarial attacks. Previous work has used adversarial training and data augmentation to partially mitigate such brittleness, but these are unlikely to find worst-case adversaries due to the complexity of the search space arising from discrete text perturbations. In this work, we approach the problem from the opposite direction: to formally verify a system's robustness against a predefined class of adversarial attacks. We study text classification under synonym replacements or character flip perturbations. We propose modeling these input perturbations as a simplex and then using Interval Bound Propagation -- a formal model verification method. We modify the conventional log-likelihood training objective to train models that can be efficiently verified, which would otherwise come with exponential search complexity. The resulting models show only little difference in terms of nominal accuracy, but have much improved verified accuracy under perturbations and come with an efficiently computable formal guarantee on worst case adversaries.

arXiv commentEMNLP 2019

References in corpus (8)

• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Defenses against Adversarial Examples
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Formal Security Analysis of Neural Networks using Symbolic Intervals
• Provably Minimally-Distorted Adversarial Examples
• Training verified learners with learned verifiers
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples

Cited by in corpus (9)

• Reevaluating Adversarial Examples in Natural Language
• Towards Robustness Against Natural Language Word Substitutions
• Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
• Robustness Verification for Transformers
• Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Grey-box Adversarial Attack And Defence For Sentiment Classification
• T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack
• Adversarial Machine Learning in Text Analysis and Generation