A Research Agenda: Dynamic Models to Defend Against Correlated Attacks

Abstract

In this article I describe a research agenda for securing machine learning models against adversarial inputs at test time. This article does not present results but instead shares some of my thoughts about where I think that the field needs to go. Modern machine learning works very well on I.I.D. data: data for which each example is drawn {\em independently} and for which the distribution generating each example is {\em identical}. When these assumptions are relaxed, modern machine learning can perform very poorly. When machine learning is used in contexts where security is a concern, it is desirable to design models that perform well even when the input is designed by a malicious adversary. So far most research in this direction has focused on an adversary who violates the {\em identical} assumption, and imposes some kind of restricted worst-case distribution shift. I argue that machine learning security researchers should also address the problem of relaxing the {\em independence} assumption and that current strategies designed for robustness to distribution shift will not do so. I recommend {\em dynamic models} that change each time they are run as a potential solution path to this problem, and show an example of a simple attack using correlated data that can be mitigated by a simple dynamic defense. This is not intended as a real-world security measure, but as a recommendation to explore this research direction and develop more realistic defenses.

References in corpus (6)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Detecting Adversarial Samples from Artifacts
• Motivating the Rules of the Game for Adversarial Example Research
• Training verified learners with learned verifiers

Cited by in corpus (8)

• Testing Robustness Against Unforeseen Adversaries
• Morphence: Moving Target Defense Against Adversarial Examples
• Adaptive Generation of Unrestricted Adversarial Inputs
• Anomalous Example Detection in Deep Learning: A Survey
• Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
• Hidden Incentives for Auto-Induced Distributional Shift
• Towards Evaluating the Robustness of Neural Networks Learned by Transduction
• Towards Adversarial Robustness via Transductive Learning