Security and Machine Learning in the Real World

Abstract

Machine learning (ML) models deployed in many safety- and business-critical systems are vulnerable to exploitation through adversarial examples. A large body of academic research has thoroughly explored the causes of these blind spots, developed sophisticated algorithms for finding them, and proposed a few promising defenses. A vast majority of these works, however, study standalone neural network models. In this work, we build on our experience evaluating the security of a machine learning software product deployed on a large scale to broaden the conversation to include a systems security view of these vulnerabilities. We describe novel challenges to implementing systems security best practices in software with ML components. In addition, we propose a list of short-term mitigation suggestions that practitioners deploying machine learning modules can use to secure their systems. Finally, we outline directions for new research into machine learning attacks and defenses that can serve to advance the state of ML systems security.

References in corpus (14)

• Intriguing properties of neural networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• Synthesizing Robust Adversarial Examples
• Efficient Formal Safety Analysis of Neural Networks
• Motivating the Rules of the Game for Adversarial Example Research
• On Adaptive Attacks to Adversarial Example Defenses
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• Adversarial Policies: Attacking Deep Reinforcement Learning
• Unrestricted Adversarial Examples
• MobilBye: Attacking ADAS with Camera Spoofing
• A Marauder's Map of Security and Privacy in Machine Learning

Cited by in corpus (5)

• Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection
• On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples
• Rethinking Image-Scaling Attacks: The Interplay Between Vulnerabilities in Machine Learning Systems
• Lower Bounds on Cross-Entropy Loss in the Presence of Test-time Adversaries
• Mental Models of Adversarial Machine Learning