CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition

Abstract

The popularity of ASR (automatic speech recognition) systems, like Google Voice, Cortana, brings in security concerns, as demonstrated by recent attacks. The impacts of such threats, however, are less clear, since they are either less stealthy (producing noise-like voice commands) or requiring the physical presence of an attack device (using ultrasound). In this paper, we demonstrate that not only are more practical and surreptitious attacks feasible but they can even be automatically constructed. Specifically, we find that the voice commands can be stealthily embedded into songs, which, when played, can effectively control the target system through ASR without being noticed. For this purpose, we developed novel techniques that address a key technical challenge: integrating the commands into a song in a way that can be effectively recognized by ASR through the air, in the presence of background noise, while not being detected by a human listener. Our research shows that this can be done automatically against real world ASR applications. We also demonstrate that such CommanderSongs can be spread through Internet (e.g., YouTube) and radio, potentially affecting millions of ASR users. We further present a new mitigation technique that controls this threat.

Accepted by USENIX Security 2018

References in corpus (9)

• Deep Learning in Neural Networks: An Overview
• Distilling the Knowledge in a Neural Network
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Voice Recognition Algorithms using Mel Frequency Cepstral Coefficient (MFCC) and Dynamic Time Warping (DTW) Techniques
• Robust Physical-World Attacks on Deep Learning Models
• The Microsoft 2016 Conversational Speech Recognition System
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• BEBP: An Poisoning Method Against Machine Learning Based IDSs

Cited by in corpus (11)

• Robust Audio Adversarial Example for a Physical Attack
• Characterizing Audio Adversarial Examples Using Temporal Dependency
• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information
• Weighted-Sampling Audio Adversarial Example Attack
• VSMask: Defending Against Voice Synthesis Attack via Real-Time Predictive Perturbation
• Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
• Semantic-preserving Reinforcement Learning Attack Against Graph Neural Networks for Malware Detection
• LanCe: A Comprehensive and Lightweight CNN Defense Methodology against Physical Adversarial Attacks on Embedded Multimedia Applications
• FakeWake: Understanding and Mitigating Fake Wake-up Words of Voice Assistants
• Compensating Removed Frequency Components: Thwarting Voice Spectrum Reduction Attacks