VSMask: Defending Against Voice Synthesis Attack via Real-Time Predictive Perturbation
arXiv:2305.05736 · doi:10.1145/3558482.3590189
Abstract
Deep learning based voice synthesis technology generates artificial human-like speeches, which has been used in deepfakes or identity theft attacks. Existing defense mechanisms inject subtle adversarial perturbations into the raw speech audios to mislead the voice synthesis models. However, optimizing the adversarial perturbation not only consumes substantial computation time, but it also requires the availability of entire speech. Therefore, they are not suitable for protecting live speech streams, such as voice messages or online meetings. In this paper, we propose VSMask, a real-time protection mechanism against voice synthesis attacks. Different from offline protection schemes, VSMask leverages a predictive neural network to forecast the most effective perturbation for the upcoming streaming speech. VSMask introduces a universal perturbation tailored for arbitrary speech input to shield a real-time speech in its entirety. To minimize the audio distortion within the protected speech, we implement a weight-based perturbation constraint to reduce the perceptibility of the added perturbation. We comprehensively evaluate VSMask protection performance under different scenarios. The experimental results indicate that VSMask can effectively defend against 3 popular voice synthesis models. None of the synthetic voice could deceive the speaker verification models or human ears with VSMask protection. In a physical world experiment, we demonstrate that VSMask successfully safeguards the real-time speech by injecting the perturbation over the air.
References in corpus (26)
- WaveNet: A Generative Model for Raw Audio
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Certified Adversarial Robustness via Randomized Smoothing
- SpeechBrain: A General-Purpose Speech Toolkit
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- Deep Voice 2: Multi-Speaker Neural Text-to-Speech
- One-class Learning Towards Synthetic Voice Spoofing Detection
- AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
- Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
- Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
- Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
- Robust Audio Adversarial Example for a Physical Attack
- AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
- Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
- Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information
- CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
- "Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real World
- GhostTalk: Interactive Attack on Smartphone Voice System Through Power Line
- Emotionless: Privacy-Preserving Speech Analysis for Voice Assistants
- WaveGuard: Understanding and Mitigating Audio Adversarial Examples
- Defending against Adversarial Audio via Diffusion Model
- Channel-wise Gated Res2Net: Towards Robust Detection of Synthetic Speech Attacks
- Defending Your Voice: Adversarial Attack on Voice Conversion
- V-Cloak: Intelligibility-, Naturalness- & Timbre-Preserving Real-Time Voice Anonymization
- Exploring Targeted Universal Adversarial Perturbations to End-to-end ASR Models
- Real-Time Neural Voice Camouflage