Understanding the Tradeoffs in Client-side Privacy for Downstream Speech Tasks

Abstract

As users increasingly rely on cloud-based computing services, it is important to ensure that uploaded speech data remains private. Existing solutions rely either on server-side methods or focus on hiding speaker identity. While these approaches reduce certain security concerns, they do not give users client-side control over whether their biometric information is sent to the server. In this paper, we formally define client-side privacy and discuss its three unique technical challenges: (1) direct manipulation of raw data on client devices, (2) adaptability with a broad range of server-side processing models, and (3) low time and space complexity for compatibility with limited-bandwidth devices. Solving these challenges requires new models that achieve high-fidelity reconstruction, privacy preservation of sensitive personal attributes, and efficiency during training and inference. As a step towards client-side privacy for speech recognition, we investigate three techniques spanning signal processing, disentangled representation learning, and adversarial training. Through a series of gender and accent masking tasks, we observe that each method has its unique strengths, but none manage to effectively balance the trade-offs between performance, privacy, and complexity. These insights call for more research in client-side privacy to ensure a safer deployment of cloud-based speech processing services.

References in corpus (17)

• Deep Learning with Differential Privacy
• WaveNet: A Generative Model for Raw Audio
• Instance Normalization: The Missing Ingredient for Fast Stylization
• VoxCeleb: a large-scale speaker identification dataset
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Differentially Private Federated Learning: A Client Level Perspective
• Scalable Private Learning with PATE
• Video (language) modeling: a baseline for generative models of natural videos
• LEAF: A Benchmark for Federated Settings
• Understanding disentangling in $\beta$-VAE
• Differential Privacy and Machine Learning: a Survey and Review
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Jukebox: A Generative Model for Music
• Introducing the VoicePrivacy Initiative
• Preech: A System for Privacy-Preserving Speech Transcription
• Adversarial representation learning for private speech generation
• Automatically Identifying Language Family from Acoustic Examples in Low Resource Scenarios

Cited by in corpus (2)

• Augmented Datasheets for Speech Datasets and Ethical Decision-Making
• Beyond Neural-on-Neural Approaches to Speaker Gender Protection