Privacy Accounting and Quality Control in the Sage Differentially Private ML Platform

Abstract

Companies increasingly expose machine learning (ML) models trained over sensitive user data to untrusted domains, such as end-user devices and wide-access model stores. We present Sage, a differentially private (DP) ML platform that bounds the cumulative leakage of training data through models. Sage builds upon the rich literature on DP ML algorithms and contributes pragmatic solutions to two of the most pressing systems challenges of global DP: running out of privacy budget and the privacy-utility tradeoff. To address the former, we develop block composition, a new privacy loss accounting method that leverages the growing database regime of ML workloads to keep training models endlessly on a sensitive data stream while enforcing a global DP guarantee for the stream. To address the latter, we develop privacy-adaptive training, a process that trains a model on growing amounts of data and/or with increasing privacy parameters until, with high probability, the model meets developer-configured quality criteria. They illustrate how a systems focus on characteristics of ML workloads enables pragmatic solutions that are not apparent when one focuses on individual algorithms, as most DP ML literature does.

arXiv commentExtended version of a paper presented at the 27th ACM Symposium on Operating Systems Principles (SOSP '19)

References in corpus (14)

• Deep Learning with Differential Privacy
• Learning Differentially Private Recurrent Language Models
• Stealing Machine Learning Models via Prediction APIs
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Differentially Private Model Publishing for Deep Learning
• On the `Semantics' of Differential Privacy: A Bayesian Formulation
• A General Approach to Adding Differential Privacy to Iterative Training Procedures
• Evaluating Differentially Private Machine Learning in Practice
• Privacy Odometers and Filters: Pay-as-you-Go Composition
• Privacy Accounting and Quality Control in the Sage Differentially Private ML Platform
• Ektelo: A Framework for Defining Differentially-Private Computations
• Lower Bounds for Locally Private Estimation via Communication Complexity
• Differential Privacy for Growing Databases
• Statistical Model Compression for Small-Footprint Natural Language Understanding

Cited by in corpus (10)

• BlindFL: Vertical Federated Machine Learning without Peeking into Your Data
• Privacy Accounting and Quality Control in the Sage Differentially Private ML Platform
• Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation
• DPIS: An Enhanced Mechanism for Differentially Private SGD with Importance Sampling
• FPGA-Based Hardware Accelerator of Homomorphic Encryption for Efficient Federated Learning
• DP-Sync: Hiding Update Patterns in Secure Outsourced Databases with Differential Privacy
• Practical Privacy Filters and Odometers with Rényi Differential Privacy and Applications to Differentially Private Deep Learning
• Turbo: Effective Caching in Differentially-Private Databases
• DPack: Efficiency-Oriented Privacy Budget Scheduling
• PPVF: An Efficient Privacy-Preserving Online Video Fetching Framework with Correlated Differential Privacy