Generating private data with user customization
arXiv:2012.01467
Abstract
Personal devices such as mobile phones can produce and store large amounts of data that can enhance machine learning models; however, this data may contain private information specific to the data owner that prevents the release of the data. We want to reduce the correlation between user-specific private information and the data while retaining the useful information. Rather than training a large model to achieve privatization from end to end, we first decouple the creation of a latent representation, and then privatize the data that allows user-specific privatization to occur in a setting with limited computation and minimal disturbance on the utility of the data. We leverage a Variational Autoencoder (VAE) to create a compact latent representation of the data that remains fixed for all devices and all possible private labels. We then train a small generative filter to perturb the latent representation based on user specified preferences regarding the private and utility information. The small filter is trained via a GAN-type robust optimization that can take place on a distributed device such as a phone or tablet. Under special conditions of our linear filter, we disclose the connections between our generative approach and renyi differential privacy. We conduct experiments on multiple datasets including MNIST, UCI-Adult, and CelebA, and give a thorough evaluation including visualizing the geometry of the latent embeddings and estimating the empirical mutual information to show the effectiveness of our approach.
arXiv commentarXiv admin note: substantial text overlap with arXiv:1904.09415
References in corpus (18)
- Auto-Encoding Variational Bayes
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Equality of Opportunity in Supervised Learning
- Renyi Differential Privacy
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Estimating divergence functionals and the likelihood ratio by convex risk minimization
- Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
- Learning Adversarially Fair and Transferable Representations
- Flexibly Fair Representation Learning by Disentanglement
- Learning to Protect Communications with Adversarial Neural Cryptography
- Stronger Data Poisoning Attacks Break Data Sanitization Defenses
- Learning Controllable Fair Representations
- Minimax Filter: Learning to Preserve Privacy from Inference Attacks
- DEEProtect: Enabling Inference-based Access Control on Mobile Sensing Applications
- Differentially Private Variational Inference for Non-conjugate Models
- Understanding Compressive Adversarial Privacy
- Application-driven Privacy-preserving Data Publishing with Correlated Attributes