On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approach

Abstract

Privacy-utility tradeoff remains as one of the fundamental issues of differentially private machine learning. This paper introduces a geometrically inspired kernel-based approach to mitigate the accuracy-loss issue in classification. In this approach, a representation of the affine hull of given data points is learned in Reproducing Kernel Hilbert Spaces (RKHS). This leads to a novel distance measure that hides privacy-sensitive information about individual data points and improves the privacy-utility tradeoff via significantly reducing the risk of membership inference attacks. The effectiveness of the approach is demonstrated through experiments on MNIST dataset, Freiburg groceries dataset, and a real biomedical dataset. It is verified that the approach remains computationally practical. The application of the approach to federated learning is considered and it is observed that the accuracy-loss due to data being distributed is either marginal or not significantly high.

References in corpus (12)

• Deep Learning with Differential Privacy
• Differentially Private Empirical Risk Minimization
• To understand deep learning we need to understand kernel learning
• Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising
• The Freiburg Groceries Dataset
• The Deep Kernelized Autoencoder
• Optimal Noise-Adding Mechanism in Additive Differential Privacy
• Autoencoding any Data through Kernel Autoencoders
• Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nyström Method, and Use of Kernels in Machine Learning: Tutorial and Survey
• Optimal Noise Adding Mechanisms for Approximate Differential Privacy
• Membership-Mappings for Practical Secure Distributed Deep Learning
• Differentially Private Transferrable Deep Learning with Membership-Mappings