Privately Learning Subspaces
arXiv:2106.00001
Abstract
Private data analysis suffers a costly curse of dimensionality. However, the data often has an underlying low-dimensional structure. For example, when optimizing via gradient descent, the gradients often lie in or near a low-dimensional subspace. If that low-dimensional structure can be identified, then we can avoid paying (in terms of privacy or accuracy) for the high ambient dimension. We present differentially private algorithms that take input data sampled from a low-dimensional linear subspace (possibly with a small amount of error) and output that subspace (or an approximation to it). These algorithms can serve as a pre-processing step for other procedures.
References in corpus (10)
- Deep Learning with Differential Privacy
- Visualizing the Loss Landscape of Neural Nets
- How To Break Anonymity of the Netflix Prize Dataset
- Gradient Descent Happens in a Tiny Subspace
- Beating Randomized Response on Incoherent Matrices
- Measuring the Intrinsic Dimension of Objective Landscapes
- Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification
- Private Query Release Assisted by Public Data
- Fast Dimension Independent Private AdaGrad on Publicly Estimated Subspaces
- How neural networks find generalizable solutions: Self-tuned annealing in deep learning