Scaling Private Deep Learning with Low-Rank and Sparse Gradients

Ryuichi Ito, Seng Pei Liew, Tsubasa Takahashi +2

Applying Differentially Private Stochastic Gradient Descent (DPSGD) to training modern, large-scale neural networks such as transformer-based models is a challenging task, as the m…