204 citations · 848 across the 22 of their papers we have counts for
9 papers · 1 filter
TextHide: Tackling Data Privacy in Language Understanding Tasks
Yangsibo Huang, Zhao Song, Danqi Chen +2
An unsolved challenge in distributed or federated learning is to effectively mitigate privacy risks without slowing down training or reducing accuracy. In this paper, we propose Te…
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate
Zhiyuan Li, Kaifeng Lyu, Sanjeev Arora
The paper proposes an 'intrinsic learning rate' that combines the usual learning rate and weight decay to model the training dynamics of normalized deep networks via stochastic dif…
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?
Zhiyuan Li, Yi Zhang, Sanjeev Arora
Convolutional neural networks often dominate fully-connected counterparts in generalization performance, especially on image classification tasks. This is often explained in terms…
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks
Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora
Autoregressive language models, pretrained using large text corpora to do well on next word prediction, have been successful at solving many downstream tasks, even with zero-shot u…
InstaHide: Instance-hiding Schemes for Private Distributed Learning
Yangsibo Huang, Zhao Song, Kai Li +1
How can multiple distributed entities collaboratively train a shared deep net on their private data while preserving privacy? This paper introduces InstaHide, a simple encryption o…
Privacy-preserving Learning via Deep Net Pruning
Yangsibo Huang, Yushan Su, Sachin Ravi +3
This paper attempts to answer the question whether neural network pruning can be used as a tool to achieve differential privacy without losing much data utility. As a first step to…
A Sample Complexity Separation between Non-Convex and Convex Meta-Learning
Nikunj Saunshi, Yi Zhang, Mikhail Khodak +1
One popular trend in meta-learning is to learn from many training tasks a common initialization for a gradient-based method that can be used to solve a new task with few samples. T…
Provable Representation Learning for Imitation Learning via Bi-level Optimization
Sanjeev Arora, Simon S. Du, Sham Kakade +2
A common strategy in modern learning systems is to learn a representation that is useful for many tasks, a.k.a. representation learning. We study this strategy in the imitation lea…
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality
Yi Zhang, Orestis Plevrakis, Simon S. Du +3
Adversarial training is a popular method to give neural nets robustness against adversarial perturbations. In practice adversarial training leads to low robust training loss. Howev…