204 citations · 295 across the 20 of their papers we have counts for
4 papers · 1 filter
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks
Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora
Autoregressive language models, pretrained using large text corpora to do well on next word prediction, have been successful at solving many downstream tasks, even with zero-shot u…
Predicting What You Already Know Helps: Provable Self-Supervised Learning
Jason D. Lee, Qi Lei, Nikunj Saunshi +1
Self-supervised representation learning solves auxiliary prediction tasks (known as pretext tasks) without requiring labeled data to learn useful semantic representations. These pr…
A Sample Complexity Separation between Non-Convex and Convex Meta-Learning
Nikunj Saunshi, Yi Zhang, Mikhail Khodak +1
One popular trend in meta-learning is to learn from many training tasks a common initialization for a gradient-based method that can be used to solve a new task with few samples. T…
Provable Representation Learning for Imitation Learning via Bi-level Optimization
Sanjeev Arora, Simon S. Du, Sham Kakade +2
A common strategy in modern learning systems is to learn a representation that is useful for many tasks, a.k.a. representation learning. We study this strategy in the imitation lea…