69 citations · 157 across the 10 of their papers we have counts for
10 papers
Provable Meta-Learning with Low-Rank Adaptations
Jacob L. Block, Sundararajan Srinivasan, Liam Collins +2
The power of foundation models (FMs) lies in their capacity to learn highly expressive representations that can be adapted to a broad spectrum of tasks. However, these pretrained m…
In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
Liam Collins, Advait Parulekar, Aryan Mokhtari +2
A striking property of transformers is their ability to perform in-context learning (ICL), a machine learning framework in which the learner is presented with a novel context durin…
Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning
Liam Collins, Shanshan Wu, Sewoong Oh +1
In many applications of federated learning (FL), clients desire models that are personalized using their local data, yet are also robust in the sense that they retain general globa…
Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks
Liam Collins, Hamed Hassani, Mahdi Soltanolkotabi +2
An increasingly popular machine learning paradigm is to pretrain a neural network (NN) on many tasks offline, then adapt it to downstream tasks, often by re-training only the last…
InfoNCE Loss Provably Learns Cluster-Preserving Representations
Advait Parulekar, Liam Collins, Karthikeyan Shanmugam +2
The goal of contrasting learning is to learn a representation that preserves underlying clusters by keeping samples with similar content, e.g. the ``dogness'' of a dog, close to ea…
FedAvg with Fine Tuning: Local Updates Lead to Representation Learning
Liam Collins, Hamed Hassani, Aryan Mokhtari +1
The Federated Averaging (FedAvg) algorithm, which consists of alternating between a few local stochastic gradient updates at client nodes, followed by a model averaging update at t…
MAML and ANIL Provably Learn Representations
Liam Collins, Aryan Mokhtari, Sewoong Oh +1
Recent empirical evidence has driven conventional wisdom to believe that gradient-based meta-learning (GBML) methods perform well at few-shot learning because they learn an express…
Exploiting Shared Representations for Personalized Federated Learning
Liam Collins, Hamed Hassani, Aryan Mokhtari +1
Deep neural networks have shown the ability to extract universal feature representations from data such as images and text that have been useful for a variety of learning tasks. Ho…
How Does the Task Landscape Affect MAML Performance?
Liam Collins, Aryan Mokhtari, Sanjay Shakkottai
Model-Agnostic Meta-Learning (MAML) has become increasingly popular for training models that can quickly adapt to new tasks via one or few stochastic gradient descent steps. Howeve…
Task-Robust Model-Agnostic Meta-Learning
Liam Collins, Aryan Mokhtari, Sanjay Shakkottai
Meta-learning methods have shown an impressive ability to train models that rapidly learn new tasks. However, these methods only aim to perform well in expectation over tasks comin…