37 citations · 57 across the 7 of their papers we have counts for
4 papers · 1 filter
Online Learning-guided Learning Rate Adaptation via Gradient Alignment
Ruichen Jiang, Ali Kavis, Aryan Mokhtari
The performance of an optimizer on large-scale deep learning models depends critically on fine-tuning the learning rate, often requiring an extensive grid search over base learning…
Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting
Sunny Sanyal, Hayden Prairie, Rudrajit Das +2
Fine-tuning a pre-trained model on a downstream task often degrades its original capabilities, a phenomenon known as "catastrophic forgetting". This is especially an issue when one…
Understanding Self-Supervised Learning via Gaussian Mixture Models
Parikshit Bansal, Ali Kavis, Sujay Sanghavi
Self-supervised learning attempts to learn representations from un-labeled data; it does so via a loss function that encourages the embedding of a point to be close to that of its…
Efficient learning of smooth probability functions from Bernoulli tests with guarantees
Paul Rolland, Ali Kavis, Alex Immer +2
We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests. We provide a scalable algorithm for efficiently solving this pro…