63 citations · 231 across the 30 of their papers we have counts for
7 papers · 1 filter
DIVA: Dataset Derivative of a Learning Task
Yonatan Dukler, Alessandro Achille, Giovanni Paolini +3
We present a method to compute the derivative of a learning task with respect to a dataset. A learning task is a function from a training set to the validation error, which can be…
Uniform Sampling over Episode Difficulty
Sébastien M. R. Arnold, Guneet S. Dhillon, Avinash Ravichandran +1
Episodic training is a core ingredient of few-shot learning to train models on tasks with limited labelled data. Despite its success, episodic training remains largely understudied…
Representation Consolidation for Training Expert Students
Zhizhong Li, Avinash Ravichandran, Charless Fowlkes +3
Traditionally, distillation has been used to train a student model to emulate the input/output functionality of a teacher. A more useful goal than emulation, yet under-explored, is…
A linearized framework and a new benchmark for model selection for fine-tuning
Aditya Deshpande, Alessandro Achille, Avinash Ravichandran +6
Fine-tuning from a collection of models pre-trained on different domains (a "model zoo") is emerging as a technique to improve test accuracy in the low-data regime. However, model…
Supervised Momentum Contrastive Learning for Few-Shot Classification
Orchid Majumder, Avinash Ravichandran, Subhransu Maji +3
Few-shot learning aims to transfer information from one task to enable generalization on novel tasks given a few examples. This information is present both in the domain and the cl…
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
Zhaowei Cai, Avinash Ravichandran, Subhransu Maji +3
We present a plug-in replacement for batch normalization (BN) called exponential moving average normalization (EMAN), which improves the performance of existing student-teacher bas…