63 citations · 100 across the 14 of their papers we have counts for
11 papers · 1 filter
Task Adaptive Parameter Sharing for Multi-Task Learning
Matthew Wallingford, Hao Li, Alessandro Achille +4
Adapting pre-trained models with broad capabilities has become standard practice for learning a wide range of downstream tasks. The typical approach of fine-tuning different models…
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning
Zhaowei Cai, Avinash Ravichandran, Subhransu Maji +3
We present a plug-in replacement for batch normalization (BN) called exponential moving average normalization (EMAN), which improves the performance of existing student-teacher bas…
Estimating informativeness of samples with Smooth Unique Information
Hrayr Harutyunyan, Alessandro Achille, Giovanni Paolini +4
We define a notion of information that an individual sample provides to the training of a neural network, and we specialize it to measure both how much a sample informs the final w…
LQF: Linear Quadratic Fine-Tuning
Alessandro Achille, Aditya Golatkar, Avinash Ravichandran +2
Classifiers that are linear in their parameters, and trained by optimizing a convex loss function, have predictable behavior with respect to changes in the training data, initial c…
Mixed-Privacy Forgetting in Deep Networks
Aditya Golatkar, Alessandro Achille, Avinash Ravichandran +2
We show that the influence of a subset of the training samples can be removed -- or "forgotten" -- from the weights of a network trained on large-scale image classification tasks,…
Predicting Training Time Without Training
Luca Zancato, Alessandro Achille, Avinash Ravichandran +2
We tackle the problem of predicting the number of optimization steps that a pre-trained deep network needs to converge to a given value of the loss function. To do so, we leverage…