343 citations · 494 across the 18 of their papers we have counts for
10 papers · 1 filter
Benchmarking_Fast_Domain_Adaptation_for_Unsupervised_Speech_Units
Robin San Roman, Manel Khentout, Tu Anh Nguyen +3
Representation learning has attracted great atten- tion and managed to reach good performances as a pretraining method for downstream tasks or as a first step towards unsu- pervise…
Curriculum Learning with Adam: The Devil Is in the Wrong Details
Lucas Weber, Jaap Jumelet, Paul Michel +2
Curriculum learning (CL) posits that machine learning models -- similar to humans -- may learn more efficiently from data that match their current learning progress. However, CL me…
AANG: Automating Auxiliary Learning
Lucio M. Dery, Paul Michel, Mikhail Khodak +2
Auxiliary objectives, supplementary learning signals that are introduced to help aid learning on data-starved or highly complex end-tasks, are commonplace in machine learning. Whil…
Distributionally Robust Models with Parametric Likelihood Ratios
Paul Michel, Tatsunori Hashimoto, Graham Neubig
As machine learning models are deployed ever more broadly, it becomes increasingly important that they are not only able to perform well on their training distribution, but also yi…
Balancing Average and Worst-case Accuracy in Multitask Learning
Paul Michel, Sebastian Ruder, Dani Yogatama
When training and evaluating machine learning models on a large number of tasks, it is important to not only look at average task accuracy -- which may be biased by easy or redunda…
Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative
Lucio M. Dery, Paul Michel, Ameet Talwalkar +1
In most settings of practical concern, machine learning practitioners know in advance what end-task they wish to boost with auxiliary tasks. However, widely used methods for levera…
Examining and Combating Spurious Features under Distribution Shift
Chunting Zhou, Xuezhe Ma, Paul Michel +1
A central goal of machine learning is to learn robust representations that capture the causal relationship between inputs features and output labels. However, minimizing empirical…
Modeling the Second Player in Distributionally Robust Optimization
Paul Michel, Tatsunori Hashimoto, Graham Neubig
Distributionally robust optimization (DRO) provides a framework for training machine learning models that are able to perform well on a collection of related data distributions (th…
Weight Poisoning Attacks on Pre-trained Models
Keita Kurita, Paul Michel, Graham Neubig
Recently, NLP has seen a surge in the usage of large pre-trained models. Users download weights of models pre-trained on large datasets, then fine-tune the weights on a task of the…
Optimizing Data Usage via Differentiable Rewards
Xinyi Wang, Hieu Pham, Paul Michel +3
To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content…