282 citations · 334 across the 4 of their papers we have counts for
8 papers
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning
Prasetya Ajie Utama, Nafise Sadat Moosavi, Victor Sanh +1
Recent prompt-based approaches allow pretrained language models to achieve strong performances on few-shot finetuning by reformulating downstream tasks as a language modeling probl…
Low-Complexity Probing via Finding Subnetworks
Steven Cao, Victor Sanh, Alexander M. Rush
The dominant approach in probing neural networks for linguistic properties is to train a new shallow multi-layer perceptron (MLP) on top of the model's internal representations. Th…
Learning from others' mistakes: Avoiding dataset biases without modeling them
Victor Sanh, Thomas Wolf, Yonatan Belinkov +1
State-of-the-art natural language processing (NLP) models often learn to model dataset biases and surface form correlations instead of features that target the intended underlying…
Movement Pruning: Adaptive Sparsity by Fine-Tuning
Victor Sanh, Thomas Wolf, Alexander M. Rush
Magnitude pruning is a widely used strategy for reducing model size in pure supervised learning; however, it is less effective in the transfer learning regime that has become stand…
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Thomas Wolf, Lysandre Debut, Victor Sanh +19
Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building hig…
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond +1
As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under const…