2.2k citations · 2.5k across the 23 of their papers we have counts for
6 papers · 1 filter
HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation
Hamish Ivison, Akshita Bhagia, Yizhong Wang +2
Recent NLP models have shown the remarkable ability to effectively generalise `zero-shot' to new tasks using only natural language instructions as guidance. However, many of these…
Does Self-Rationalization Improve Robustness to Spurious Correlations?
Alexis Ross, Matthew E. Peters, Ana Marasović
Rationalization is fundamental to human reasoning and learning. NLP models trained to produce rationales along with predictions, called self-rationalization models, have been inves…
Extracting Latent Steering Vectors from Pretrained Language Models
Nishant Subramani, Nivedita Suresh, Matthew E. Peters
Prior work on controllable text generation has focused on learning how to control language models through trainable decoding, smart-prompt design, or fine-tuning based on a desired…
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
Akari Asai, Mohammadreza Salehi, Matthew E. Peters +1
This work introduces a new multi-task, parameter-efficient language model (LM) tuning method that learns to transfer knowledge across different tasks via a mixture of soft prompts-…
Staged Training for Transformer Language Models
Sheng Shen, Pete Walsh, Kurt Keutzer +3
The current standard approach to scaling transformer language models trains each model size from a different random initialization. As an alternative, we consider a staged training…
Hyperdecoders: Instance-specific decoders for multi-task NLP
Hamish Ivison, Matthew E. Peters
We investigate input-conditioned hypernetworks for multi-tasking in NLP, generating parameter-efficient adaptations for a decoder using a hypernetwork conditioned on the output of…