113 citations · 164 across the 14 of their papers we have counts for
14 papers
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Rituraj Sharma, Tu Vu
Looped language models turn hidden states into runtime state: each state is decoded for prediction and fed back into future computation. This creates a basic supervision question:…
What Matters for Model Merging at Scale?
Prateek Yadav, Tu Vu, Jonathan Lai +4
Model merging aims to combine multiple expert models into a more capable single model, offering benefits such as reduced storage and serving costs, improved generalization, and sup…
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation
Tu Vu, Mohit Iyyer, Xuezhi Wang +8
Most large language models (LLMs) are trained once and never updated; thus, they lack the ability to dynamically adapt to our ever-changing world. In this work, we perform a detail…
Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models
Sheng Shen, Le Hou, Yanqi Zhou +17
Sparse Mixture-of-Experts (MoE) is a neural architecture design that can be utilized to add learnable parameters to Large Language Models (LLMs) without increasing inference cost.…
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
Shayne Longpre, Le Hou, Tu Vu +8
We study the design decisions of publicly available instruction tuning methods, and break down the development of Flan 2022 (Chung et al., 2022). Through careful ablation studies o…
Dialect-robust Evaluation of Generated Text
Jiao Sun, Thibault Sellam, Elizabeth Clark +6
Evaluation metrics that are not robust to dialect variation make it impossible to tell how well systems perform for many groups of users, and can even penalize systems for producin…
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
Tu Vu, Aditya Barua, Brian Lester +3
In this paper, we explore the challenging problem of performing a generative task in a target language when labeled data is only available in English, using summarization as a case…
Leveraging QA Datasets to Improve Generative Data Augmentation
Dheeraj Mekala, Tu Vu, Timo Schick +1
The ability of generative language models (GLMs) to generate text has improved considerably in the last few years, enabling their use for generative data augmentation. In this work…
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Tu Vu, Brian Lester, Noah Constant +2
There has been growing interest in parameter-efficient methods to apply pre-trained language models to downstream tasks. Building on the Prompt Tuning approach of Lester et al. (20…
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
Tu Vu, Minh-Thang Luong, Quoc V. Le +2
Despite their recent successes in tackling many NLP tasks, large-scale pre-trained language models do not perform as well in few-shot settings where only a handful of training exam…
Exploring and Predicting Transferability across NLP Tasks
Tu Vu, Tong Wang, Tsendsuren Munkhdalai +5
Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. Can fine-tuning these models on tasks other…
Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification
Tu Vu, Mohit Iyyer
While paragraph embedding models are remarkably effective for downstream classification tasks, what they learn and encode into a single vector remains opaque. In this paper, we inv…
Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment
Tu Vu, Vered Shwartz
Supervised distributional methods are applied successfully in lexical entailment, but recent work questioned whether these methods actually learn a relation between two words. Spec…
Sentence Simplification with Memory-Augmented Neural Networks
Tu Vu, Baotian Hu, Tsendsuren Munkhdalai +1
Sentence simplification aims to simplify the content and structure of complex sentences, and thus make them easier to interpret for human readers, and easier to process for downstr…