77 citations · 232 across the 13 of their papers we have counts for
15 papers
GenEARL: A Training-Free Generative Framework for Multimodal Event Argument Role Labeling
Hritik Bansal, Po-Nien Kung, P. Jeffrey Brantingham +2
Multimodal event argument role labeling (EARL), a task that assigns a role for each event participant (object) in an image is a complex challenge. It requires reasoning over the en…
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark
Momchil Hardalov, Pepa Atanasova, Todor Mihaylov +7
We present bgGLUE(Bulgarian General Language Understanding Evaluation), a benchmark for evaluating language models on Natural Language Understanding (NLU) tasks in Bulgarian. Our b…
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
Zeyu Leo Liu, Tim Dettmers, Xi Victoria Lin +2
Large and sparse feed-forward layers (S-FFN) such as Mixture-of-Experts (MoE) have proven effective in scaling up Transformers model size for \textit{pretraining} large language mo…
LEVER: Learning to Verify Language-to-Code Generation with Execution
Ansong Ni, Srini Iyer, Dragomir Radev +4
The advent of large language models trained on code (code LLMs) has led to significant progress in language-to-code generation. State-of-the-art approaches in this area combine LLM…
Complementary Explanations for Effective In-Context Learning
Xi Ye, Srinivasan Iyer, Asli Celikyilmaz +3
Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts, but there has been limited understanding of exactly how these explanat…
Improving In-Context Few-Shot Learning via Self-Supervised Training
Mingda Chen, Jingfei Du, Ramakanth Pasunuru +4
Self-supervised pretraining has made few-shot learning possible for many NLP tasks. But the pretraining objectives are not typically adapted specifically for in-context few-shot le…
On the Role of Bidirectionality in Language Model Pre-Training
Mikel Artetxe, Jingfei Du, Naman Goyal +2
Prior work on language model pre-training has explored different architectures and learning objectives, but differences in data, hyperparameters and evaluation make a principled co…
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models
Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson +4
Current methods for few-shot fine-tuning of pretrained masked language models (PLMs) require carefully engineered prompts and verbalizers for each new task to convert examples into…
Efficient Language Modeling with Sparse all-MLP
Ping Yu, Mikel Artetxe, Myle Ott +4
All-MLP architectures have attracted increasing interest as an alternative to attention-based models. In NLP, recent work like gMLP shows that all-MLPs can match Transformers in la…
Efficient Large Scale Language Modeling with Mixtures of Experts
Mikel Artetxe, Shruti Bhosale, Naman Goyal +21
Mixture of Experts layers (MoEs) enable efficient scaling of language models through conditional computation. This paper presents a detailed empirical study of how autoregressive M…
Few-shot Learning with Multilingual Language Models
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe +18
Large-scale generative language models such as GPT-3 are competitive few-shot learners. While these models are known to be able to jointly represent many different languages, their…
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
Beliz Gunel, Jingfei Du, Alexis Conneau +1
State-of-the-art natural language understanding classification models follow two-stages: pre-training a large language model on an auxiliary task, and then fine-tuning the model on…
Self-training Improves Pre-training for Natural Language Understanding
Jingfei Du, Edouard Grave, Beliz Gunel +5
Unsupervised pre-training has led to much recent progress in natural language understanding. In this paper, we study self-training as another way to leverage unlabeled data through…
Conversational Semantic Parsing
Armen Aghajanyan, Jean Maillard, Akshat Shrivastava +8
The structured representation for semantic parsing in task-oriented assistant systems is geared towards simple understanding of one-turn queries. Due to the limitations of the repr…
Preserving Integrity in Online Social Networks
Alon Halevy, Cristian Canton Ferrer, Hao Ma +5
Online social networks provide a platform for sharing information and free expression. However, these networks are also used for malicious purposes, such as distributing misinforma…