106 citations · 171 across the 12 of their papers we have counts for
15 papers
High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning
Tim Franzmeyer, Archie Sravankumar, Lijuan Liu +6
Large Language Models (LLMs) currently respond to every prompt. However, they can produce incorrect answers when they lack knowledge or capability -- a problem known as hallucinati…
Preference Optimization with Multi-Sample Comparisons
Chaoqi Wang, Zhuokai Zhao, Chen Zhu +8
Recent advancements in generative models, particularly large language models (LLMs) and diffusion models, have been driven by extensive pretraining on large datasets followed by po…
Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefixes
Kuan-Hao Huang, Liang Tan, Rui Hou +3
Many real-world applications require making multiple predictions from the same text. Fine-tuning a large pre-trained language model for each downstream task causes computational bu…
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning
Ajinkya Tejankar, Maziar Sanjabi, Qifan Wang +4
Recently, self-supervised learning (SSL) was shown to be vulnerable to patch-based data poisoning backdoor attacks. It was shown that an adversary can poison a small part of the un…
Representation Deficiency in Masked Language Modeling
Yu Meng, Jitin Krishnan, Sinong Wang +6
Masked Language Modeling (MLM) has been one of the most prominent approaches for pretraining bidirectional text encoders due to its simplicity and effectiveness. One notable concer…
Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler
Yifang Chen, Karthik Sankararaman, Alessandro Lazaric +6
Active learning with strong and weak labelers considers a practical setting where we have access to both costly but accurate strong labelers and inaccurate but cheap predictions pr…
BayesFormer: Transformer with Uncertainty Estimation
Karthik Abinav Sankararaman, Sinong Wang, Han Fang
Transformer has become ubiquitous due to its dominant performance in various NLP and image processing tasks. However, it lacks understanding of how to generate mathematically groun…
Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem
Khalil Mrini, Shaoliang Nie, Jiatao Gu +3
We propose an autoregressive entity linking model, that is trained with two auxiliary tasks, and learns to re-rank generated samples at inference time. Our proposed novelties addre…
IDPG: An Instance-Dependent Prompt Generation Method
Zhuofeng Wu, Sinong Wang, Jiatao Gu +4
Prompt tuning is a new, efficient NLP transfer learning paradigm that adds a task-specific prompt in each input instance during the model training stage. It freezes the pre-trained…
Reducing Target Group Bias in Hate Speech Detectors
Darsh J Shah, Sinong Wang, Han Fang +2
The ubiquity of offensive and hateful content on online fora necessitates the need for automatic solutions that detect such content competently across target groups. In this paper…
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models
Qinyuan Ye, Madian Khabsa, Mike Lewis +3
Distilling state-of-the-art transformer models into lightweight student models is an effective way to reduce computation cost at inference time. The student models are typically co…
Luna: Linear Unified Nested Attention
Xuezhe Ma, Xiang Kong, Sinong Wang +4
The quadratic computational and memory complexities of the Transformer's attention mechanism have limited its scalability for modeling long sequences. In this paper, we propose Lun…
Entailment as Few-Shot Learner
Sinong Wang, Han Fang, Madian Khabsa +2
Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners. However, their success hinges largely on scaling model parameters to a degree tha…
On Unifying Misinformation Detection
Nayeon Lee, Belinda Z. Li, Sinong Wang +4
In this paper, we introduce UnifiedM2, a general-purpose misinformation model that jointly models multiple domains of misinformation with a single, unified setup. The model is trai…
On the Influence of Masking Policies in Intermediate Pre-training
Qinyuan Ye, Belinda Z. Li, Sinong Wang +5
Current NLP models are predominantly trained through a two-stage "pre-train then fine-tune" pipeline. Prior work has shown that inserting an intermediate pre-training stage, using…