activity
20172026
most citedLinformer: Self-Attention with Linear Complexity

887 citations · 1.3k across the 14 of their papers we have counts for

collaborators
Showing 2021Show all

8 papers · 1 filter

cs.LG2021

RID-Noise: Towards Robust Inverse Design under Noisy Environments

Jia-Qi Yang, Ke-Bin Fan, Hao Ma +1

From an engineering perspective, a design should not only perform well in an ideal condition, but should also resist noises. Such a design methodology, namely robust design, has be…

cs.CL2021

Reducing Target Group Bias in Hate Speech Detectors

Darsh J Shah, Sinong Wang, Han Fang +2

The ubiquity of offensive and hateful content on online fora necessitates the need for automatic solutions that detect such content competently across target groups. In this paper…

cs.CL2021★ 21 cited

UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

Yuning Mao, Lambert Mathias, Rui Hou +5

Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when…

cs.LG2021★ 49 cited

Luna: Linear Unified Nested Attention

Xuezhe Ma, Xiang Kong, Sinong Wang +4

The quadratic computational and memory complexities of the Transformer's attention mechanism have limited its scalability for modeling long sequences. In this paper, we propose Lun…

cs.CL2021★ 106 cited

Entailment as Few-Shot Learner

Sinong Wang, Han Fang, Madian Khabsa +2

Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners. However, their success hinges largely on scaling model parameters to a degree tha…

cs.AI2021

On Unifying Misinformation Detection

Nayeon Lee, Belinda Z. Li, Sinong Wang +4

In this paper, we introduce UnifiedM2, a general-purpose misinformation model that jointly models multiple domains of misinformation with a single, unified setup. The model is trai…