343 citations · 1.7k across the 69 of their papers we have counts for
23 papers · 1 filter
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
Fei Yuan, Yinquan Lu, WenHao Zhu +4
Multilingual neural machine translation (MNMT) aims to build a unified model for many language directions. Existing monolithic models for MNMT encounter two challenges: parameter i…
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering
Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye +1
Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context. This paper advocates a new…
Explanation Regeneration via Information Bottleneck
Qintong Li, Zhiyong Wu, Lingpeng Kong +1
Explaining the black-box predictions of NLP models naturally and accurately is an important open problem in natural language generation. These free-text explanations are expected t…
Unsupervised Explanation Generation via Correct Instantiations
Sijie Cheng, Zhiyong Wu, Jiangjie Chen +3
While large pre-trained language models (PLM) have shown their great skills at solving discriminative tasks, a significant gap remains when compared with humans for explanation-rel…
An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks
Changlong Yu, Tianyi Xiao, Lingpeng Kong +2
Though linguistic knowledge emerges during large-scale language model pretraining, recent work attempt to explicitly incorporate human-defined linguistic priors into task-specific…
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Jiacheng Ye, Jiahui Gao, Jiangtao Feng +3
Recently, dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language model…
The Devil in Linear Transformer
Zhen Qin, XiaoDong Han, Weixuan Sun +4
Linear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus.…
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Shansan Gong, Mukai Li, Jiangtao Feng +2
Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusio…
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling
Jun Zhang, Shuyang Jiang, Jiangtao Feng +2
Transformer has achieved remarkable success in language, image, and speech processing. Recently, various efficient attention architectures have been proposed to improve transformer…
Audio-Visual Segmentation
Jinxing Zhou, Jianyuan Wang, Jiayi Zhang +7
We propose to explore a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the…
Vicinity Vision Transformer
Weixuan Sun, Zhen Qin, Hui Deng +7
Vision transformers have shown great success on numerous computer vision tasks. However, its central component, softmax attention, prohibits vision transformers from scaling up to…
Language Models Can See: Plugging Visual Controls in Text Generation
Yixuan Su, Tian Lan, Yahui Liu +5
Generative language models (LMs) such as GPT-2/3 can be prompted to generate text with remarkable quality. While they are designed for text-prompted generation, it remains an open…
Lexical Knowledge Internalization for Neural Dialog Generation
Zhiyong Wu, Wei Bi, Xiang Li +2
We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models. Instead of further conditioning the knowledge-grounded dialog (…
CoNT: Contrastive Neural Text Generation
Chenxin An, Jiangtao Feng, Kai Lv +3
Recently, contrastive learning attracts increasing interests in neural text generation as a new solution to alleviate the exposure bias problem. It introduces a sequence-level trai…
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
Jiahui Gao, Renjie Pi, Yong Lin +7
There is a rising interest in further exploring the zero-shot learning potential of large pre-trained language models (PLMs). A new paradigm called data-generation-based zero-shot…
Event Transition Planning for Open-ended Text Generation
Qintong Li, Piji Li, Wei Bi +3
Open-ended text generation tasks, such as dialogue generation and story completion, require models to generate a coherent continuation given limited preceding context. The open-end…
Linear Complexity Randomized Self-attention Mechanism
Lin Zheng, Chong Wang, Lingpeng Kong
Recently, random feature attentions (RFAs) are proposed to approximate the softmax attention in linear time and space complexity by linearizing the exponential kernel. In this pape…
SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples
Hao Wang, Yangguang Li, Zhen Huang +3
Unsupervised sentence embedding aims to obtain the most appropriate embedding for a sentence to reflect its semantic. Contrastive learning has been attracting developing attention.…
cosFormer: Rethinking Softmax in Attention
Zhen Qin, Weixuan Sun, Hui Deng +6
Transformer has shown great successes in natural language processing, computer vision, and audio processing. As one of its core components, the softmax attention helps to capture l…
Revisiting Over-smoothing in BERT from the Perspective of Graph
Han Shi, Jiahui Gao, Hang Xu +5
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields. However, no existing work has delved deeper to further investigate th…
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
Jiacheng Ye, Jiahui Gao, Qintong Li +5
There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs). In this paper, we study a flexible a…
A Contrastive Framework for Neural Text Generation
Yixuan Su, Tian Lan, Yan Wang +3
Text generation is of great importance to many natural language processing applications. However, maximization-based decoding methods (e.g. beam search) of neural language models o…
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Tianbao Xie, Chen Henry Wu, Peng Shi +20
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.…