94 citations · 251 across the 22 of their papers we have counts for
22 papers
AMix-2: Establishing Protein as a Native Modality in Large Language Models
Keyue Qiu, Yixin Wu, Lihao Wang +19
We present AMix-2, a protein-text foundation model that establishes protein as a native modality in large language models (LLMs), unifying protein understanding and sequence design…
Linear Attention via Orthogonal Memory
Jun Zhang, Shuyang Jiang, Jiangtao Feng +2
Efficient attentions have greatly improved the computational efficiency of Transformers. However, most existing linear attention mechanisms suffer from an \emph{efficiency degradat…
DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models
Shansan Gong, Mukai Li, Jiangtao Feng +2
Diffusion models have gained prominence in generating high-quality sequences of text. Nevertheless, current approaches predominantly represent discrete text within a continuous dif…
Attentive Multi-Layer Perceptron for Non-autoregressive Generation
Shuyang Jiang, Jun Zhang, Jiangtao Feng +2
Autoregressive~(AR) generation almost dominates sequence generation for its efficacy. Recently, non-autoregressive~(NAR) generation gains increasing popularity for its efficiency a…
Optimizing Non-Autoregressive Transformers with Contrastive Learning
Chenxin An, Jiangtao Feng, Fei Huang +2
Non-autoregressive Transformers (NATs) reduce the inference latency of Autoregressive Transformers (ATs) by predicting words all at once rather than in sequential order. They have…
OpenICL: An Open-Source Framework for In-context Learning
Zhenyu Wu, YaoXiang Wang, Jiacheng Ye +4
In recent years, In-context Learning (ICL) has gained increasing attention and emerged as the new paradigm for large language model (LLM) evaluation. Unlike traditional fine-tuning…
In-Context Learning with Many Demonstration Examples
Mukai Li, Shansan Gong, Jiangtao Feng +4
Large pre-training language models (PLMs) have shown promising in-context learning abilities. However, due to the backbone transformer architecture, existing PLMs are bottlenecked…
Compositional Exemplars for In-context Learning
Jiacheng Ye, Zhiyong Wu, Jiangtao Feng +2
Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL) ability, where the model learns to do an unseen task via a prompt consisting of input-output…
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Jiacheng Ye, Jiahui Gao, Jiangtao Feng +3
Recently, dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language model…
PARAGEN : A Parallel Generation Toolkit
Jiangtao Feng, Yi Zhou, Jun Zhang +7
PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation. PARAGEN provides thirteen types of customizable plugins, helping users to experiment quickly…
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Shansan Gong, Mukai Li, Jiangtao Feng +2
Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusio…
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling
Jun Zhang, Shuyang Jiang, Jiangtao Feng +2
Transformer has achieved remarkable success in language, image, and speech processing. Recently, various efficient attention architectures have been proposed to improve transformer…
CoNT: Contrastive Neural Text Generation
Chenxin An, Jiangtao Feng, Kai Lv +3
Recently, contrastive learning attracts increasing interests in neural text generation as a new solution to alleviate the exposure bias problem. It introduces a sequence-level trai…
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
Jiacheng Ye, Jiahui Gao, Qintong Li +5
There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs). In this paper, we study a flexible a…
Learning Logic Rules for Document-level Relation Extraction
Dongyu Ru, Changzhi Sun, Jiangtao Feng +5
Document-level relation extraction aims to identify relations between entities in a whole document. Prior efforts to capture long-range dependencies have relied heavily on implicit…
The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21
Lihua Qian, Yi Zhou, Zaixiang Zheng +7
This paper describes the Volctrans' submission to the WMT21 news translation shared task for German->English translation. We build a parallel (i.e., non-autoregressive) translation…
Counter-Interference Adapter for Multilingual Machine Translation
Yaoming Zhu, Jiangtao Feng, Chengqi Zhao +2
Developing a unified multilingual model has long been a pursuit for machine translation. However, existing approaches suffer from performance degradation -- a single multilingual m…
Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks
Liping Yuan, Jiangtao Feng, Xiaoqing Zheng +1
A popular strategy to train recurrent neural networks (RNNs), known as ``teacher forcing'' takes the ground truth as input at each time step and makes the later predictions partly…
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information
Zehui Lin, Xiao Pan, Mingxuan Wang +4
We investigate the following question for machine translation (MT): can we develop a single universal MT model to serve as the common seed and obtain derivative and improved models…
Non-autoregressive Transformer by Position Learning
Yu Bao, Hao Zhou, Jiangtao Feng +4
Non-autoregressive models are promising on various text generation tasks. Previous work hardly considers to explicitly model the positions of generated words. However, position mod…
Cross-Lingual Vision-Language Navigation
An Yan, Xin Eric Wang, Jiangtao Feng +2
Commanding a robot to navigate with natural language instructions is a long-term goal for grounded language understanding and robotics. But the dominant language is English, accord…
Neural Phrase-to-Phrase Machine Translation
Jiangtao Feng, Lingpeng Kong, Po-Sen Huang +5
In this paper, we propose Neural Phrase-to-Phrase Machine Translation (NPMT). Our model uses a phrase attention mechanism to discover relevant input (source) segments that are…