84 citations · 141 across the 9 of their papers we have counts for
9 papers
FlashTrie: A GPU-Accelerated Constrained Beam Search for Generative Retrieval
Dakshitha Anandakumar, Anurag Mukkara, Wenxiang Hu +5
Constrained decoding is essential in generative retrieval, where document identifiers generated directly from a query must exactly match a predefined library of valid IDs. At scale…
FastSeq: Make Sequence Generation Faster
Yu Yan, Fei Hu, Jiusheng Chen +7
Transformer-based models have made tremendous impacts in natural language generation. However the inference speed is a bottleneck due to large model size and intensive computing in…
EL-Attention: Memory Efficient Lossless Attention for Generation
Yu Yan, Jiusheng Chen, Weizhen Qi +4
Transformer model with multi-head attention requires caching intermediate results for efficient inference in generation tasks. However, cache brings new memory-related costs and pr…
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
Weizhen Qi, Yeyun Gong, Yu Yan +9
Now, the pre-training technique is ubiquitous in natural language processing field. ProphetNet is a pre-training based natural language generation method which shows powerful perfo…
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
Weizhen Qi, Yeyun Gong, Jian Jiao +9
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation. AR and NAR generation can be uniforml…
GLGE: A New General Language Generation Evaluation Benchmark
Dayiheng Liu, Yu Yan, Yeyun Gong +15
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP). These benchmarks mostly focus…
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
Dayiheng Liu, Yeyun Gong, Jie Fu +5
In this paper, we propose a novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension (MRC),…
RikiNet: Reading Wikipedia Pages for Natural Question Answering
Dayiheng Liu, Yeyun Gong, Jie Fu +5
Reading long documents to answer open-domain questions remains challenging in natural language understanding. In this paper, we introduce a new model, called RikiNet, which reads W…
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
Weizhen Qi, Yu Yan, Yeyun Gong +5
This paper presents a new sequence-to-sequence pre-training model called ProphetNet, which introduces a novel self-supervised objective named future n-gram prediction and the propo…