71 citations · 179 across the 16 of their papers we have counts for
6 papers · 2 filters
-gram Is Back: Residual Learning of Neural Text Generation with -gram Language Model
Huayang Li, Deng Cai, Jin Xu +1
-gram language models (LM) have been largely superseded by neural LMs as the latter exhibits better performance. However, we find that -gram models can achieve satisfactory p…
Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation
Deng Cai, Xin Li, Jackie Chun-Sing Ho +2
We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR). Compared with the original textual input, AMR is a struct…
Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
Jin Xu, Xiaojiang Liu, Jianhao Yan +3
While large-scale neural language models, such as GPT2 and BART, have achieved impressive results on various text generation tasks, they tend to get stuck in undesirable sentence-l…
Linearizing Transformer with Key-Value Memory
Yizhe Zhang, Deng Cai
Efficient transformer variants with linear time complexity have been developed to mitigate the quadratic computational overhead of the vanilla transformer. Among them are low-rank…
A Survey on Retrieval-Augmented Text Generation
Huayang Li, Yixuan Su, Deng Cai +2
Recently, retrieval-augmented text generation attracted increasing attention of the computational linguistics community. Compared with conventional generation models, retrieval-aug…
Measuring and Reducing Model Update Regression in Structured Prediction for NLP
Deng Cai, Elman Mansimov, Yi-An Lai +3
Recent advance in deep learning has led to the rapid adoption of machine learning-based NLP models in a wide range of applications. Despite the continuous gain in accuracy, backwar…