1.2k citations · 2.7k across the 27 of their papers we have counts for
4 papers · 1 filter
Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover Distance
Xi Chen, Nan Ding, Tomer Levinboim +1
Recent advances in automatic evaluation metrics for text have shown that deep contextualized word representations, such as those generated by BERT encoders, are helpful for designi…
TeaForN: Teacher-Forcing with N-grams
Sebastian Goodman, Nan Ding, Radu Soricut
Sequence generation models trained with teacher-forcing suffer from issues related to exposure bias and lack of differentiability across timesteps. Our proposed method, Teacher-For…
Attention that does not Explain Away
Nan Ding, Xinjie Fan, Zhenzhong Lan +2
Models based on the Transformer architecture have achieved better accuracy than the ones based on competing architectures for a large set of tasks. A unique feature of the Transfor…
Talking-Heads Attention
Noam Shazeer, Zhenzhong Lan, Youlong Cheng +2
We introduce "talking-heads attention" - a variation on multi-head attention which includes linearprojections across the attention-heads dimension, immediately before and after the…