33 citations · 59 across the 5 of their papers we have counts for
3 papers · 1 filter
A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration
Shaojie Jiang, Ruqing Zhang, Svitlana Vakulenko +1
The cross-entropy objective has proved to be an all-purpose training objective for autoregressive language models (LMs). However, without considering the penalization of problemati…
TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation
Shaojie Jiang, Thomas Wolf, Christof Monz +1
Natural Language Generation (NLG) models are prone to generating repetitive utterances. In this work, we study the repetition problem for encoder-decoder models, using both recurre…
Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots
Shaojie Jiang, Maarten de Rijke
Diversity is a long-studied topic in information retrieval that usually refers to the requirement that retrieved results should be non-repetitive and cover different aspects. In a…