5.2k citations · 10k across the 79 of their papers we have counts for
9 papers · 1 filter
Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity
Kurt Shuster, Jack Urbanek, Arthur Szlam +1
State-of-the-art dialogue models still often stumble with regards to factual accuracy and self-contradiction. Anecdotally, they have been observed to fail to maintain character ide…
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue
Leonard Adolphs, Kurt Shuster, Jack Urbanek +2
Large language models can produce fluent dialogue but often hallucinate factual inaccuracies. While retrieval-augmented models help alleviate this issue, they still face a difficul…
NormFormer: Improved Transformer Pretraining with Extra Normalization
Sam Shleifer, Jason Weston, Myle Ott
During pretraining, the Pre-LayerNorm transformer suffers from a gradient magnitude mismatch: gradients at early layers are much larger than at later layers. These issues can be al…
Hash Layers For Large Sparse Models
Stephen Roller, Sainbayar Sukhbaatar, Arthur Szlam +1
We investigate the training of sparse layers that use different parameters for different inputs based on hashing in large Transformer models. Specifically, we modify the feedforwar…
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Jing Xu, Arthur Szlam, Jason Weston
Despite recent improvements in open-domain dialogue models, state of the art models are trained and evaluated on short conversations with little context. In contrast, the long-term…
Internet-Augmented Dialogue Generation
Mojtaba Komeili, Kurt Shuster, Jason Weston
The largest store of continually updating knowledge on our planet can be accessed via internet search. In this work we study giving access to this information to conversational age…