98 citations · 122 across the 7 of their papers we have counts for
8 papers
The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation
Kushal Arora, Timothy J. O'Donnell, Doina Precup +2
State-of-the-art language generation models can degenerate when applied to open-ended generation problems such as text completion, story generation, or dialog modeling. This degene…
Lexi: Self-Supervised Learning of the UI Language
Pratyay Banerjee, Shweti Mahajan, Kushal Arora +2
Humans can learn to operate the user interface (UI) of an application by reading an instruction manual or how-to guide. Along with text, these resources include visual content such…
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback
Jing Xu, Megan Ung, Mojtaba Komeili +3
Frozen models trained to mimic static datasets can never improve their performance. Models that can employ internet-retrieval for up-to-date information and obtain feedback from hu…
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
Kurt Shuster, Jing Xu, Mojtaba Komeili +15
We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large…
DIRECTOR: Generator-Classifiers For Supervised Language Modeling
Kushal Arora, Kurt Shuster, Sainbayar Sukhbaatar +1
Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions. The standard language modeling…
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
Kushal Arora, Layla El Asri, Hareesh Bahuleyan +1
Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis is that this brittleness of generation model…
A Compositional Approach to Language Modeling
Kushal Arora, Anand Rangarajan
Traditional language models treat language as a finite state automaton on a probability space over words. This is a very strong assumption when modeling something inherently comple…
Contrastive Entropy: A new evaluation metric for unnormalized language models
Kushal Arora, Anand Rangarajan
Perplexity (per word) is the most widely used metric for evaluating language models. Despite this, there has been no dearth of criticism for this metric. Most of these criticisms c…