623 citations · 1.1k across the 25 of their papers we have counts for
7 papers · 1 filter
Contrastive Distillation Is a Sample-Efficient Self-Supervised Loss Policy for Transfer Learning
Chris Lengerich, Gabriel Synnaeve, Amy Zhang +4
Traditional approaches to RL have focused on learning decision policies directly from episodic decisions, while slowly and implicitly learning the semantics of compositional repres…
The CRINGE Loss: Learning what language not to model
Leonard Adolphs, Tianyu Gao, Jing Xu +3
Standard language model training employs gold human documents or human-human interaction data, and treats all training data as positive examples. Growing evidence shows that even w…
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels
Weiyan Shi, Emily Dinan, Kurt Shuster +2
Deployed dialogue agents have the potential to integrate human feedback to continuously improve themselves. However, humans may not always provide explicit signals when the chatbot…
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
Kurt Shuster, Jing Xu, Mojtaba Komeili +15
We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large…
OPT: Open Pre-trained Transformer Language Models
Susan Zhang, Stephen Roller, Naman Goyal +16
Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computatio…
DIRECTOR: Generator-Classifiers For Supervised Language Modeling
Kushal Arora, Kurt Shuster, Sainbayar Sukhbaatar +1
Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions. The standard language modeling…