Calibrate your listeners! Robust communication-based training for pragmatic speakers

Abstract

To be good conversational partners, natural language processing (NLP) systems should be trained to produce contextually useful utterances. Prior work has investigated training NLP systems with communication-based objectives, where a neural listener stands in as a communication partner. However, these systems commonly suffer from semantic drift where the learned language diverges radically from natural language. We propose a method that uses a population of neural listeners to regularize speaker training. We first show that language drift originates from the poor uncertainty calibration of a neural listener, which makes high-certainty predictions on novel sentences. We explore ensemble- and dropout-based populations of listeners and find that the former results in better uncertainty quantification. We evaluate both population-based objectives on reference games, and show that the ensemble method with better calibration enables the speaker to generate pragmatic utterances while scaling to a large vocabulary and generalizing to new games and listeners.

arXiv commentFindings of EMNLP 2021 Code: https://github.com/rosewang2008/calibrate_your_listeners

References in corpus (6)

• Categorical Reparameterization with Gumbel-Softmax
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Fine-Tuning Language Models from Human Preferences
• Multi-Agent Cooperation and the Emergence of (Natural) Language
• ShapeWorld - A new test methodology for multimodal language understanding
• Shaping representations through communication: community size effect in artificial learning systems