Emergence of Pragmatics from Referential Game between Theory of Mind Agents

Abstract

Pragmatics studies how context can contribute to language meanings. In human communication, language is never interpreted out of context, and sentences can usually convey more information than their literal meanings. However, this mechanism is missing in most multi-agent systems, restricting the communication efficiency and the capability of human-agent interaction. In this paper, we propose an algorithm, using which agents can spontaneously learn the ability to "read between lines" without any explicit hand-designed rules. We integrate the theory of mind (ToM) in a cooperative multi-agent pedagogical situation and propose an adaptive reinforcement learning (RL) algorithm to develop a communication protocol. ToM is a profound cognitive science concept, claiming that people regularly reason about other's mental states, including beliefs, goals, and intentions, to obtain performance advantage in competition, cooperation or coalition. With this ability, agents consider language as not only messages but also rational acts reflecting others' hidden states. Our experiments demonstrate the advantage of pragmatic protocols over non-pragmatic protocols. We also show the teaching complexity following the pragmatic protocol empirically approximates to recursive teaching dimension (RTD).

References in corpus (10)

• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Counterfactual Multi-Agent Policy Gradients
• Multi-Agent Cooperation and the Emergence of (Natural) Language
• Monte Carlo Sampling Methods for Approximating Interactive POMDPs
• Compositional Obverter Communication Learning From Raw Visual Input
• Emergent Communication through Negotiation
• Reasoning About Pragmatics with Neural Listeners and Speakers
• Emergence of Communication in an Interactive World with Consistent Speakers
• Emergent Communication in a Multi-Modal, Multi-Step Referential Game

Cited by in corpus (5)

• Towards More Human-like AI Communication: A Review of Emergent Communication Research
• Emergent Language: A Survey and Taxonomy
• Model-Based Opponent Modeling
• Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability
• Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel