Emergence of Grounded Compositional Language in Multi-Agent Populations
arXiv:1703.04908
Abstract
By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis. However, for agents to intelligently interact with humans, simply capturing the statistical patterns is insufficient. In this paper we investigate if, and how, grounded compositional language can emerge as a means to achieve goals in multi-agent populations. Towards this end, we propose a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language. This language is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax. We also observe emergence of non-verbal communication such as pointing and guiding when language communication is unavailable.
Cited by in corpus (99)
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- Emergent Tool Use From Multi-Agent Autocurricula
- Learning Attentional Communication for Multi-Agent Cooperation
- Actor-Attention-Critic for Multi-Agent Reinforcement Learning
- Survey on reinforcement learning for language processing
- Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics
- Learning Policy Representations in Multiagent Systems
- Compositional Obverter Communication Learning From Raw Visual Input
- InfoBot: Transfer and Exploration via the Information Bottleneck
- Off-Policy Multi-Agent Decomposed Policy Gradients
- CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning
- Learning Efficient Multi-agent Communication: An Information Bottleneck Approach
- Compositional Languages Emerge in a Neural Iterated Learning Model
- Emergence of Compositional Language with Deep Generational Transmission
- On the Critical Role of Conventions in Adaptive Human-AI Collaboration
- Emergent Multi-Agent Communication in the Deep Learning Era
- On the interaction between supervision and self-play in emergent communication
- Optimizing Online Matching for Ride-Sourcing Services with Multi-Agent Deep Reinforcement Learning
- ROMA: Multi-Agent Reinforcement Learning with Emergent Roles
- Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordination
- One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control
- Learning Nearly Decomposable Value Functions Via Communication Minimization
- MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shielding
- Forbidden knowledge in machine learning -- Reflections on the limits of research and publication
- Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning
- The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning
- Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations
- Multi-Agent Collaboration via Reward Attribution Decomposition
- Collaborative Visual Navigation
- BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
- Delay-Aware Multi-Agent Reinforcement Learning for Cooperative and Competitive Environments
- Ease-of-Teaching and Language Structure from Emergent Communication
- Multimodal Grounding for Language Processing
- Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks
- Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning
- Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
- Learning Latent Representations to Influence Multi-Agent Interaction
- Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning
- Learning Multi-agent Communication under Limited-bandwidth Restriction for Internet Packet Routing
- Networked Multi-Agent Reinforcement Learning with Emergent Communication
- Multi-Agent Reinforcement Learning with Multi-Step Generative Models
- Multi-Agent Trust Region Policy Optimization
- Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spaces
- Emergence of Theory of Mind Collaboration in Multiagent Systems
- Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence
- HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
- Measuring non-trivial compositionality in emergent communication
- Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
- Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic
- The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget
- Learning Autocomplete Systems as a Communication Game
- Learning to cooperate: Emergent communication in multi-agent navigation
- Adversarially Guided Self-Play for Adopting Social Conventions
- Emergent Graphical Conventions in a Visual Communication Game
- Compositional properties of emergent languages in deep learning
- Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI
- Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
- Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game
- Agent Modelling under Partial Observability for Deep Reinforcement Learning
- Rat big, cat eaten! Ideas for a useful deep-agent protolanguage
- A Practical Guide to Studying Emergent Communication through Grounded Language Games
- Emergence of Numeric Concepts in Multi-Agent Autonomous Communication
- Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning
- Inter-Level Cooperation in Hierarchical Reinforcement Learning
- A perspective on multi-agent communication for information fusion
- Modeling Conceptual Understanding in Image Reference Games
- Correcting Experience Replay for Multi-Agent Communication
- Designing a Multi-Objective Reward Function for Creating Teams of Robotic Bodyguards Using Deep Reinforcement Learning
- Learning to refer informatively by amortizing pragmatic reasoning
- Structural Inductive Biases in Emergent Communication
- Winning an Election: On Emergent Strategic Communication in Multi-Agent Networks
- Towards Graph Representation Learning in Emergent Communication
- Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations
- Interactive Agent Modeling by Learning to Probe
- SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents
- Learning Altruistic Behaviours in Reinforcement Learning without External Rewards
- Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environments
- Zero-Shot Generalization using Intrinsically Motivated Compositional Emergent Protocols
- Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following
- AGENT: A Benchmark for Core Psychological Reasoning
- Compositionality Through Language Transmission, using Artificial Neural Networks
- Paying Attention to Function Words
- Focus on What's Informative and Ignore What's not: Communication Strategies in a Referential Game
- Learning to Request Guidance in Emergent Communication
- Generalizing Emergent Communication
- A Dynamics Perspective of Pursuit-Evasion Games of Intelligent Agents with the Ability to Learn
- PatchGame: Learning to Signal Mid-level Patches in Referential Games
- Agent Probing Interaction Policies
- Learning Compositional Negation in Populations of Roth-Erev and Neural Agents
- Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel
- The Power of Communication in a Distributed Multi-Agent System
- Emergent Communication with World Models
- TexRel: a Green Family of Datasets for Emergent Communications on Relations
- Planning, Inference and Pragmatics in Sequential Language Games
- Influence-Based Reinforcement Learning for Intrinsically-Motivated Agents
- The Emergence of the Shape Bias Results from Communicative Efficiency
- On Memory Mechanism in Multi-Agent Reinforcement Learning