Robust Multi-agent Communication via Multi-view Message Certification

Abstract

Many multi-agent scenarios require message sharing among agents to promote coordination, hastening the robustness of multi-agent communication when policies are deployed in a message perturbation environment. Major relevant works tackle this issue under specific assumptions, like a limited number of message channels would sustain perturbations, limiting the efficiency in complex scenarios. In this paper, we take a further step addressing this issue by learning a robust multi-agent communication policy via multi-view message certification, dubbed CroMAC. Agents trained under CroMAC can obtain guaranteed lower bounds on state-action values to identify and choose the optimal action under a worst-case deviation when the received messages are perturbed. Concretely, we first model multi-agent communication as a multi-view problem, where every message stands for a view of the state. Then we extract a certificated joint message representation by a multi-view variational autoencoder (MVAE) that uses a product-of-experts inference network. For the optimization phase, we do perturbations in the latent space of the state for a certificate guarantee. Then the learned joint message representation is used to approximate the certificated state representation during training. Extensive experiments in several cooperative multi-agent benchmarks validate the effectiveness of the proposed CroMAC.

References in corpus (31)

• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games
• A Survey of Multi-View Representation Learning
• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
• Cooperative Multi-Agent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
• Multi-Agent Reinforcement Learning is a Sequence Modeling Problem
• Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks
• Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificates
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
• Learning Nearly Decomposable Value Functions Via Communication Minimization
• ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mind
• Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
• Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing
• Robust Deep Reinforcement Learning through Adversarial Loss
• Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL
• Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks
• Robust Reinforcement Learning using Adversarial Populations
• Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning
• Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorization
• Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning
• Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning
• Towards a Standardised Performance Evaluation Protocol for Cooperative MARL
• Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems
• CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
• Robust Multi-Agent Reinforcement Learning with Social Empowerment for Coordination and Communication
• Multi-View Dreaming: Multi-View World Model with Contrastive Learning
• LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammates
• COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks