313 citations · 1.8k across the 83 of their papers we have counts for
24 papers · 1 filter
A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning
Xidong Feng, Bo Liu, Jie Ren +5
Gradient-based Meta-RL (GMRL) refers to methods that maintain two-level optimisation procedures wherein the outer-loop meta-learner guides the inner-loop gradient-based reinforceme…
Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks
Linghui Meng, Muning Wen, Yaodong Yang +7
Offline reinforcement learning leverages previously-collected offline datasets to learn optimal policies with no necessity to access the real environment. Such a paradigm is also d…
LIGS: Learnable Intrinsic-Reward Generation Selection for Multi-Agent Learning
David Henry Mguni, Taher Jafferjee, Jianhong Wang +7
Efficient exploration is important for reinforcement learners to achieve high rewards. In multi-agent systems, coordinated exploration and behaviour is critical for agents to joint…
Measuring the Non-Transitivity in Chess
Ricky Sanjaya, Jun Wang, Yaodong Yang
It has long been believed that Chess is the \emph{Drosophila} of Artificial Intelligence (AI). Studying Chess can productively provide valid knowledge about complex systems. Althou…
A Game-Theoretic Approach for Improving Generalization Ability of TSP Solvers
Chenguang Wang, Yaodong Yang, Oliver Slumbers +4
In this paper, we introduce a two-player zero-sum framework between a trainable \emph{Solver} and a \emph{Data Generator} to improve the generalization ability of deep learning-bas…
DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Intervention
David Mguni, Usman Islam, Yaqi Sun +7
Reinforcement learning (RL) involves performing exploratory actions in an unknown system. This can place a learning agent in dangerous and potentially catastrophic system states. C…
Online Markov Decision Processes with Non-oblivious Strategic Adversary
Le Cong Dinh, David Henry Mguni, Long Tran-Thanh +2
We study a novel setting in Online Markov Decision Processes (OMDPs) where the loss function is chosen by a non-oblivious strategic adversary who follows a no-external regret algor…
Multi-Agent Constrained Policy Optimisation
Shangding Gu, Jakub Grudzien Kuba, Munning Wen +6
Developing reinforcement learning algorithms that satisfy safety constraints is becoming increasingly important in real-world applications. In multi-agent reinforcement learning (M…
Revisiting the Characteristics of Stochastic Gradient Noise and Dynamics
Yixin Wu, Rui Luo, Chen Zhang +2
In this paper, we characterize the noise of stochastic gradients and analyze the noise-induced dynamics during training deep neural networks by gradient-based optimizers. Specifica…
Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning
Jakub Grudzien Kuba, Ruiqing Chen, Muning Wen +4
Trust region methods rigorously enabled reinforcement learning (RL) agents to learn monotonically improving policies, leading to superior performance on a variety of tasks. Unfortu…
On the Complexity of Computing Markov Perfect Equilibrium in General-Sum Stochastic Games
Xiaotie Deng, Ningyuan Li, David Mguni +2
Similar to the role of Markov decision processes in reinforcement learning, Stochastic Games (SGs) lay the foundation for the study of multi-agent reinforcement learning (MARL) and…
Settling the Variance of Multi-Agent Policy Gradients
Jakub Grudzien Kuba, Muning Wen, Yaodong Yang +5
Policy gradient (PG) methods are popular reinforcement learning (RL) methods where a baseline is often applied to reduce the variance of gradient estimates. In multi-agent RL (MARL…
Is Nash Equilibrium Approximator Learnable?
Zhijian Duan, Wenhan Huang, Dinghuai Zhang +4
In this paper, we investigate the learnability of the function approximator that approximates Nash equilibrium (NE) for games generated from a distribution. First, we offer a gener…
A Game-Theoretic Approach to Multi-Agent Trust Region Optimization
Ying Wen, Hui Chen, Yaodong Yang +4
Trust region methods are widely applied in single-agent reinforcement learning problems due to their monotonic performance-improvement guarantee at every iteration. Nonetheless, wh…
Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Games
Xiangyu Liu, Hangtian Jia, Ying Wen +5
Measuring and promoting policy diversity is critical for solving games with strong non-transitive dynamics where strategic cycles exist, and there is no consistent winner (e.g., Ro…
MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning
Ming Zhou, Ziyu Wan, Hanjing Wang +6
Population-based multi-agent reinforcement learning (PB-MARL) refers to the series of methods nested with reinforcement learning (RL) algorithms, which produces a self-generated se…
Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment
Tianze Zhou, Fubiao Zhang, Kun Shao +10
Extending transfer learning to cooperative multi-agent reinforcement learning (MARL) has recently received much attention. In contrast to the single-agent setting, the coordination…
Neural Auto-Curricula
Xidong Feng, Oliver Slumbers, Ziyu Wan +5
When solving two-player zero-sum games, multi-agent reinforcement learning (MARL) algorithms often create populations of agents where, at each iteration, a new agent is discovered…
Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction
Hongyao Tang, Jianye Hao, Guangyong Chen +6
Value function is the central notion of Reinforcement Learning (RL). Value estimation, especially with function approximation, can be challenging since it involves the stochasticit…
Learning in Nonzero-Sum Stochastic Games with Potentials
David Mguni, Yutong Wu, Yali Du +6
Multi-agent reinforcement learning (MARL) has become effective in tackling discrete cooperative game scenarios. However, MARL has yet to penetrate settings beyond those modelled by…
Learning to Shape Rewards using a Game of Two Partners
David Mguni, Taher Jafferjee, Jianhong Wang +9
Reward shaping (RS) is a powerful method in reinforcement learning (RL) for overcoming the problem of sparse or uninformative rewards. However, RS typically relies on manually engi…
Modelling Behavioural Diversity for Learning in Open-Ended Games
Nicolas Perez Nieves, Yaodong Yang, Oliver Slumbers +3
Promoting behavioural diversity is critical for solving games with non-transitive dynamics where strategic cycles exist, and there is no consistent winner (e.g., Rock-Paper-Scissor…
Online Double Oracle
Le Cong Dinh, Yaodong Yang, Stephen McAleer +6
Solving strategic games with huge action space is a critical yet under-explored topic in economics, operations research and artificial intelligence. This paper proposes new learnin…
Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
Yaodong Yang, Jun Luo, Ying Wen +5
Multiagent reinforcement learning (MARL) has achieved a remarkable amount of success in solving various types of video games. A cornerstone of this success is the auto-curriculum f…