All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization

Abstract

In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. Besides dealing with the increased dynamics of the scenarios due to the opponents' actions, they usually have to understand how to overcome the opponent's strategies. Most of the common solutions, usually based on continual learning or centralized multi-agent experiences, however, do not allow the development of personalized strategies to face individual opponents. In this paper, we propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. The entire model is trained online, using a composed loss based on a contrastive optimization, to learn competitive and multiplayer games. We evaluate our model on a pokemon duel scenario and the four-player competitive Chef's Hat card game. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times. We also present a discussion on the impact of our model, in particular on how well it deals with on specific strategy learning for each of the two scenarios.

References in corpus (14)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Progressive Neural Networks
• Experience Replay for Continual Learning
• Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• Reinforced Continual Learning
• Third-Person Imitation Learning
• Multi-Agent Generative Adversarial Imitation Learning
• Opponent Modeling in Deep Reinforcement Learning
• Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting
• Personalizing a Dialogue System with Transfer Reinforcement Learning
• Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization