Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games
arXiv:2007.01542 · doi:10.1145/3402942.3402944
Abstract
While traditionally a labour intensive task, the testing of game content is progressively becoming more automated. Among the many directions in which this automation is taking shape, automatic play-testing is one of the most promising thanks also to advancements of many supervised and reinforcement learning (RL) algorithms. However these type of algorithms, while extremely powerful, often suffer in production environments due to issues with reliability and transparency in their training and usage. In this research work we are investigating and evaluating strategies to apply the popular RL method Proximal Policy Optimization (PPO) in a casual mobile puzzle game with a specific focus on improving its reliability in training and generalization during game playing. We have implemented and tested a number of different strategies against a real-world mobile puzzle game (Lily's Garden from Tactile Games). We isolated the conditions that lead to a failure in either training or generalization during testing and we identified a few strategies to ensure a more stable behaviour of the algorithm in this game genre.
arXiv comment10 pages, 8 figures, to be published in 2020 Foundations of Digital Games conference
References in corpus (18)
- Overcoming catastrophic forgetting in neural networks
- Playing Atari with Deep Reinforcement Learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
- Emergence of Locomotion Behaviours in Rich Environments
- Unity: A General Platform for Intelligent Agents
- Rainbow: Combining Improvements in Deep Reinforcement Learning
- Soft Actor-Critic for Discrete Action Settings
- Quantifying Generalization in Reinforcement Learning
- Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning
- Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
- Assessing Generalization in Deep Reinforcement Learning
- Generalization and Regularization in DQN
- Understanding the impact of entropy on policy optimization
- Decoupling Dynamics and Reward for Transfer Learning
- Generalizing from a few environments in safety-critical reinforcement learning
- I'm sorry Dave, I'm afraid I can't do that, Deep Q-learning from forbidden action
- Using Restart Heuristics to Improve Agent Performance in Angry Birds