A study of first-passage time minimization via Q-learning in heated gridworlds
arXiv:2110.02129
Abstract
Optimization of first-passage times is required in applications ranging from nanobots navigation to market trading. In such settings, one often encounters unevenly distributed noise levels across the environment. We extensively study how a learning agent fares in 1- and 2- dimensional heated gridworlds with an uneven temperature distribution. The results show certain bias effects in agents trained via simple tabular Q-learning, SARSA, Expected SARSA and Double Q-learning. While high learning rate prevents exploration of regions with higher temperature, low enough rate increases the presence of agents in such regions. The discovered peculiarities and biases of temporal-difference-based reinforcement learning methods should be taken into account in real-world physical applications and agent design.
References in corpus (8)
- Playing Atari with Deep Reinforcement Learning
- Machine learning and the physical sciences
- Flow Navigation by Smart Microswimmers via Reinforcement Learning
- Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
- Adversarial Policies: Attacking Deep Reinforcement Learning
- Optimizing Brownian escape rates by potential shaping
- Smart Inertial Particles
- First-passage and risk evaluation under stochastic volatility