A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments

Abstract

Reinforcement learning (RL) algorithms find applications in inventory control, recommender systems, vehicular traffic management, cloud computing and robotics. The real-world complications of many tasks arising in these domains makes them difficult to solve with the basic assumptions underlying classical RL algorithms. RL agents in these applications often need to react and adapt to changing operating conditions. A significant part of research on single-agent RL techniques focuses on developing algorithms when the underlying assumption of stationary environment model is relaxed. This paper provides a survey of RL methods developed for handling dynamically varying environment models. The goal of methods not limited by the stationarity assumption is to help autonomous agents adapt to varying operating conditions. This is possible either by minimizing the rewards lost during learning by RL agent or by finding a suitable policy for the RL agent which leads to efficient operation of the underlying system. A representative collection of these algorithms is discussed in detail in this work along with their categorization and their relative merits and demerits. Additionally we also review works which are tailored to application domains. Finally, we discuss future enhancements for this field.

References in corpus (4)

• Challenges of Real-World Reinforcement Learning
• A Survey of Deep Reinforcement Learning in Video Games
• Policy Consolidation for Continual Reinforcement Learning
• Change Point Detection for Compositional Multivariate Data

Cited by in corpus (11)

• Neurosymbolic Reinforcement Learning and Planning: A Survey
• Quantifying the Impact of Non-Stationarity in Reinforcement Learning-Based Traffic Signal Control
• Dynamic Regret of Policy Optimization in Non-stationary Environments
• Model-Free Non-Stationary RL: Near-Optimal Regret and Applications in Multi-Agent RL and Inventory Control
• Optimizing for the Future in Non-Stationary MDPs
• Online reinforcement learning with sparse rewards through an active inference capsule
• System Design for an Integrated Lifelong Reinforcement Learning Agent for Real-Time Strategy Games
• RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN
• Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architectures
• Adapting the Behavior of Reinforcement Learning Agents to Changing Action Spaces and Reward Functions
• Collaboration Promotes Group Resilience in Multi-Agent RL