Dynamic Weights in Multi-Objective Deep Reinforcement Learning

Abstract

Many real-world decision problems are characterized by multiple conflicting objectives which must be balanced based on their relative importance. In the dynamic weights setting the relative importance changes over time and specialized algorithms that deal with such change, such as a tabular Reinforcement Learning (RL) algorithm by Natarajan and Tadepalli (2005), are required. However, this earlier work is not feasible for RL settings that necessitate the use of function approximators. We generalize across weight changes and high-dimensional inputs by proposing a multi-objective Q-network whose outputs are conditioned on the relative importance of objectives and we introduce Diverse Experience Replay (DER) to counter the inherent non-stationarity of the Dynamic Weights setting. We perform an extensive experimental evaluation and compare our methods to adapted algorithms from Deep Multi-Task/Multi-Objective Reinforcement Learning and show that our proposed network in combination with DER dominates these adapted algorithms across weight change scenarios and problem domains.

References in corpus (6)

• Playing Atari with Deep Reinforcement Learning
• Dueling Network Architectures for Deep Reinforcement Learning
• Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
• Multi-Objective Deep Reinforcement Learning
• Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving
• Universal Successor Representations for Transfer Reinforcement Learning

Cited by in corpus (9)

• A Multi-Objective Deep Reinforcement Learning Framework
• Levels of explainable artificial intelligence for human-aligned conversational explanations
• Toward Pareto Efficient Fairness-Utility Trade-off inRecommendation through Reinforcement Learning
• MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
• Reducing Idleness in Financial Cloud Services via Multi-objective Evolutionary Reinforcement Learning based Load Balancer
• Reinforcement Learning for Multi-Objective and Constrained Markov Decision Processes
• Deep W-Networks: Solving Multi-Objective Optimisation Problems With Deep Reinforcement Learning
• Safe Policy Improvement in Constrained Markov Decision Processes
• How to Train your Quadrotor: A Framework for Consistently Smooth and Responsive Flight Control via Reinforcement Learning