A Practical Guide to Multi-Objective Reinforcement Learning and Planning

Abstract

Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

References in corpus (30)

• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• A Survey of Multi-Objective Sequential Decision-Making
• Deep Reinforcement Learning for Multi-objective Optimization
• Risk-Sensitive Reinforcement Learning Applied to Control under Constraints
• Successor Features for Transfer in Reinforcement Learning
• A Multi-Objective Deep Reinforcement Learning Framework
• Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey
• Levels of explainable artificial intelligence for human-aligned conversational explanations
• A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation
• Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning
• Multi-Objective Deep Reinforcement Learning
• Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning
• Pareto Monte Carlo Tree Search for Multi-Objective Informative Planning
• Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learning
• Fleetwide data-enabled reliability improvement of wind turbines
• A Distributional View on Multi-Objective Policy Optimization
• Interpretable Multi-Objective Reinforcement Learning through Policy Orchestration
• Universal Successor Representations for Transfer Reinforcement Learning
• Multi-Agent Thompson Sampling for Bandit Applications with Sparse Neighbourhood Structures
• Identification and Off-Policy Learning of Multiple Objectives Using Adaptive Clustering
• A utility-based analysis of equilibria in multi-objective normal form games
• A Demonstration of Issues with Value-Based Multiobjective Reinforcement Learning Under Stochastic State Transitions
• Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewards
• Multi-objective Contextual Bandit Problem with Similarity Information
• Risk Aware and Multi-Objective Decision Making with Distributional Monte Carlo Tree Search
• MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding
• Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes
• Approximation of Lorenz-Optimal Solutions in Multiobjective Markov Decision Processes
• Relationship Explainable Multi-objective Reinforcement Learning with Semantic Explainability Generation

Cited by in corpus (1)

• Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making