A Survey of Multi-Objective Sequential Decision-Making

Abstract

Sequential decision-making problems with multiple objectives arise naturally in practice and pose unique challenges for research in decision-theoretic planning and learning, which has largely focused on single-objective settings. This article surveys algorithms designed for sequential decision-making problems with multiple objectives. Though there is a growing body of literature on this subject, little of it makes explicit under what circumstances special methods are needed to solve multi-objective problems. Therefore, we identify three distinct scenarios in which converting such a problem to a single-objective one is impossible, infeasible, or undesirable. Furthermore, we propose a taxonomy that classifies multi-objective methods according to the applicable scenario, the nature of the scalarization function (which projects multi-objective values to scalar ones), and the type of policies considered. We show how these factors determine the nature of an optimal solution, which can be a single policy, a convex hull, or a Pareto front. Using this taxonomy, we survey the literature on multi-objective methods for planning and learning. Finally, we discuss key applications of such methods and outline opportunities for future work.

References in corpus (5)

• Decision-Theoretic Planning: Structural Assumptions and Computational Leverage
• Anytime Point-Based Approximations for Large POMDPs
• Incremental Pruning: A Simple, Fast, Exact Method for Partially Observable Markov Decision Processes
• An Evolutionary Algorithm with Advanced Goal and Priority Specification for Multi-objective Optimization
• Approximation of Lorenz-Optimal Solutions in Multiobjective Markov Decision Processes

Cited by in corpus (30)

• Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning
• A Multi-Objective Deep Reinforcement Learning Framework
• Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey
• Evolutionary Reinforcement Learning: A Survey
• MO-MIX: Multi-Objective Multi-Agent Cooperative Decision-Making With Deep Reinforcement Learning
• Multi-Objective Path-Based D* Lite
• Pareto Monte Carlo Tree Search for Multi-Objective Informative Planning
• Subdimensional Expansion for Multi-objective Multi-agent Path Finding
• Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector
• A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Review
• A Practical Guide to Multi-Objective Reinforcement Learning and Planning
• A utility-based analysis of equilibria in multi-objective normal form games
• Identification and Off-Policy Learning of Multiple Objectives Using Adaptive Clustering
• A Demonstration of Issues with Value-Based Multiobjective Reinforcement Learning Under Stochastic State Transitions
• Goal-directed graph construction using reinforcement learning
• Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimization
• A Local Optimization Framework for Multi-Objective Ergodic Search
• Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making
• Safety Optimized Reinforcement Learning via Multi-Objective Policy Optimization
• Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care
• Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes
• Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learning
• Linear programming-based solution methods for constrained partially observable Markov decision processes
• Model-Free Learning of Safe yet Effective Controllers
• AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning
• Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis
• A Fairness-Oriented Multi-Objective Reinforcement Learning approach for Autonomous Intersection Management
• Philosophy-informed Machine Learning
• Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Model
• From Preference-Based to Multiobjective Sequential Decision-Making