Deep Reinforcement Learning for Multi-objective Optimization

Abstract

This study proposes an end-to-end framework for solving multi-objective optimization problems (MOPs) using Deep Reinforcement Learning (DRL), that we call DRL-MOA. The idea of decomposition is adopted to decompose the MOP into a set of scalar optimization subproblems. Then each subproblem is modelled as a neural network. Model parameters of all the subproblems are optimized collaboratively according to a neighborhood-based parameter-transfer strategy and the DRL training algorithm. Pareto optimal solutions can be directly obtained through the trained neural network models. In specific, the multi-objective travelling salesman problem (MOTSP) is solved in this work using the DRL-MOA method by modelling the subproblem as a Pointer Network. Extensive experiments have been conducted to study the DRL-MOA and various benchmark methods are compared with it. It is found that, once the trained model is available, it can scale to newly encountered problems with no need of re-training the model. The solutions can be directly obtained by a simple forward calculation of the neural network; thereby, no iteration is required and the MOP can be always solved in a reasonable time. The proposed method provides a new way of solving the MOP by means of DRL. It has shown a set of new characteristics, e.g., strong generalization ability and fast solving speed in comparison with the existing methods for multi-objective optimizations. Experimental results show the effectiveness and competitiveness of the proposed method in terms of model performance and running time.

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Sequence to Sequence Learning with Neural Networks
• Neural Combinatorial Optimization with Reinforcement Learning
• PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization
• MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
• Multi-Objective Deep Reinforcement Learning

Cited by in corpus (19)

• A Review of Deep Reinforcement Learning for Smart Building Energy Management
• Multimodal Multi-objective Optimization: Comparative Study of the State-of-the-Art
• A Survey on Learnable Evolutionary Algorithms for Scalable Multiobjective Optimization
• Deep Reinforcement Learning for Combinatorial Optimization: Covering Salesman Problems
• Coevolutionary Framework for Generalized Multimodal Multi-objective Optimization
• The Road to the Ideal Stent: A Review of Stent Design Optimisation Methods, Findings, and Opportunities
• An empirical investigation of the challenges of real-world reinforcement learning
• Deep Reinforcement Learning-based Content Migration for Edge Content Delivery Networks with Vehicular Nodes
• A Practical Guide to Multi-Objective Reinforcement Learning and Planning
• Multi-objective application placement in fog computing using graph neural network-based reinforcement learning
• Learning for routing: A guided review of recent developments and future directions
• A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems
• Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyML
• SED2AM: Solving Multi-Trip Time-Dependent Vehicle Routing Problem using Deep Reinforcement Learning
• Multi-Objective Graph Heuristic Search for Terrestrial Robot Design
• Training Generative Adversarial Networks with Adaptive Composite Gradient
• MODRL/D-AM: Multiobjective Deep Reinforcement Learning Algorithm Using Decomposition and Attention Model for Multiobjective Optimization
• RapidLayout: Fast Hard Block Placement of FPGA-optimized Systolic Arrays using Evolutionary Algorithms
• MODRL/D-EL: Multiobjective Deep Reinforcement Learning with Evolutionary Learning for Multiobjective Optimization