Learning Improvement Heuristics for Solving Routing Problems

Abstract

Recent studies in using deep learning to solve routing problems focus on construction heuristics, the solutions of which are still far from optimality. Improvement heuristics have great potential to narrow this gap by iteratively refining a solution. However, classic improvement heuristics are all guided by hand-crafted rules which may limit their performance. In this paper, we propose a deep reinforcement learning framework to learn the improvement heuristics for routing problems. We design a self-attention based deep architecture as the policy network to guide the selection of next solution. We apply our method to two important routing problems, i.e. travelling salesman problem (TSP) and capacitated vehicle routing problem (CVRP). Experiments show that our method outperforms state-of-the-art deep learning based approaches. The learned policies are more effective than the traditional hand-crafted ones, and can be further enhanced by simple diversifying strategies. Moreover, the policies generalize well to different problem sizes, initial solutions and even real-world dataset.

10 pages, 4 figures

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Learning Combinatorial Optimization Algorithms over Graphs
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Neural Combinatorial Optimization with Reinforcement Learning
• Attention, Learn to Solve Routing Problems!
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks

Cited by in corpus (14)

• Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learning
• Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem
• Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking
• Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning
• POMO: Policy Optimization with Multiple Optima for Reinforcement Learning
• Learning 2-opt Heuristics for the Traveling Salesman Problem via Deep Reinforcement Learning
• The Transformer Network for the Traveling Salesman Problem
• Learning Collaborative Policies to Solve NP-hard Routing Problems
• Learning the Travelling Salesperson Problem Requires Rethinking Generalization
• Learning Variable Ordering Heuristics for Solving Constraint Satisfaction Problems
• ScheduleNet: Learn to solve multi-agent scheduling problems with reinforcement learning
• Graph Neural Network Guided Local Search for the Traveling Salesperson Problem
• Learning Enhanced Optimisation for Routing Problems
• QROSS: QUBO Relaxation Parameter Optimisation via Learning Solver Surrogates