Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directions

Abstract

Machine scheduling aims to optimize job assignments to machines while adhering to manufacturing rules and job specifications. This optimization leads to reduced operational costs, improved customer demand fulfillment, and enhanced production efficiency. However, machine scheduling remains a challenging combinatorial problem due to its NP-hard nature. Deep Reinforcement Learning (DRL), a key component of artificial general intelligence, has shown promise in various domains like gaming and robotics. Researchers have explored applying DRL to machine scheduling problems since 1995. This paper offers a comprehensive review and comparison of DRL-based approaches, highlighting their methodology, applications, advantages, and limitations. It categorizes these approaches based on computational components: conventional neural networks, encoder-decoder architectures, graph neural networks, and metaheuristic algorithms. Our review concludes that DRL-based methods outperform exact solvers, heuristics, and tabular reinforcement learning algorithms in terms of computation speed and generating near-global optimal solutions. These DRL-based approaches have been successfully applied to static and dynamic scheduling across diverse machine environments and job characteristics. However, DRL-based schedulers face limitations in handling complex operational constraints, configurable multi-objective optimization, generalization, scalability, interpretability, and robustness. Addressing these challenges will be a crucial focus for future research in this field. This paper serves as a valuable resource for researchers to assess the current state of DRL-based machine scheduling and identify research gaps. It also aids experts and practitioners in selecting the appropriate DRL approach for production scheduling.

References in corpus (12)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Semi-Supervised Classification with Graph Convolutional Networks
• A Brief Survey of Deep Reinforcement Learning
• Conservative Q-Learning for Offline Reinforcement Learning
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to schedule job-shop problems: Representation and policy learning using graph neural network and reinforcement learning
• Learning Combinatorial Optimization on Graphs: A Survey with Applications to Networking
• Graph Neural Networks for Natural Language Processing: A Survey
• An actor-critic algorithm with policy gradients to solve the job shop scheduling problem using deep double recurrent agents
• Synthesizing Rolling Bearing Fault Samples in New Conditions: A framework based on a modified CGAN
• Simulation-guided Beam Search for Neural Combinatorial Optimization