Online 3D Bin Packing with Constrained Deep Reinforcement Learning
arXiv:2006.14978
Abstract
We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In our problem, the agent has limited information about the items to be packed into the bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item's placement also subjects to the constraints of collision avoidance and physical stability. We formulate this online 3D-BPP as a constrained Markov decision process. To solve the problem, we propose an effective and easy-to-implement constrained deep reinforcement learning (DRL) method under the actor-critic framework. In particular, we introduce a feasibility predictor to predict the feasibility mask for the placement actions and use it to modulate the action probabilities output by the actor during training. Such supervisions and transformations to DRL facilitate the agent to learn feasible policies efficiently. Our method can also be generalized e.g., with the ability to handle lookahead or items with different orientations. We have conducted extensive evaluation showing that the learned policy significantly outperforms the state-of-the-art methods. A user study suggests that our method attains a human-level performance.
arXiv commentAAAI 2021
References in corpus (8)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Continuous control with deep reinforcement learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Dueling Network Architectures for Deep Reinforcement Learning
- Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
- Distributed Distributional Deterministic Policy Gradients
- Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method
- Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization