Accelerating Quadratic Optimization with Reinforcement Learning

Abstract

First-order methods for quadratic optimization such as OSQP are widely used for large-scale machine learning and embedded optimal control, where many related problems must be rapidly solved. These methods face two persistent challenges: manual hyperparameter tuning and convergence time to high-accuracy solutions. To address these, we explore how Reinforcement Learning (RL) can learn a policy to tune parameters to accelerate convergence. In experiments with well-known QP benchmarks we find that our RL policy, RLQP, significantly outperforms state-of-the-art QP solvers by up to 3x. RLQP generalizes surprisingly well to previously unseen problems with varying dimension and structure from different applications, including the QPLIB, Netlib LP and Maros-Meszaros problems. Code for RLQP is available at https://github.com/berkeleyautomation/rlqp.

25 pages, 7 figures. Code available at https://github.com/berkeleyautomation/rlqp

References in corpus (11)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Continuous control with deep reinforcement learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• OSQP: An Operator Splitting Solver for Quadratic Programs
• Learning Combinatorial Optimization Algorithms over Graphs
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• On First-Order Meta-Learning Algorithms
• Learning to Optimize
• Understanding and correcting pathologies in the training of learned optimizers