Neural Fixed-Point Acceleration for Convex Optimization
arXiv:2107.10254
Abstract
Fixed-point iterations are at the heart of numerical computing and are often a computational bottleneck in real-time applications that typically need a fast solution of moderate accuracy. We present neural fixed-point acceleration which combines ideas from meta-learning and classical acceleration methods to automatically learn to accelerate fixed-point problems that are drawn from a distribution. We apply our framework to SCS, the state-of-the-art solver for convex cone programming, and design models and loss functions to overcome the challenges of learning over unrolled optimization and acceleration instabilities. Our work brings neural acceleration into any optimization problem expressible with CVXPY. The source code behind this paper is available at https://github.com/facebookresearch/neural-scs
arXiv commentAutoML@ICML2021
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- The NumPy array: a structure for efficient numerical computation
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Fourier Neural Operator for Parametric Partial Differential Equations
- Learning to Optimize
- Differentiable Convex Optimization Layers
- On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
- Learned Optimizers that Scale and Generalize
- Multiscale Deep Equilibrium Models
- Solving Mixed Integer Programs Using Neural Networks
- Hypersolvers: Toward Fast Continuous-Depth Models
- A Learning-boosted Quasi-Newton Method for AC Optimal Power Flow
- Using learned optimizers to make models robust to input noise
- Training Learned Optimizers with Randomly Initialized Learned Optimizers