Investigating Customization Strategies and Convergence Behaviors of Task-specific ADMM
arXiv:1909.10819
Abstract
Alternating Direction Method of Multiplier (ADMM) has been a popular algorithmic framework for separable optimization problems with linear constraints. For numerical ADMM fail to exploit the particular structure of the problem at hand nor the input data information, leveraging task-specific modules (e.g., neural networks and other data-driven architectures) to extend ADMM is a significant but challenging task. This work focuses on designing a flexible algorithmic framework to incorporate various task-specific modules (with no additional constraints) to improve the performance of ADMM in real-world applications. Specifically, we propose Guidance from Optimality (GO), a new customization strategy, to embed task-specific modules into ADMM (GO-ADMM). By introducing an optimality-based criterion to guide the propagation, GO-ADMM establishes an updating scheme agnostic to the choice of additional modules. The existing task-specific methods just plug their task-specific modules into the numerical iterations in a straightforward manner. Even with some restrictive constraints on the plug-in modules, they can only obtain some relatively weaker convergence properties for the resulted ADMM iterations. Fortunately, without any restrictions on the embedded modules, we prove the convergence of GO-ADMM regarding objective values and constraint violations, and derive the worst-case convergence rate measured by iteration complexity. Extensive experiments are conducted to verify the theoretical results and demonstrate the efficiency of GO-ADMM.
arXiv commentAccepted at TIP
References in corpus (11)
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Robust Recovery of Subspace Structures by Low-Rank Representation
- Clearing the Skies: A deep network architecture for single-image rain removal
- Denoising Prior Driven Deep Neural Network for Image Restoration
- An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
- FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors
- ADMM-Net: A Deep Learning Approach for Compressive Sensing MRI
- Linearized Alternating Direction Method with Parallel Splitting and Adaptive Penalty for Separable Convex Programs in Machine Learning
- Differentiable Linearized ADMM
- Toward Designing Convergent Deep Operator Splitting Methods for Task-specific Nonconvex Optimization
- Converged Deep Framework Assembling Principled Modules for CS-MRI