Bilevel methods for image reconstruction

Abstract

This review discusses methods for learning parameters for image reconstruction problems using bilevel formulations. Image reconstruction typically involves optimizing a cost function to recover a vector of unknown variables that agrees with collected measurements and prior assumptions. State-of-the-art image reconstruction methods learn these prior assumptions from training data using various machine learning techniques, such as bilevel methods. One can view the bilevel problem as formalizing hyperparameter optimization, as bridging machine learning and cost function based optimization methods, or as a method to learn variables best suited to a specific task. More formally, bilevel problems attempt to minimize an upper-level loss function, where variables in the upper-level loss function are themselves minimizers of a lower-level cost function. This review contains a running example problem of learning tuning parameters and the coefficients for sparsifying filters used in a regularizer. Such filters generalize the popular total variation regularization method, and learned filters are closely related to convolutional neural networks approaches that are rapidly gaining in popularity. Here, the lower-level problem is to reconstruct an image using a regularizer with learned sparsifying filters; the corresponding upper-level optimization problem involves a measure of reconstructed image quality based on training data. This review discusses multiple perspectives to motivate the use of bilevel methods and to make them more easily accessible to different audiences. We then turn to ways to optimize the bilevel problem, providing pros and cons of the variety of proposed approaches. Finally we overview bilevel applications in image reconstruction.

arXiv comment125 pages, 19 figures

References in corpus (40)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment
• Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network
• Task-Driven Dictionary Learning
• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• A Tutorial on Bayesian Optimization
• Non-convex Optimization for Machine Learning
• Generalized SURE for Exponential Families: Applications to Regularization
• Structure-sensitive Multi-scale Deep Neural Network for Low-Dose CT Denoising
• Deep Equilibrium Models
• Convolutional Dictionary Learning: A Comparative Review and New Algorithms
• Analysis Operator Learning and Its Application to Image Reconstruction
• A Unified Framework for Multistage and Multilevel Mixed Integer Linear Optimization
• Convolutional Dictionary Learning: Acceleration and Convergence
• Hyperparameter optimization with approximate gradient
• Insights into analysis operator learning: From patch-based sparse models to higher-order MRFs
• On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
• Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling
• Transform Learning for Magnetic Resonance Image Reconstruction: From Model-based Learning to Building Neural Networks
• Approximation Methods for Bilevel Programming
• Convolutional Analysis Operator Learning: Acceleration and Convergence
• A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
• Revisiting loss-specific training of filter-based MRFs for image restoration
• Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks
• On the convergence analysis of the optimized gradient method
• Bilevel Optimization: Convergence Analysis and Enhanced Design
• Learning Filter Bank Sparsifying Transforms
• A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum
• Dualization and Automatic Distributed Parameter Selection of Total Generalized Variation via Bilevel Optimization
• Penalty Method for Inversion-Free Deep Bilevel Optimization
• HARK Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning
• Total Deep Variation: A Stable Regularizer for Inverse Problems
• Supervised Learning of Sparsity-Promoting Regularizers for Denoising
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding
• Learned Cone-Beam CT Reconstruction Using Neural Ordinary Differential Equations
• Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction
• Bilevel learning of l1-regularizers with closed-form gradients(BLORC)
• Learning Multiscale Convolutional Dictionaries for Image Reconstruction

Cited by in corpus (5)

• An adaptively inexact first-order method for bilevel optimization with application to hyperparameter learning
• A multiscale Consensus-Based algorithm for multi-level optimization
• Dyadic partition-based training schemes for TV/TGV denoising
• Adaptive Local Neighborhood-based Neural Networks for MR Image Reconstruction from Undersampled Data
• Bilevel Learning with Inexact Stochastic Gradients