Convolutional Proximal Neural Networks and Plug-and-Play Algorithms

Abstract

In this paper, we introduce convolutional proximal neural networks (cPNNs), which are by construction averaged operators. For filters of full length, we propose a stochastic gradient descent algorithm on a submanifold of the Stiefel manifold to train cPNNs. In case of filters with limited length, we design algorithms for minimizing functionals that approximate the orthogonality constraints imposed on the operators by penalizing the least squares distance to the identity operator. Then, we investigate how scaled cPNNs with a prescribed Lipschitz constant can be used for denoising signals and images, where the achieved quality depends on the Lipschitz constant. Finally, we apply cPNN based denoisers within a Plug-and-Play (PnP) framework and provide convergence results for the corresponding PnP forward-backward splitting algorithm based on an oracle construction.

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Spectral Normalization for Generative Adversarial Networks
• Message Passing Algorithms for Compressed Sensing
• BM3D Frames and Variational Image Deblurring
• CNN-Based Projected Gradient Descent for Consistent Image Reconstruction
• Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
• An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• The Singular Values of Convolutional Layers
• Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
• One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Models
• Quantitative convergence analysis of iterated expansive, set-valued mappings
• Parseval Proximal Neural Networks
• Stabilizing Invertible Neural Networks Using Mixture Models
• Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform
• Understanding and Mitigating Exploding Inverses in Invertible Neural Networks

Cited by in corpus (8)

• Image reconstruction algorithms in radio interferometry: from handcrafted to learned regularization denoisers
• PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization
• Wasserstein Patch Prior for Image Superresolution
• Generalized Normalizing Flows via Markov Chains
• Inertial Stochastic PALM (iSPALM) and Applications in Machine Learning
• WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution
• Proximal Residual Flows for Bayesian Inverse Problems
• On $\alpha$-Firmly Nonexpansive Operators in $r$-Uniformly Convex Spaces