Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration

Abstract

The deep unfolding approach has attracted significant attention in computer vision tasks, which well connects conventional image processing modeling manners with more recent deep learning techniques. Specifically, by establishing a direct correspondence between algorithm operators at each implementation step and network modules within each layer, one can rationally construct an almost ``white box'' network architecture with high interpretability. In this architecture, only the predefined component of the proximal operator, known as a proximal network, needs manual configuration, enabling the network to automatically extract intrinsic image priors in a data-driven manner. In current deep unfolding methods, such a proximal network is generally designed as a CNN architecture, whose necessity has been proven by a recent theory. That is, CNN structure substantially delivers the translational invariant image prior, which is the most universally possessed structural prior across various types of images. However, standard CNN-based proximal networks have essential limitations in capturing the rotation symmetry prior, another universal structural prior underlying general images. This leaves a large room for further performance improvement in deep unfolding approaches. To address this issue, this study makes efforts to suggest a high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework. Especially, we deduce, for the first time, the theoretical equivariant error for such a designed proximal network with arbitrary layers under arbitrary rotation degrees. This analysis should be the most refined theoretical conclusion for such error evaluation to date and is also indispensable for supporting the rationale behind such networks with intrinsic interpretability requirements.

Published in TPAMI 2024

References in corpus (17)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• Clearing the Skies: A deep network architecture for single-image rain removal
• Learned Primal-dual Reconstruction
• BM3D Frames and Variational Image Deblurring
• Denoising Prior Driven Deep Neural Network for Image Restoration
• Solving ill-posed inverse problems using iterative deep neural networks
• Convolutional Neural Network Based Metal Artifact Reduction in X-ray Computed Tomography
• Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems
• Rotation equivariant vector field networks
• Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis
• ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction
• Recurrent Inference Machines for Solving Inverse Problems
• Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction
• Orientation-Shared Convolution Representation for CT Metal Artifact Learning
• Understanding the Learned Iterative Soft Thresholding Algorithm with matrix factorization

Cited by in corpus (2)

• Technical Report: Towards Spatial Feature Regularization in Deep-Learning-Based Array-SAR Reconstruction
• PreCM: The Padding-based Rotation Equivariant Convolution Mode for Semantic Segmentation