Uniformly convex neural networks and non-stationary iterated network Tikhonov (iNETT) method
arXiv:2210.03314 · doi:10.1088/1361-6420/acc2b6
Abstract
We propose a non-stationary iterated network Tikhonov (iNETT) method for the solution of ill-posed inverse problems. The iNETT employs deep neural networks to build a data-driven regularizer, and it avoids the difficult task of estimating the optimal regularization parameter. To achieve the theoretical convergence of iNETT, we introduce uniformly convex neural networks to build the data-driven regularizer. Rigorous theories and detailed algorithms are proposed for the construction of convex and uniformly convex neural networks. In particular, given a general neural network architecture, we prescribe sufficient conditions to achieve a trained neural network which is component-wise convex or uniformly convex; moreover, we provide concrete examples of realizing convexity and uniform convexity in the modern U-net architecture. With the tools of convex and uniformly convex neural networks, the iNETT algorithm is developed and a rigorous convergence analysis is provided. Lastly, we show applications of the iNETT algorithm in 2D computerized tomography, where numerical examples illustrate the efficacy of the proposed algorithm.
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- A guide to convolution arithmetic for deep learning
- MoDL: Model Based Deep Learning Architecture for Inverse Problems
- Adversarial Regularizers in Inverse Problems
- Iterated fractional Tikhonov regularization
- Nonstationary iterated Tikhonov regularization in Banach spaces with uniformly convex penalty terms
- Regularization preconditioners for frame-based image deblurring with reduced boundary artifacts
- A stochastic gradient descent approach with partitioned-truncated singular value decomposition for large-scale inverse problems of magnetic modulus data