Continuous-Domain Solutions of Linear Inverse Problems with Tikhonov vs. Generalized TV Regularization

Abstract

We consider linear inverse problems that are formulated in the continuous domain. The object of recovery is a function that is assumed to minimize a convex objective functional. The solutions are constrained by imposing a continuous-domain regularization. We derive the parametric form of the solution (representer theorems) for Tikhonov (quadratic) and generalized total-variation (gTV) regularizations. We show that, in both cases, the solutions are splines that are intimately related to the regularization operator. In the Tikhonov case, the solution is smooth and constrained to live in a fixed subspace that depends on the measurement operator. By contrast, the gTV regularization results in a sparse solution composed of only a few dictionary elements that are upper-bounded by the number of measurements and independent of the measurement operator. Our findings for the gTV regularization resonates with the minimization of the $l_1$ norm, which is its discrete counterpart and also produces sparse solutions. Finally, we find the experimental solutions for some measurement models in one dimension. We discuss the special case when the gTV regularization results in multiple solutions and devise an algorithm to find an extreme point of the solution set which is guaranteed to be sparse.

References in corpus (6)

• Sparsity and Incoherence in Compressive Sampling
• Compressed Sensing and Redundant Dictionaries
• Geometric properties of solutions to the total variation denoising problem
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$
• Splines are Universal Solutions of Linear Inverse Problems with Generalized-TV regularization
• Discussion: The Dantzig selector: Statistical estimation when $p$ is much larger than $n$

Cited by in corpus (9)

• The Role of Neural Network Activation Functions
• Sampling and Reconstruction of Sparse Signals in Shift-Invariant Spaces: Generalized Shannon's Theorem Meets Compressive Sensing
• TV-based Reconstruction of Periodic Functions
• Periodic Splines and Gaussian Processes for the Resolution of Linear Inverse Problems
• Native Banach spaces for splines and variational inverse problems
• A Fast and Scalable Polyatomic Frank-Wolfe Algorithm for the LASSO
• A representer theorem for deep neural networks
• Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures
• Spline-based Reconstruction of Periodic Signals with Sparse Innovations