Continuous Generative Neural Networks: A Wavelet-Based Architecture in Function Spaces

Abstract

In this work, we present and study Continuous Generative Neural Networks (CGNNs), namely, generative models in the continuous setting: the output of a CGNN belongs to an infinite-dimensional function space. The architecture is inspired by DCGAN, with one fully connected layer, several convolutional layers and nonlinear activation functions. In the continuous $L^2$ setting, the dimensions of the spaces of each layer are replaced by the scales of a multiresolution analysis of a compactly supported wavelet. We present conditions on the convolutional filters and on the nonlinearity that guarantee that a CGNN is injective. This theory finds applications to inverse problems, and allows for deriving Lipschitz stability estimates for (possibly nonlinear) infinite-dimensional inverse problems with unknowns belonging to the manifold generated by a CGNN. Several numerical simulations, including signal deblurring, illustrate and validate this approach.

40 pages, 8 figures

References in corpus (24)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Deep Scattering Spectrum
• Deep learning for undersampled MRI reconstruction
• Implicit Neural Representations with Periodic Activation Functions
• Analyzing Inverse Problems with Invertible Neural Networks
• A learning-based method for solving ill-posed nonlinear inverse problems: a simulation study of Lung EIT
• Local Analysis of Inverse Problems: Hölder Stability and Iterative Reconstruction
• Uniqueness and Lipschitz stability in Electrical Impedance Tomography with finitely many electrodes
• Stabilizing Invertible Neural Networks Using Mixture Models
• Infinite-dimensional inverse problems with finite measurements
• Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpoint
• Lipschitz stability for the electrostatic inverse boundary value problem with piecewise linear conductivities
• Calderón's Inverse Problem with a Finite Number of Measurements
• Lipschitz continuous dependence of piecewise constant Lamé coefficients from boundary data: the case of non flat interfaces
• Representation Equivalent Neural Operators: a Framework for Alias-free Operator Learning
• Inverse problems on low-dimensional manifolds
• Globally Injective ReLU Networks
• Generative networks as inverse problems with Scattering transforms
• Trumpets: Injective Flows for Inference and Inverse Problems
• Compressed sensing photoacoustic tomography reduces to compressed sensing for undersampled Fourier measurements
• FunkNN: Neural Interpolation for Functional Generation
• Manifold Learning by Mixture Models of VAEs for Inverse Problems
• Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows