SUNLayer: Stable denoising with generative networks

Abstract

Deep neural networks are often used to implement powerful generative models for real-world data. Notable applications include image denoising, as well as other classical inverse problems like compressed sensing and super-resolution. To provide a rigorous but simplified analysis of generative models, in this work, we introduce an elegant theoretical framework based on spherical harmonics, namely \textbf{SUNLayer}. Our theoretical framework identifies explicit conditions on activation functions that guarantee denoising under local optimization. Numerical experiments examine the theoretical properties on commonly used activation functions and demonstrate their stable denoising performance.

References in corpus (4)

• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Searching for Activation Functions
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Adjusting for Dropout Variance in Batch Normalization and Weight Initialization

Cited by in corpus (7)

• Compressed Sensing with Deep Image Prior and Learned Regularization
• Robust Compressive Phase Retrieval via Deep Generative Priors
• Exact asymptotics for phase retrieval and compressed sensing with random generative priors
• Inverting Deep Generative models, One layer at a time
• Inference in Multi-Layer Networks with Matrix-Valued Unknowns
• Nonasymptotic Guarantees for Spiked Matrix Recovery with Generative Priors
• Using Supervised Learning to Construct the Best Regularization Term and the Best Multiresolution Analysis