Convolutional dictionary learning based auto-encoders for natural exponential-family distributions

Abstract

We introduce a class of auto-encoder neural networks tailored to data from the natural exponential family (e.g., count data). The architectures are inspired by the problem of learning the filters in a convolutional generative model with sparsity constraints, often referred to as convolutional dictionary learning (CDL). Our work is the first to combine ideas from convolutional generative models and deep learning for data that are naturally modeled with a non-Gaussian distribution (e.g., binomial and Poisson). This perspective provides us with a scalable and flexible framework that can be re-purposed for a wide range of tasks and assumptions on the generative model. Specifically, the iterative optimization procedure for solving CDL, an unsupervised task, is mapped to an unfolded and constrained neural network, with iterative adjustments to the inputs to account for the generative distribution. We also show that the framework can easily be extended for discriminative training, appropriate for a supervised task. We demonstrate 1) that fitting the generative model to learn, in an unsupervised fashion, the latent stimulus that underlies neural spiking data leads to better goodness-of-fit compared to other baselines, 2) competitive performance compared to state-of-the-art algorithms for supervised Poisson image denoising, with significantly fewer parameters, and 3) gradient dynamics of shallow binomial auto-encoder.

References in corpus (9)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
• Convolutional Dictionary Learning: A Comparative Review and New Algorithms
• Learned Convolutional Sparse Coding
• Bits from Photons: Oversampled Image Acquisition Using Binary Poisson Statistics
• Neural Proximal Gradient Descent for Compressive Imaging
• Sparsity Based Poisson Denoising with Dictionary Learning
• Deep Residual Autoencoders for Expectation Maximization-inspired Dictionary Learning
• Rethinking the CSC Model for Natural Images