Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition

Abstract

Adaptive sparse coding methods learn a possibly overcomplete set of basis functions, such that natural image patches can be reconstructed by linearly combining a small subset of these bases. The applicability of these methods to visual object recognition tasks has been limited because of the prohibitive cost of the optimization algorithms required to compute the sparse representation. In this work we propose a simple and efficient algorithm to learn basis functions. After training, this model also provides a fast and smooth approximator to the optimal representation, achieving even better accuracy than exact sparse coding algorithms on visual object recognition tasks.

References in corpus (10)

• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Rejoinder to "Least angle regression" by Efron et al

Cited by in corpus (50)

• Auto-Encoding Variational Bayes
• An Introduction to Variational Autoencoders
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
• Video (language) modeling: a baseline for generative models of natural videos
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Towards Biologically Plausible Deep Learning
• Stacked What-Where Auto-encoders
• Neural Variational Inference and Learning in Belief Networks
• k-Sparse Autoencoders
• Deep Networks for Image Super-Resolution with Sparse Prior
• Deep Learning of Representations: Looking Forward
• Representation Learning: A Review and New Perspectives
• What Do We Understand About Convolutional Networks?
• Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
• New Algorithms for Learning Incoherent and Overcomplete Dictionaries
• Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields
• Distributed optimization of deeply nested systems
• Zero-bias autoencoders and the benefits of co-adapting features
• On Learning Where To Look
• Sparse Recovery and Dictionary Learning from Nonlinear Compressive Measurements
• Image denoising with multi-layer perceptrons, part 2: training trade-offs and analysis of their mechanisms
• Learning with hidden variables
• Learning Efficient Structured Sparse Models
• Sparse similarity-preserving hashing
• Learning invariant features through local space contraction
• Learning efficient sparse and low rank models
• Why Regularized Auto-Encoders learn Sparse Representation?
• Learning to Adapt by Minimizing Discrepancy
• Spectral classification using convolutional neural networks
• Online Learning for Matrix Factorization and Sparse Coding
• MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning
• Minimizing FLOPs to Learn Efficient Sparse Representations
• Adversarial Robustness of Supervised Sparse Coding
• Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting
• Learning Robust Low-Rank Representations
• LSALSA: Accelerated Source Separation via Learned Sparse Coding
• Generative Deep Deconvolutional Learning
• On the Reconstruction Risk of Convolutional Sparse Dictionary Learning
• Multi-Label Zero-Shot Human Action Recognition via Joint Latent Ranking Embedding
• On Optimality Conditions for Auto-Encoder Signal Recovery
• Rank Ordered Autoencoders
• A manifold learning perspective on representation learning: Learning decoder and representations without an encoder
• Learnable Subspace Clustering
• Towards Understanding Residual and Dilated Dense Neural Networks via Convolutional Sparse Coding
• CTNN: Corticothalamic-inspired neural network
• Tensor-Generative Adversarial Network with Two-dimensional Sparse Coding: Application to Real-time Indoor Localization
• Learning Sparsity of Representations with Discrete Latent Variables
• A linear approach for sparse coding by a two-layer neural network
• What is the Best Feature Learning Procedure in Hierarchical Recognition Architectures?