Trainlets: Dictionary Learning in High Dimensions

Abstract

Sparse representations has shown to be a very powerful model for real world signals, and has enabled the development of applications with notable performance. Combined with the ability to learn a dictionary from signal examples, sparsity-inspired algorithms are often achieving state-of-the-art results in a wide variety of tasks. Yet, these methods have traditionally been restricted to small dimensions mainly due to the computational constraints that the dictionary learning problem entails. In the context of image processing, this implies handling small image patches. In this work we show how to efficiently handle bigger dimensions and go beyond the small patches in sparsity-based signal and image processing methods. We build our approach based on a new cropped wavelet decomposition, which enables a multi-scale analysis with virtually no border effects. We then employ this as the base dictionary within a double sparsity model to enable the training of adaptive dictionaries. To cope with the increase of training data, while at the same time improving the training performance, we present an Online Sparse Dictionary Learning (OSDL) algorithm to train this model effectively, enabling it to handle millions of examples. This work shows that dictionary learning can be up-scaled to tackle a new level of signal dimensions, obtaining large adaptable atoms that we call trainlets.

Cited by in corpus (16)

• Multi-Layer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
• Approximate fast graph Fourier transforms via multi-layer sparse approximations
• Dictionary Learning for Adaptive GPR Landmine Classification
• Patch2Self: Denoising Diffusion MRI with Self-Supervised Learning
• Convolutional Neural Networks Analyzed via Convolutional Sparse Coding
• Con-Patch: When a Patch Meets its Context
• Optimization of light fields in ghost imaging using dictionary learning
• Learning Fast Sparsifying Transforms
• MMSE Approximation For Sparse Coding Algorithms Using Stochastic Resonance
• A Local Block Coordinate Descent Algorithm for the Convolutional Sparse Coding Model
• Stable safe screening and structured dictionaries for faster L1 regularization
• Learning a collaborative multiscale dictionary based on robust empirical mode decomposition
• Optimized Structured Sparse Sensing Matrices for Compressive Sensing
• A Tree-based Dictionary Learning Framework
• Online Learning Sensing Matrix and Sparsifying Dictionary Simultaneously for Compressive Sensing
• Sparse Models for Machine Learning