Learning Non-linear Wavelet Transformation via Normalizing Flow
arXiv:2101.11306
Abstract
Wavelet transformation stands as a cornerstone in modern data analysis and signal processing. Its mathematical essence is an invertible transformation that discerns slow patterns from fast ones in the frequency domain. Such an invertible transformation can be learned by a designed normalizing flow model. With a generalized lifting scheme as coupling layers, a factor-out layer resembling the downsampling, and parameter sharing at different levels of the model, one can train the normalizing flow to filter high-frequency elements at different levels, thus extending traditional linear wavelet transformations to learnable non-linear deep learning models. In this paper, a way of building such flow is proposed, along with a numerical analysis of the learned transformation. Then, we demonstrate the model's ability in image lossless compression, show it can achieve SOTA compression scores while achieving a small model size, substantial generalization ability, and the ability to handle high-dimensional data.
Main text: 8 pages, 4 figures. Supplement: 9 pages, 2 figures. Github link: https://github.com/li012589/NeuralWavelet
References in corpus (9)
- NICE: Non-linear Independent Components Estimation
- A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
- Guided Image Generation with Conditional Invertible Neural Networks
- Asymmetric numeral systems
- Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
- IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
- Wavelet Flow: Fast Training of High Resolution Normalizing Flows
- RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
- Using Wavelets and Spectral Methods to Study Patterns in Image-Classification Datasets