Learning Non-linear Wavelet Transformation via Normalizing Flow
arXiv:2101.11306
Abstract
Wavelet transformation stands as a cornerstone in modern data analysis and signal processing. Its mathematical essence is an invertible transformation that discerns slow patterns from fast ones in the frequency domain. Such an invertible transformation can be learned by a designed normalizing flow model. With a generalized lifting scheme as coupling layers, a factor-out layer resembling the downsampling, and parameter sharing at different levels of the model, one can train the normalizing flow to filter high-frequency elements at different levels, thus extending traditional linear wavelet transformations to learnable non-linear deep learning models. In this paper, a way of building such flow is proposed, along with a numerical analysis of the learned transformation. Then, we demonstrate the model's ability in image lossless compression, show it can achieve SOTA compression scores while achieving a small model size, substantial generalization ability, and the ability to handle high-dimensional data.
arXiv commentMain text: 8 pages, 4 figures. Supplement: 9 pages, 2 figures. Github link: https://github.com/li012589/NeuralWavelet
References in corpus (16)
- NICE: Non-linear Independent Components Estimation
- Variational Inference with Normalizing Flows
- Density estimation using Real NVP
- A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
- Guided Image Generation with Conditional Invertible Neural Networks
- Neural Network Renormalization Group
- Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding
- Asymmetric numeral systems
- A-NICE-MC: Adversarial Training for MCMC
- Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
- IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
- Joint Sub-bands Learning with Clique Structures for Wavelet Domain Super-Resolution
- Neural Canonical Transformation with Symplectic Flows
- Wavelet Flow: Fast Training of High Resolution Normalizing Flows
- RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
- Using Wavelets and Spectral Methods to Study Patterns in Image-Classification Datasets