Iterative Gaussianization: from ICA to Random Rotations

Abstract

Most signal processing problems involve the challenging task of multidimensional probability density function (PDF) estimation. In this work, we propose a solution to this problem by using a family of Rotation-based Iterative Gaussianization (RBIG) transforms. The general framework consists of the sequential application of a univariate marginal Gaussianization transform followed by an orthonormal transform. The proposed procedure looks for differentiable transforms to a known PDF so that the unknown PDF can be estimated at any point of the original domain. In particular, we aim at a zero mean unit covariance Gaussian for convenience. RBIG is formally similar to classical iterative Projection Pursuit (PP) algorithms. However, we show that, unlike in PP methods, the particular class of rotations used has no special qualitative relevance in this context, since looking for interestingness is not a critical issue for PDF estimation. The key difference is that our approach focuses on the univariate part (marginal Gaussianization) of the problem rather than on the multivariate part (rotation). This difference implies that one may select the most convenient rotation suited to each practical application. The differentiability, invertibility and convergence of RBIG are theoretically and experimentally analyzed. Relation to other methods, such as Radial Gaussianization (RG), one-class support vector domain description (SVDD), and deep neural networks (DNN) is also pointed out. The practical performance of RBIG is successfully illustrated in a number of multidimensional problems such as image synthesis, classification, denoising, and multi-information estimation.

References in corpus (2)

• Natural Image Coding in V1: How Much Use is Orientation Selectivity?
• On the Suitable Domain for SVM Training in Image Coding

Cited by in corpus (35)

• Unsupervised Deep Feature Extraction for Remote Sensing Image Classification
• Normalizing Flows for Probabilistic Modeling and Inference
• Optimizing the Latent Space of Generative Networks
• An introduction to sampling via measure transport
• Transport map accelerated Markov chain Monte Carlo
• Multi-Fold Gabor, PCA and ICA Filter Convolution Descriptor for Face Recognition
• Gaussian-binary Restricted Boltzmann Machines on Modeling Natural Image Statistics
• Dimensionality Reduction via Regression in Hyperspectral Imagery
• Nonlinearities and Adaptation of Color Vision from Sequential Principal Curves Analysis
• Functional Connectome of the Human Brain with Total Correlation
• Principal Polynomial Analysis
• Cosmological Inference from Galaxy-Clustering Power Spectrum: Gaussianization and Covariance Decomposition
• Geostatistics in the presence of multivariate complexities: comparison of multi-Gaussian transforms
• Functional Connectivity via Total Correlation: Analytical results in Visual Areas
• Greedy inference with structure-exploiting lazy maps
• From Dependence to Causation
• Inference via low-dimensional couplings
• Encoding Robust Representation for Graph Generation
• Unsupervised Anomaly and Change Detection with Multivariate Gaussianization
• I see artifacts: ICA-based EEG artifact removal does not improve deep network decoding across three BCI tasks
• Gaussianization Flows
• Information Theory in Density Destructors
• Information Flow in Biological Networks for Color Vision
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Spatial noise-aware temperature retrieval from infrared sounder data
• Gaussian Lower Bound for the Information Bottleneck Limit
• Visual Information flow in Wilson-Cowan networks
• Sliced Iterative Normalizing Flows
• Information Theory Measures via Multidimensional Gaussianization
• Transfer Learning with Convolutional Networks for Atmospheric Parameter Retrieval
• Rapid updating of multivariate resource models based on new information using EnKF-MDA and multi-Gaussian transformation
• Generative flow-based warm start of the variational quantum eigensolver
• A Multivariate Model for Representing Semantic Non-compositionality
• Pluri-Gaussian rapid updating of geological domains
• Iterative Alignment Flows