Optimal whitening and decorrelation

Abstract

Whitening, or sphering, is a common preprocessing step in statistical analysis to transform random variables to orthogonality. However, due to rotational freedom there are infinitely many possible whitening procedures. Consequently, there is a diverse range of sphering methods in use, for example based on principal component analysis (PCA), Cholesky matrix decomposition and zero-phase component analysis (ZCA), among others. Here we provide an overview of the underlying theory and discuss five natural whitening procedures. Subsequently, we demonstrate that investigating the cross-covariance and the cross-correlation matrix between sphered and original variables allows to break the rotational invariance and to identify optimal whitening transformations. As a result we recommend two particular approaches: ZCA-cor whitening to produce sphered variables that are maximally similar to the original variables, and PCA-cor whitening to obtain sphered variables that maximally compress the original variables.

14 pages, 2 tables

References in corpus (3)

• Covariance Estimation: The GLM and Regularization Perspectives
• Gene ranking and biomarker discovery under correlation
• A novel algorithm for simultaneous SNP selection in high-dimensional genome-wide association studies

Cited by in corpus (60)

• Automatic Classification of Defective Photovoltaic Module Cells in Electroluminescence Images
• Infrared and Visible Image Fusion with ResNet and zero-phase component analysis
• Randomized Matrix Decompositions using R
• DijetGAN: A Generative-Adversarial Network Approach for the Simulation of QCD Dijet Events at the LHC
• Real-time stain-free classification of cancer cells and blood cells using interferometric phase microscopy and machine learning
• Robust Differentiable SVD
• Detecting and diagnosing prior and likelihood sensitivity with power-scaling
• Unsupervised machine learning for detection of phase transitions in off-lattice systems I. Foundations
• Normalization Techniques in Training DNNs: Methodology, Analysis and Application
• Unsupervised machine learning for detection of phase transitions in off-lattice systems II. Applications
• A whitening approach to probabilistic canonical correlation analysis for omics data integration
• Generation of scenarios from calibrated ensemble forecasts with a dual ensemble copula coupling approach
• Discrete least-squares finite element methods
• Robust SINR-Constrained Symbol-Level Multiuser Precoding with Imperfect Channel Knowledge
• Multiple-view clustering for identifying subject clusters and brain sub-networks using functional connectivity matrices without vectorization
• How informative are summaries of the cosmic 21-cm signal?
• Inferring the photometric and size evolution of galaxies from image simulations
• A fractional PDE model for turbulent velocity fields near solid walls
• Quaternion Convolutional Neural Networks: Current Advances and Future Directions
• Using Supervised Deep-Learning to Model Edge-FBG Shape Sensors
• Keep it Simple: Image Statistics Matching for Domain Adaptation
• Backpropagation-Friendly Eigendecomposition
• A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• Handling confounding variables in statistical shape analysis -- application to cardiac remodelling
• Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms
• Interactive Visual Data Exploration with Subjective Feedback: An Information-Theoretic Approach
• A Closed-form Solution to Universal Style Transfer
• Cluster Index Modulation for Reconfigurable Intelligent Surface-Assisted mmWave Massive MIMO
• Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting
• Stellar Karaoke: deep blind separation of terrestrial atmospheric effects out of stellar spectra by velocity whitening
• Deep Multi-View Learning via Task-Optimal CCA
• ThundeRiNG: Generating Multiple Independent Random Number Sequences on FPGAs
• Covariance-modulated optimal transport and gradient flows
• Robust Hidden Topology Identification in Distribution Systems
• MIP-BOOST: Efficient and Effective $L_0$ Feature Selection for Linear Regression
• CAST: A Correlation-based Adaptive Spectral Clustering Algorithm on Multi-scale Data
• Machine learning based identification of buried objects using sparse whitened NMF
• Linear and non-linear Modified Gravity forecasts with future surveys
• Quantum Annealing for Semi-Supervised Learning
• The Role of Integration Cycles in Complex Langevin Simulations
• Decorrelation Deep Learning for Fingerprint-based Indoor Localization
• Is Batch Norm unique? An empirical investigation and prescription to emulate the best properties of common normalizers without batch dependence
• Adjusted composite likelihood for robust Bayesian meta-analysis
• On Batch Orthogonalization Layers
• Feature Whitening via Gradient Transformation for Improved Convergence
• The Linear Relationship between Temporal Persistence, Number of Independent Features and Target EER
• Volatility model calibration with neural networks a comparison between direct and indirect methods
• Posterior Collapse as a Phase Transition in Variational Autoencoders
• Classification of high-dimensional data with spiked covariance matrix structure
• Robust Data Interpretation for Perturbed Nulling Interferometers via Proper Handling of Correlated Errors
• Context Normalization Layer with Applications
• Stochastic Whitening Batch Normalization
• Interpretable Face Manipulation Detection via Feature Whitening
• Why Temporal Persistence of Biometric Features is so Valuable for Classification Performance
• Human-guided data exploration using randomisation
• SACOBRA with Online Whitening for Solving Optimization Problems with High Conditioning
• A Human Ear Reconstruction Autoencoder
• Group Whitening: Balancing Learning Efficiency and Representational Capacity
• An Investigation into the Stochasticity of Batch Whitening