Information Theoretical Estimators Toolbox

Abstract

We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes a prototype application in a central problem class of signal processing, independent subspace analysis and its extensions.

5 pages; ITE toolbox: https://bitbucket.org/szzoli/ite/

References in corpus (1)

• Undercomplete Blind Subspace Deconvolution

Cited by in corpus (24)

• Distinguishing cause from effect using observational data: methods and benchmarks
• Estimating Mutual Information for Discrete-Continuous Mixtures
• Efficient Estimation of Mutual Information for Strongly Dependent Variables
• A Theory of Usable Information Under Computational Constraints
• Functional Connectivity via Total Correlation: Analytical results in Visual Areas
• Direct estimation of density functionals using a polynomial basis
• MONK -- Outlier-Robust Mean Embedding Estimation by Median-of-Means
• Inflation as an Information Bottleneck - A strategy for identifying universality classes and making robust predictions
• Influence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations
• Computing Entropies With Nested Sampling
• Copula Index for Detecting Dependence and Monotonicity between Stochastic Signals
• Information Theory in Density Destructors
• Shredder: Learning Noise Distributions to Protect Inference Privacy
• Finite-Sample Analysis of Fixed-k Nearest Neighbor Density Functional Estimators
• Picking Up Quantization Steps for Compressed Image Classification
• Information Theory Measures via Multidimensional Gaussianization
• Random Matrix based Physical Layer Secret Key Generation in Static Channels
• Learning to compress and search visual data in large-scale systems
• Enabling Inference Privacy with Adaptive Noise Injection
• The Effect of Noise Level on Causal Identification with Additive Noise Models
• Efficient Multi-Modal Embeddings from Structured Data
• Adaptive MCMC via Combining Local Samplers
• On the Capacity of Correlated Phase-Noise Channels: An Electro-Optic Frequency Comb Example
• Causal Identification with Additive Noise Models: Quantifying the Effect of Noise