6 papers
Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation
Alberto Foresti, Ivan Butakov, Alexander Tolmachev +3
Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low-dimensiona…
InfoBridge: Mutual Information estimation via Bridge Matching
Sergei Kholkin, Ivan Butakov, Evgeny Burnaev +2
Diffusion bridge models have recently become a powerful tool in the field of generative modeling. In this work, we leverage their power to address another important problem in mach…
Discrete Bridges for Mutual Information Estimation
Iryna Zabarianska, Sergei Kholkin, Grigoriy Ksenofontov +2
Diffusion bridge models in both continuous and discrete state spaces have recently become powerful tools in the field of generative modeling. In this work, we leverage the discrete…
FMMI: Flow Matching Mutual Information Estimation
Ivan Butakov, Alexander Semenenko, Valeriya Kirova +2
We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and m…
Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence
Alexander Semenenko, Ivan Butakov, Alexey Frolov +1
Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as fast…
Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax
Ivan Butakov, Alexander Semenenko, Alexander Tolmachev +3
Deep InfoMax (DIM) is a well-established method for self-supervised representation learning (SSRL) based on maximization of the mutual information between the input and the output…