activity
20182020
most citedEntropy and mutual information in models of deep neural networks

139 citations · 163 across the 3 of their papers we have counts for

collaborators

5 papers

cs.IT2020

Tensor estimation with structured priors

Clément Luneau, Nicolas Macris

We consider rank-one symmetric tensor estimation when the tensor is corrupted by Gaussian noise and the spike forming the tensor is a structured signal coming from a generalized li…

cs.IT2020

Information theoretic limits of learning a sparse rule

Clément Luneau, Jean Barbier, Nicolas Macris

We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signa…

math.PR2020★ 18 cited

High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case

Clément Luneau, Nicolas Macris, Jean Barbier

We consider the problem of estimating a rank-one nonsymmetric matrix under additive white Gaussian noise. The matrix to estimate can be written as the outer product of two vectors…

cs.IT2019★ 6 cited

Mutual information for low-rank even-order symmetric tensor estimation

Clément Luneau, Jean Barbier, Nicolas Macris

We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its asymptotic mutual information when the tenso…

cs.LG2018★ 139 cited

Entropy and mutual information in models of deep neural networks

Marylou Gabrié, Andre Manoel, Clément Luneau +4

We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutu…