113 citations · 392 across the 22 of their papers we have counts for
5 papers · 1 filter
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens
Vittorio Erba, Emanuele Troiani, Luca Biggio +2
Current progress in artificial intelligence is centered around so-called large language models that consist of neural networks processing long sequences of high-dimensional vectors…
Injectivity of ReLU networks: perspectives from statistical physics
Antoine Maillard, Afonso S. Bandeira, David Belius +2
When can the input of a ReLU neural network be inferred from its output? In other words, when is the network injective? We consider a single layer, , w…
Optimal denoising of rotationally invariant rectangular matrices
Emanuele Troiani, Vittorio Erba, Florent Krzakala +2
In this manuscript we consider denoising of large rectangular matrices: given a noisy observation of a signal matrix, what is the best way of recovering the signal matrix itself? F…
Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising
Antoine Maillard, Florent Krzakala, Marc Mézard +1
Factorization of matrices where the rank of the two factors diverges linearly with their sizes has many applications in diverse areas such as unsupervised representation learning,…
High-temperature Expansions and Message Passing Algorithms
Antoine Maillard, Laura Foini, Alejandro Lage Castellanos +3
Improved mean-field technics are a central theme of statistical physics methods applied to inference and learning. We revisit here some of these methods using high-temperature expa…