4 papers
Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block model
O. Duranthon, L. Zdeborová
Graph neural networks (GNNs) are designed to process data associated with graphs. They are finding an increasing range of applications; however, as with other modern machine learni…
Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model
Hugo Tabanelli, Pierre Mergny, Lenka Zdeborova +1
We study the recovery of multiple high-dimensional signals from two noisy, correlated modalities: a spiked matrix and a spiked tensor sharing a common low-rank structure. This sett…
Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks
Luca Arnaboldi, Bruno Loureiro, Ludovic Stephan +2
We study the dynamics of stochastic gradient descent (SGD) for a class of sequence models termed Sequence Single-Index (SSI) models, where the target depends on a single direction…
On the Atypical Solutions of the Symmetric Binary Perceptron
Damien Barbier, Ahmed El Alaoui, Florent Krzakala +1
We study the random binary symmetric perceptron problem, focusing on the behavior of rare high-margin solutions. While most solutions are isolated, we demonstrate that these rare s…