9 citations · 9 across the 5 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.LG2025
Statistical Advantage of Softmax Attention: Insights from Single-Location Regression
O. Duranthon, P. Marion, C. Boyer +2
Large language models rely on attention mechanisms with a softmax activation. Yet the dominance of softmax over alternatives (e.g., component-wise or linear) remains poorly underst…
cond-mat.dis-nn2025
Statistical physics analysis of graph neural networks: Approaching optimality in the contextual stochastic block model
O. Duranthon, L. Zdeborová
Graph neural networks (GNNs) are designed to process data associated with graphs. They are finding an increasing range of applications; however, as with other modern machine learni…