output
20022026
most citedPaying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

1.6k citations

Showing cs.LGShow all

12 papers · 1 filter

cs.LG2025

Network-Wide Traffic Volume Estimation from Speed Profiles using a Spatio-Temporal Graph Neural Network with Directed Spatial Attention

Léo Hein, Giovanni de Nunzio, Giovanni Chierchia +2

Existing traffic volume estimation methods typically address either forecasting traffic on sensor-equipped roads or spatially imputing missing volumes using nearby sensors. While f…

cs.LG2025

Sliced ReLU attention: Quasi-linear contextual expressivity via sorting

François-Xavier Vialard, Siwan Boufadène

We introduce sliced ReLU attention, a new attention mechanism that departs structurally from both softmax and its approximation alternatives. Instead of applying a nonlinearity to…

cs.LG2025

Data Science: a Natural Ecosystem

Emilio Porcu, Roy El Moukari, Laurent Najman +2

This manuscript provides a systemic and data-centric view of what we term essential data science, as a natural ecosystem with challenges and missions stemming from the fusion of da…

cs.LG2025

Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime

Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard

We study the convergence of gradient methods for the training of mean-field single-hidden-layer neural networks with square loss. For this high-dimensional and non-convex optimizat…

cs.LG2025

The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks

Sholom Schechtman, Nicolas Schreuder

We analyze the implicit bias of constant step stochastic subgradient descent (SGD). We consider the setting of binary classification with homogeneous neural networks - a large clas…

cs.LG20231 cited

SWMLP: Shared Weight Multilayer Perceptron for Car Trajectory Speed Prediction using Road Topographical Features

Sarah Almeida Carneiro, Giovanni Chierchia, Jean Charléty +2

Although traffic is one of the massively collected data, it is often only available for specific regions. One concern is that, although there are studies that give good results for…