6 papers · 1 filter
Across the Loss Landscape with Progressive Growth
Paul Caillon, Christophe Cerisara, Alexandre Allauzen
Deep neural networks generalize well despite their highly nonconvex, overparameterized loss landscapes, a phenomenon often associated with the geometry of the minima found by stoch…
Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs
Alex Colagrande, Paul Caillon, Eva Feillet +1
Neural operators provide deep neural networks for learning mappings between function spaces. Among them, the Fourier Neural Operator (FNO) is particularly effective: its spectral c…
Limits of Resolution Equivariance in Fourier Neural Operators
Alex Colagrande, Paul Caillon, Eva Feillet +1
Fourier Neural Operators are often assumed to generalize across spatial resolutions, enabling training on a coarse grid and deployment on a finer grid. We test this assumption by c…
Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing
Erwan Fagnou, Paul Caillon, Blaise Delattre +1
Token mixing layers play a key role in how language models can learn and generate long-range dependencies. Their efficiency relies on the necessary trade-off between decoding speed…
Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
Hangyue Zhao, Paul Caillon, Erwan Fagnou +1
Entity tracking requires maintaining and updating latent states for entities and attributes over long sequences. Recent task-specific attention operators can compress deep Transfor…
Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing
Blaise Delattre, Hengyu Wu, Paul Caillon +2
Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in i…