14 papers
Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs
Alex Colagrande, Paul Caillon, Eva Feillet +1
Neural operators provide deep neural networks for learning mappings between function spaces. Among them, the Fourier Neural Operator (FNO) is particularly effective: its spectral c…
Limits of Resolution Equivariance in Fourier Neural Operators
Alex Colagrande, Paul Caillon, Eva Feillet +1
Fourier Neural Operators are often assumed to generalize across spatial resolutions, enabling training on a coarse grid and deployment on a finer grid. We test this assumption by c…
Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing
Erwan Fagnou, Paul Caillon, Blaise Delattre +1
Token mixing layers play a key role in how language models can learn and generate long-range dependencies. Their efficiency relies on the necessary trade-off between decoding speed…
Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
Hangyue Zhao, Paul Caillon, Erwan Fagnou +1
Entity tracking requires maintaining and updating latent states for entities and attributes over long sequences. Recent task-specific attention operators can compress deep Transfor…
Forward Only Learning for Orthogonal Neural Networks of any Depth
Paul Caillon, Alex Colagrande, Erwan Fagnou +2
Backpropagation is still the de facto algorithm used today to train neural networks. With the exponential growth of recent architectures, the computational cost of this algorithm a…
On the MIA Vulnerability Gap Between Private GANs and Diffusion Models
Ilana Sebag, Jean-Yves Franceschi, Alain Rakotomamonjy +2
Generative Adversarial Networks (GANs) and diffusion models have emerged as leading approaches for high-quality image synthesis. While both can be trained under differential privac…