9 papers
Forward Only Learning for Orthogonal Neural Networks of any Depth
Paul Caillon, Alex Colagrande, Erwan Fagnou +2
Backpropagation is still the de facto algorithm used today to train neural networks. With the exponential growth of recent architectures, the computational cost of this algorithm a…
On the MIA Vulnerability Gap Between Private GANs and Diffusion Models
Ilana Sebag, Jean-Yves Franceschi, Alain Rakotomamonjy +2
Generative Adversarial Networks (GANs) and diffusion models have emerged as leading approaches for high-quality image synthesis. While both can be trained under differential privac…
Improving Diversity in Language Models: When Temperature Fails, Change the Loss
Alexandre Verine, Florian Le Bronnec, Kunhao Zheng +3
Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach…
Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physics
Alex Colagrande, Paul Caillon, Eva Feillet +1
Transformers have become the de facto standard for a wide range of tasks, from image classification to physics simulations. Despite their impressive performance, the quadratic comp…
Bridging the Theoretical Gap in Randomized Smoothing
Blaise Delattre, Paul Caillon, Quentin Barthélemy +2
Randomized smoothing has become a leading approach for certifying adversarial robustness in machine learning models. However, a persistent gap remains between theoretical certified…
Fast Training of Recurrent Neural Networks with Stationary State Feedbacks
Paul Caillon, Erwan Fagnou, Alexandre Allauzen
Recurrent neural networks (RNNs) have recently demonstrated strong performance and faster inference than Transformers at comparable parameter budgets. However, the recursive gradie…