1 citations · 2 across the 15 of their papers we have counts for
13 papers · 1 filter
Trading Complexity for Expressivity Through Structured Generalized Linear Token Mixing
Erwan Fagnou, Paul Caillon, Blaise Delattre +1
Token mixing layers play a key role in how language models can learn and generate long-range dependencies. Their efficiency relies on the necessary trade-off between decoding speed…
Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing
Blaise Delattre, Hengyu Wu, Paul Caillon +2
Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in i…
Forward Only Learning for Orthogonal Neural Networks of any Depth
Paul Caillon, Alex Colagrande, Erwan Fagnou +2
Backpropagation is still the de facto algorithm used today to train neural networks. With the exponential growth of recent architectures, the computational cost of this algorithm a…
On the Stability of Neural Networks in Deep Learning
Blaise Delattre
Deep learning has achieved remarkable success across a wide range of tasks, but its models often suffer from instability and vulnerability: small changes to the input may drastical…
Bridging the Theoretical Gap in Randomized Smoothing
Blaise Delattre, Paul Caillon, Quentin Barthélemy +2
Randomized smoothing has become a leading approach for certifying adversarial robustness in machine learning models. However, a persistent gap remains between theoretical certified…
Conditional Distribution Quantization in Machine Learning
Blaise Delattre, Sylvain Delattre, Alexandre Vérine +1
Conditional expectation \mathbb{E}(Y \mid X) often fails to capture the complexity of multimodal conditional distributions \mathcal{L}(Y \mid X). To address this, we propose using…
Accelerated Training through Iterative Gradient Propagation Along the Residual Path
Erwan Fagnou, Paul Caillon, Blaise Delattre +1
Despite being the cornerstone of deep learning, backpropagation is criticized for its inherent sequentiality, which can limit the scalability of very deep models. Such models faced…
Chain and Causal Attention for Efficient Entity Tracking
Erwan Fagnou, Paul Caillon, Blaise Delattre +1
This paper investigates the limitations of transformers for entity-tracking tasks in large language models. We identify a theoretical constraint, showing that transformers require…
Spectral Norm of Convolutional Layers with Circular and Zero Paddings
Blaise Delattre, Quentin Barthélemy, Alexandre Allauzen
This paper leverages the use of \emph{Gram iteration} an efficient, deterministic, and differentiable method for computing spectral norm with an upper bound guarantee. Designed for…
The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing
Blaise Delattre, Alexandre Araujo, Quentin Barthélemy +1
Real-life applications of deep neural networks are hindered by their unsteady predictions when faced with noisy inputs and adversarial attacks. The certified radius in this context…
Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration
Blaise Delattre, Quentin Barthélemy, Alexandre Araujo +1
Since the control of the Lipschitz constant has a great impact on the training stability, generalization, and robustness of neural networks, the estimation of this value is nowaday…
A Unified Algebraic Perspective on Lipschitz Neural Networks
Alexandre Araujo, Aaron Havens, Blaise Delattre +2
Important research efforts have focused on the design and training of neural networks with a controlled Lipschitz constant. The goal is to increase and sometimes guarantee the robu…
A Dynamical System Perspective for Lipschitz Neural Networks
Laurent Meunier, Blaise Delattre, Alexandre Araujo +1
The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building…