1.3k citations · 1.7k across the 21 of their papers we have counts for
22 papers
Cross-Entropy Games and Frost Training
Arthur Renard, Franck Gabriel, Valentin Hartmann +1
We present Frost Training, a method for improving Monte Carlo-based policy optimization for a large family of LLM-as-a-judge tasks called Cross-Entropy Games. The key idea is to ex…
Cognitive Training for Language Models: Towards General Capabilities via Cross-Entropy Games
Clément Hongler, Franck Gabriel, Valentin Hartmann +2
Defining a constructive process to build general capabilities for language models in an automatic manner is considered an open problem in artificial intelligence. Towards this, we…
Kernel-Smoothed Scores for Denoising Diffusion: A Bias-Variance Study
Franck Gabriel, François Ged, Maria Han Veiga +1
Diffusion models now set the benchmark in high-fidelity generative sampling, yet they can, in principle, be prone to memorization. In this case, their learned score overfits the fi…
Freeness of type and conditional freeness for random matrices
Guillaume Cébron, Antoine Dahlqvist, Franck Gabriel
The asymptotic freeness of independent unitarily invariant random matrices holds in expectation up to . An already known consequence is the infinitesimal fre…
Feature Learning in -regularized DNNs: Attraction/Repulsion and Sparsity
Arthur Jacot, Eugene Golikov, Clément Hongler +1
We study the loss surface of DNNs with regularization. We show that the loss in terms of the parameters can be reformulated into a loss in terms of the layerwise activation…
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
Arthur Jacot, François Ged, Berfin Şimşek +2
The dynamics of Deep Linear Networks (DLNs) is dramatically affected by the variance of the parameters at initialization . For DLNs of width , we show a phase transit…
Smart Proofs via Smart Contracts: Succinct and Informative Mathematical Derivations via Decentralized Markets
Sylvain Carré, Franck Gabriel, Clément Hongler +2
Modern mathematics is built on the idea that proofs should be translatable into formal proofs, whose validity is an objective question, decidable by a computer. Yet, in practice, p…
Kernel Alignment Risk Estimator: Risk Prediction from Training Data
Arthur Jacot, Berfin Şimşek, Francesco Spadaro +2
We study the risk (i.e. generalization error) of Kernel Ridge Regression (KRR) for a kernel with ridge and i.i.d. observations. For this, we introduce two objects: the Si…
Implicit Regularization of Random Feature Models
Arthur Jacot, Berfin Şimşek, Francesco Spadaro +2
Random Feature (RF) models are used as efficient parametric approximations of kernel methods. We investigate, by means of random matrix theory, the connection between Gaussian RF m…
The asymptotic spectrum of the Hessian of DNN throughout training
Arthur Jacot, Franck Gabriel, Clément Hongler
The dynamics of DNNs during gradient descent is described by the so-called Neural Tangent Kernel (NTK). In this article, we show that the NTK allows one to gain precise insight int…
Order and Chaos: NTK views on DNN Normalization, Checkerboard and Boundary Artifacts
Arthur Jacot, Franck Gabriel, François Ged +1
We analyze architectural features of Deep Neural Networks (DNNs) using the so-called Neural Tangent Kernel (NTK), which describes the training and generalization of DNNs in the inf…
Geometric stochastic heat equations
Yvain Bruned, Franck Gabriel, Martin Hairer +1
We consider a natural class of -valued one-dimensional stochastic PDEs driven by space-time white noise that is formally invariant under the action of the diffeomorph…
Scaling description of generalization with number of parameters in deep learning
Mario Geiger, Arthur Jacot, Stefano Spigler +6
Supervised deep learning involves the training of neural networks with a large number of parameters. For large enough , in the so-called over-parametrized regime, one can es…
Insider Trading with Penalties
Sylvain Carré, Pierre Collin-Dufresne, Franck Gabriel
We consider a one-period Kyle (1985) framework where the insider can be subject to a penalty if she trades. We establish existence and uniqueness of equilibrium for virtually any p…
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, Clément Hongler
At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit, thus connecting them to kernel methods. We prove that the evo…
Large permutation invariant random matrices are asymptotically free over the diagonal
Benson Au, Guillaume Cébron, Antoine Dahlqvist +2
We prove that independent families of permutation invariant random matrices are asymptotically free over the diagonal, both in probability and in expectation, under a uniform bound…
The Makeenko-Migdal equation for Yang-Mills theory on compact surfaces
Bruce K. Driver, Franck Gabriel, Brian C. Hall +1
We prove the Makeenko-Migdal equation for two-dimensional Euclidean Yang-Mills theory on an arbitrary compact surface, possibly with boundary. In particular, we show that two of th…
The generalized master fields
Guillaume Cébron, Antoine Dahlqvist, Franck Gabriel
The master field is the large limit of the Yang-Mills measure on the Euclidean plane. It can be viewed as a non-commutative process indexed by paths on the plane. We construct…
A combinatorial theory of random matrices III: random walks on , ramified coverings and the Yang-Mills measure
Franck Gabriel
The aim of this article is to study some asymptotics of a natural model of random ramified coverings on the disk of degree . We prove that the monodromy field, called also the h…
Combinatorial theory of permutation-invariant random matrices II: cumulants, freeness and Levy processes
Franck Gabriel
The -tracial algebras are algebras endowed with multi-linear forms, compatible with the product, and indexed by partitions. Using the notion of -cumulants…
Combinatorial theory of permutation-invariant random matrices I: partitions, geometry and renormalization
Franck Gabriel
In this article, we define and study a geometry and an order on the set of partitions of an even number of objects. One of the definitions involves the partition algebra, a structu…
Planar Markovian Holonomy Fields
Franck Gabriel
We study planar random holonomy fields which are processes indexed by paths on the plane which behave well under the concatenation and orientation-reversing operations on paths. We…