148 citations · 493 across the 13 of their papers we have counts for
13 papers
Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks
Charith Peris, Lizhen Tan, Thomas Gueudre +3
Teacher-student knowledge distillation is a popular technique for compressing today's prevailing large language models into manageable sizes that fit low-latency downstream applica…
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas +38
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller m…
Exploration-exploitation tradeoffs dictate the optimal distributions of phenotypes for populations subject to fitness fluctuations
Andrea De Martino, Thomas Gueudré, Mattia Miotto
We study a minimal model for the growth of a phenotypically heterogeneous population of cells subject to a fluctuating environment in which they can replicate (by exploiting availa…
Growth over time-correlated disorder: a spectral approach to Mean-field
Thomas Gueudré
We generalize a model of growth over a disordered environment, to a large class of Itō processes. In particular, we study how the microscopic properties of the noise influence the…
Scaling Description of Non-Local Rheology
Thomas Gueudré, Jie Lin, Alberto Rosso +1
Non-locality is crucial to understand the plastic flow of an amorphous material, and has been successfully described by the fluidity, along with a cooperativity length scale ξ. We…
Simultaneous identification of specifically interacting paralogs and inter-protein contacts by Direct-Coupling Analysis
Thomas Gueudré, Carlo Baldassi, Marco Zamparo +2
Understanding protein-protein interactions is central to our understanding of almost all complex biological processes. Computational tools exploiting rapidly growing genomic databa…
Criticality in the approach to failure in amorphous solids
Jie Lin, Thomas Gueudré, Alberto Rosso +1
Failure of amorphous solids is fundamental to various phenomena, including landslides and earthquakes. Recent experiments indicate that highly plastic regions form elongated struct…
Revisiting Directed Polymers with heavy-tailed disorder
Thomas Gueudré, Pierre Le Doussal, Jean-Philippe Bouchaud +1
In this mostly numerical study, we revisit the statistical properties of the ground state of a directed polymer in a "hilly" disorder landscape, i.e. when the quenched diso…
Strong pinning of propagation fronts in adverse flow
Thomas Gueudré, Awadhesh Kumar Dubey, Laurent Talon +1
Reaction fronts evolving in a porous medium exhibit a rich dynamical behaviour. In presence of an adverse flow, experiments show that the front slows down and eventually gets pinne…
Statistics of shocks in a toy model with heavy tails
Thomas Gueudré, Pierre Le Doussal
We study the energy minimization for a particle in a quadratic well in presence of short-ranged heavy-tailed disorder, as a toy model for an elastic manifold. The discrete model is…
Explore or exploit? A generic model and an exactly solvable case
Thomas Gueudré, Alexander Dobrinevski, Jean-Philippe Bouchaud
Finding a good compromise between the exploitation of known resources and the exploration of unknown, but potentially more profitable choices, is a general problem, which arises in…
Directed polymer near a hard wall and KPZ equation in the half-space
Thomas Gueudre, Pierre Le Doussal
We study the directed polymer with fixed endpoints near an absorbing wall, in the continuum and in presence of disorder, equivalent to the KPZ equation on the half space with dropl…
Short time growth of a KPZ interface with flat initial conditions
Thomas Gueudre, Pierre Le Doussal, Alberto Rosso +2
The short time behavior of the 1+1 dimensional KPZ growth equation with a flat initial condition is obtained from the exact expressions of the moments of the partition function of…