20 citations · 23 across the 4 of their papers we have counts for
3 papers · 1 filter
Challenges in Non-Polymeric Crystal Structure Prediction: Why a Geometric, Permutation-Invariant Loss is Needed
Emmanuel Jehanno, Romain Menegaux, Julien Mairal +1
Crystalline structure prediction is an essential prerequisite for designing materials with targeted properties. Yet, it is still an open challenge in materials design and drug disc…
Self-Attention in Colors: Another Take on Encoding Graph Structure in Transformers
Romain Menegaux, Emmanuel Jehanno, Margot Selosse +1
We introduce a novel self-attention mechanism, which we call CSA (Chromatic Self-Attention), which extends the notion of attention scores to attention _filters_, independently modu…
Rethinking Gauss-Newton for learning over-parameterized models
Michael Arbel, Romain Menegaux, Pierre Wolinski
This work studies the global convergence and implicit bias of Gauss Newton's (GN) when optimizing over-parameterized one-hidden layer networks in the mean-field regime. We first es…