Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias
James Town, Etienne Boursier, Ben Lewis +2
The successful training of neural networks hinges on the use of first order optimization methods, yet the theoretical characterization of these methods remains incomplete. This is…
cs.LG2025
Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization
Etienne Boursier, Matthew Bowditch, Matthias Englert +1
The optimization of neural networks under weight decay remains poorly understood from a theoretical standpoint. While weight decay is standard practice in modern training procedure…