4 papers
Minimal-Norm Univariate Two-Layer ReLU Classification: Exact Solutions and Global Optimality with Skip Connections
Karolina Drabik, Ben Lewis, Antoni Puch +4
We study minimal-norm interpolation and -regularized logistic-loss minimization for binary classification by univariate two-layer ReLU networks. We give complete geometric…
Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias
James Town, Etienne Boursier, Ben Lewis +2
The successful training of neural networks hinges on the use of first order optimization methods, yet the theoretical characterization of these methods remains incomplete. This is…
Velocity and stroke rate reconstruction of canoe sprint team boats based on panned and zoomed video recordings
Julian Ziegler, Daniel Matthes, Finn Gerdts +5
Pacing strategies, defined by velocity and stroke rate profiles, are essential for peak performance in canoe sprint. While GPS is the gold standard for analysis, its limited availa…
Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization
Etienne Boursier, Matthew Bowditch, Matthias Englert +1
The optimization of neural networks under weight decay remains poorly understood from a theoretical standpoint. While weight decay is standard practice in modern training procedure…