65 citations · 83 across the 6 of their papers we have counts for
4 papers · 1 filter
Implicit Regularization Makes Overparameterized Asymmetric Matrix Sensing Robust to Perturbations
Johan S. Wind
Several key questions remain unanswered regarding overparameterized learning models. It is unclear how (stochastic) gradient descent finds solutions that generalize well, and in pa…
Minimal Convex Environmental Contours
Åsmund Hausken Sande, Johan S. Wind
We develop a numerical method for the computation of a minimal convex and compact set, , in the sense of mean width. This minimisation is constraine…
Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks
Johan S. Wind, Vegard Antun, Anders C. Hansen
Understanding the implicit regularization imposed by neural network architectures and gradient based optimization methods is a key challenge in deep learning and AI. In this work w…
RWKV: Reinventing RNNs for the Transformer Era
Bo Peng, Eric Alcaide, Quentin Anthony +31
Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence leng…