1 citations · 1 across the 2 of their papers we have counts for
4 papers
RWKV-7 "Goose" with Expressive Dynamic State Evolution
Bo Peng, Ruichong Zhang, Daniel Goldstein +15
We present RWKV-7 "Goose", a new sequence modeling architecture with constant memory usage and constant inference time per token. Despite being trained on dramatically fewer tokens…
Implicit Regularization Makes Overparameterized Asymmetric Matrix Sensing Robust to Perturbations
Johan S. Wind
Several key questions remain unanswered regarding overparameterized learning models. It is unclear how (stochastic) gradient descent finds solutions that generalize well, and in pa…
Minimal Convex Environmental Contours
Åsmund Hausken Sande, Johan S. Wind
We develop a numerical method for the computation of a minimal convex and compact set, , in the sense of mean width. This minimisation is constraine…
Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks
Johan S. Wind, Vegard Antun, Anders C. Hansen
Understanding the implicit regularization imposed by neural network architectures and gradient based optimization methods is a key challenge in deep learning and AI. In this work w…