65 citations · 83 across the 6 of their papers we have counts for
6 papers
RWKV-7 "Goose" with Expressive Dynamic State Evolution
Bo Peng, Ruichong Zhang, Daniel Goldstein +15
We present RWKV-7 "Goose", a new sequence modeling architecture with constant memory usage and constant inference time per token. Despite being trained on dramatically fewer tokens…
Implicit Regularization Makes Overparameterized Asymmetric Matrix Sensing Robust to Perturbations
Johan S. Wind
Several key questions remain unanswered regarding overparameterized learning models. It is unclear how (stochastic) gradient descent finds solutions that generalize well, and in pa…
Minimal Convex Environmental Contours
Åsmund Hausken Sande, Johan S. Wind
We develop a numerical method for the computation of a minimal convex and compact set, , in the sense of mean width. This minimisation is constraine…
Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks
Johan S. Wind, Vegard Antun, Anders C. Hansen
Understanding the implicit regularization imposed by neural network architectures and gradient based optimization methods is a key challenge in deep learning and AI. In this work w…
RWKV: Reinventing RNNs for the Transformer Era
Bo Peng, Eric Alcaide, Quentin Anthony +31
Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence leng…
The Tracking Machine Learning challenge : Accuracy phase
Sabrina Amrouche, Laurent Basara, Paolo Calafiura +24
This paper reports the results of an experiment in high energy physics: using the power of the "crowd" to solve difficult experimental problems linked to tracking accurately the tr…