49 citations · 49 across the 6 of their papers we have counts for
4 papers · 1 filter
The Power of Second Order Methods for Sequence Preconditioning
Annie Marsden, Elad Hazan
Sequence prediction methods for linear dynamical systems with long memory, i.e. marginally stable systems, typically achieve regret that grows linearly with the hidden dimension of…
RubiConv -- Efficient Boundary-Respecting Convolutions
Linda Friso, Annie Marsden, Xinyi Chen +4
Convolutional architectures have emerged as powerful alternatives to Transformers for sequence modeling. The primary advantage is that they offer improved theoretical sequence leng…
Universal Sequence Preconditioning
Annie Marsden, Elad Hazan
We study the problem of preconditioning in sequential prediction. From the theoretical lens of linear dynamical systems, we show that convolving the target sequence corresponds to…
Provable Length Generalization in Sequence Prediction via Spectral Filtering
Annie Marsden, Evan Dogariu, Naman Agarwal +3
We consider the problem of length generalization in sequence prediction. We define a new metric of performance in this setting -- the Asymmetric-Regret -- which measures regret aga…