8 papers
Rates and architectures for learning geometrically non-trivial operators
T. Mitchell Roddenberry, Leo Tzou, Ivan Dokmanić +2
Deep learning methods have proven capable of recovering operators between high-dimensional spaces, such as solution maps of PDEs and similar objects in mathematical physics, from v…
Neon: Negative Extrapolation From Self-Training Improves Image Generation
Sina Alemohammad, Zhangyang Wang, Richard G. Baraniuk
Scaling generative AI models is bottlenecked by the scarcity of high-quality training data. The ease of synthesizing from a generative model suggests using (unverified) synthetic d…
GrokAlign: Geometric Characterisation and Acceleration of Grokking
Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero +1
A key challenge for the machine learning community is to understand and accelerate the training dynamics of deep networks that lead to delayed generalisation and emergent robustnes…
W4S4: WaLRUS Meets S4 for Long-Range Sequence Modeling
Hossein Babaei, Mel White, Richard G. Baraniuk
State Space Models (SSMs) have emerged as powerful components for sequence modeling, enabling efficient handling of long-range dependencies via linear recurrence and convolutional…
WaLRUS: Wavelets for Long-range Representation Using SSMs
Hossein Babaei, Mel White, Sina Alemohammad +1
State-Space Models (SSMs) have proven to be powerful tools for modeling long-range dependencies in sequential data. While the recent method known as HiPPO has demonstrated strong p…
SaFARi: State-Space Models for Frame-Agnostic Representation
Hossein Babaei, Mel White, Sina Alemohammad +1
State-Space Models (SSMs) have re-emerged as a powerful tool for online function approximation, and as the backbone of machine learning models for long-range dependent data. Howeve…