157 citations · 476 across the 27 of their papers we have counts for
3 papers · 1 filter
STAR: Synthesis of Tailored Architectures
Armin W. Thomas, Rom Parnichkun, Alexander Amini +2
Iterative improvement of model architectures is fundamental to deep learning: Transformers first enabled scaling, and recent advances in model hybridization have pushed the quality…
State-Free Inference of State-Space Models: The Transfer Function Approach
Rom N. Parnichkun, Stefano Massaroli, Alessandro Moro +10
We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel in…
Mechanistic Design and Scaling of Hybrid Architectures
Michael Poli, Armin W Thomas, Eric Nguyen +9
The development of deep learning architectures is a resource-demanding process, due to a vast design space, long prototyping times, and high compute costs associated with at-scale…