157 citations · 508 across the 24 of their papers we have counts for
25 papers
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Stefano Massaroli, Michael Poli, Daniel Y. Fu +11
Recent advances in attention-free sequence models rely on convolutions as alternatives to the attention operator at the core of Transformers. In particular, long convolution sequen…
Learning Efficient Surrogate Dynamic Models with Graph Spline Networks
Chuanbo Hua, Federico Berto, Michael Poli +2
While complex simulations of physical systems have been widely used in engineering and scientific computing, lowering their often prohibitive computational requirements has only re…
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Daniel Y. Fu, Simran Arora, Jessica Grogan +7
Machine learning models are increasingly being scaled in both sequence length and model dimension to reach longer contexts and better performance. However, existing architectures s…
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Eric Nguyen, Michael Poli, Marjan Faizi +10
Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation…
Ideal Abstractions for Decision-Focused Learning
Michael Poli, Stefano Massaroli, Stefano Ermon +2
We present a methodology for formulating simplifying abstractions in machine learning systems by identifying and harnessing the utility structure of decisions. Machine learning tas…
Effectively Modeling Time Series with Simple Discrete State Spaces
Michael Zhang, Khaled K. Saab, Michael Poli +3
Time series modeling is a well-established problem, which often requires that methods (1) expressively represent complicated dependencies, (2) forecast long horizons, and (3) effic…