157 citations · 476 across the 27 of their papers we have counts for
6 papers · 1 filter
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Stefano Massaroli, Michael Poli, Daniel Y. Fu +11
Recent advances in attention-free sequence models rely on convolutions as alternatives to the attention operator at the core of Transformers. In particular, long convolution sequen…
Learning Efficient Surrogate Dynamic Models with Graph Spline Networks
Chuanbo Hua, Federico Berto, Michael Poli +2
While complex simulations of physical systems have been widely used in engineering and scientific computing, lowering their often prohibitive computational requirements has only re…
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Eric Nguyen, Michael Poli, Marjan Faizi +10
Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation…
Improving day-ahead Solar Irradiance Time Series Forecasting by Leveraging Spatio-Temporal Context
Oussama Boussif, Ghait Boukachab, Dan Assouline +4
Solar power harbors immense potential in mitigating climate change by substantially reducing CO emissions. Nonetheless, the inherent variability of solar irradiance poses a s…
Ideal Abstractions for Decision-Focused Learning
Michael Poli, Stefano Massaroli, Stefano Ermon +2
We present a methodology for formulating simplifying abstractions in machine learning systems by identifying and harnessing the utility structure of decisions. Machine learning tas…
Hyena Hierarchy: Towards Larger Convolutional Language Models
Michael Poli, Stefano Massaroli, Eric Nguyen +6
Recent advances in deep learning have relied heavily on the use of large Transformers due to their ability to learn at scale. However, the core building block of Transformers, the…