75 citations · 81 across the 4 of their papers we have counts for
4 papers
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
Stefano Massaroli, Michael Poli, Daniel Y. Fu +11
Recent advances in attention-free sequence models rely on convolutions as alternatives to the attention operator at the core of Transformers. In particular, long convolution sequen…
Learning Efficient Surrogate Dynamic Models with Graph Spline Networks
Chuanbo Hua, Federico Berto, Michael Poli +2
While complex simulations of physical systems have been widely used in engineering and scientific computing, lowering their often prohibitive computational requirements has only re…
Ideal Abstractions for Decision-Focused Learning
Michael Poli, Stefano Massaroli, Stefano Ermon +2
We present a methodology for formulating simplifying abstractions in machine learning systems by identifying and harnessing the utility structure of decisions. Machine learning tas…
Hyena Hierarchy: Towards Larger Convolutional Language Models
Michael Poli, Stefano Massaroli, Eric Nguyen +6
Recent advances in deep learning have relied heavily on the use of large Transformers due to their ability to learn at scale. However, the core building block of Transformers, the…