Publications (5)
Identifying Subgroup and Context Effects in Conjoint Experiments
Steven Wang, Isys Johnson, Jessica Grogan +4
Conjoint experiments have become central to survey research in political science and related fields because they allow researchers to study preferences across multiple attributes s…
Zoology: Measuring and Improving Recall in Efficient Language Models
Simran Arora, Sabri Eyuboglu, Aman Timalsina +5
Attention-free language models that combine gating and convolutions are growing in popularity due to their efficiency and increasingly competitive performance. To better understand…
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
Albert Gu, Isys Johnson, Karan Goel +4
Recurrent neural networks (RNNs), temporal convolutions, and neural differential equations (NDEs) are popular families of deep learning models for time-series data, each with uniqu…
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Daniel Y. Fu, Simran Arora, Jessica Grogan +7
Machine learning models are increasingly being scaled in both sequence length and model dimension to reach longer contexts and better performance. However, existing architectures s…
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections
Albert Gu, Isys Johnson, Aman Timalsina +2
Linear time-invariant state space models (SSM) are a classical model from engineering and statistics, that have recently been shown to be very promising in machine learning through…