Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices

Andres Potapczynski, Shikai Qiu, Marc Finzi +6

Dense linear layers are the dominant computational bottleneck in large neural networks, presenting a critical need for more efficient alternatives. Previous efforts focused on a sm…