ACDC: A Structured Efficient Linear Layer

Marcin Moczulski, Misha Denil, Jeremy Appleyard +1

The linear layer is one of the most pervasive modules in deep learning representations. However, it requires $O(N^2)$ parameters and $O(N^2)$ operations. These costs can be prohibi…