Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers

Ilya Koziev, Ivan Oseledets

Fast matrix multiplication algorithms keep the product fixed and search for a cheaper way to evaluate it. We instead ask whether a Transformer's learned projections can use a diffe…