4 papers
Prefix Sums via Kronecker Products
Aleksandros Sobczyk, Anastasios Zouzias
In this work, we revisit prefix sums through the lens of linear algebra. We describe an identity that decomposes triangular all-ones matrices as a sum of two Kronecker products, an…
Segmented Operations using Matrix Multiplications
Aleksandros Sobczyk, Giuseppe Sorrentino, Anastasios Zouzias
Specialized computational units that perform small matrix multiplications as primitive operations are typically present in modern AI accelerators. However, these Matrix Multiplicat…
The Impact of Partial Computations on the Red-Blue Pebble Game
Pál András Papp, Aleksandros Sobczyk, A. N. Yzelman
We study an extension of the well-known red-blue pebble game (RBP) with partial computation steps, inspired by the recent work of Sobczyk. While the original RBP assumes that we ne…
Quantum Doubly Stochastic Transformers
Jannis Born, Filip Skogh, Kahn Rhrissorrakrai +3
At the core of the Transformer, the softmax normalizes the attention matrix to be right stochastic. Previous research has shown that this often de-stabilizes training and that enfo…