341 citations · 465 across the 18 of their papers we have counts for
9 papers · 1 filter
Arrow Matrix Decomposition: A Novel Approach for Communication-Efficient Sparse Matrix Multiplication
Lukas Gianinazzi, Alexandros Nikolaos Ziogas, Langwen Huang +9
We propose a novel approach to iterated sparse matrix dense matrix multiplication, a fundamental computational kernel in scientific computing and graph neural network training. In…
VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
Roberto L. Castro, Andrei Ivanov, Diego Andrade +3
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compu…
Temporal Vectorization: A Compiler Approach to Automatic Multi-Pumping
Carl-Johannes Johnsen, Tiziano De Matteis, Tal Ben-Nun +2
The multi-pumping resource sharing technique can overcome the limitations commonly found in single-clocked FPGA designs by allowing hardware components to operate at a higher clock…
Clairvoyant Prefetching for Distributed Machine Learning I/O
Nikoli Dryden, Roman Böhringer, Tal Ben-Nun +1
I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addr…
Substream-Centric Maximum Matchings on FPGA
Maciej Besta, Marc Fischer, Tal Ben-Nun +3
Developing high-performance and energy-efficient algorithms for maximum matchings is becoming increasingly important in social network analysis, computational sciences, scheduling,…
On the Parallel I/O Optimality of Linear Algebra Kernels: Near-Optimal LU Factorization
Grzegorz Kwasniewski, Tal Ben-Nun, Alexandros Nikolaos Ziogas +3
Dense linear algebra kernels, such as linear solvers or tensor contractions, are fundamental components of many scientific computing applications. In this work, we present a novel…