341 citations · 1.1k across the 43 of their papers we have counts for
9 papers · 1 filter
VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
Roberto L. Castro, Andrei Ivanov, Diego Andrade +3
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compu…
ComPile: A Large IR Dataset from Production Sources
Aiden Grossman, Ludger Paehler, Konstantinos Parasyris +6
Code is increasingly becoming a core data modality of modern machine learning research impacting not only the way we write code with conversational agents like OpenAI's ChatGPT, Go…
Cached Operator Reordering: A Unified View for Fast GNN Training
Julia Bazinska, Andrei Ivanov, Tal Ben-Nun +4
Graph Neural Networks (GNNs) are a powerful tool for handling structured graph data and addressing tasks such as node classification, graph classification, and clustering. However,…
Maximum Flows in Parametric Graph Templates
Tal Ben-Nun, Lukas Gianinazzi, Torsten Hoefler +1
Execution graphs of parallel loop programs exhibit a nested, repeating structure. We show how such graphs that are the result of nested repetition can be represented by succinct pa…
FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs
Philipp Schaad, Timo Schneider, Tal Ben-Nun +3
The current hardware landscape and application scale is driving performance engineers towards writing bespoke optimizations. Verifying such optimizations, and generating minimal fa…
Bridging Control-Centric and Data-Centric Optimization
Tal Ben-Nun, Berke Ates, Alexandru Calotoiu +1
With the rise of specialized hardware and new programming languages, code optimization has shifted its focus towards promoting data locality. Most production-grade compilers adopt…
STen: Productive and Efficient Sparsity in PyTorch
Andrei Ivanov, Nikoli Dryden, Tal Ben-Nun +2
As deep learning models grow, sparsity is becoming an increasingly critical component of deep neural networks, enabling improved performance and reduced storage. However, existing…
Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization
Lukas Trümper, Tal Ben-Nun, Philipp Schaad +2
Performance optimization is an increasingly challenging but often repetitive task. While each platform has its quirks, the underlying code transformations rely on data movement and…
A Theory of I/O-Efficient Sparse Neural Network Inference
Niels Gleinig, Tal Ben-Nun, Torsten Hoefler
As the accuracy of machine learning models increases at a fast rate, so does their demand for energy and compute resources. On a low level, the major part of these resources is con…