483 citations · 3k across the 177 of their papers we have counts for
17 papers · 1 filter
Lifting C Semantics for Dataflow Optimization
Alexandru Calotoiu, Tal Ben-Nun, Grzegorz Kwasniewski +4
C is the lingua franca of programming and almost any device can be programmed using C. However, programming mod-ern heterogeneous architectures such as multi-core CPUs and GPUs req…
A Data-Centric Optimization Framework for Machine Learning
Oliver Rausch, Tal Ben-Nun, Nikoli Dryden +3
Rapid progress in deep learning is leading to a diverse set of quickly changing models, with a dramatically growing demand for compute. However, as frameworks specialize performanc…
Parallel Algorithms for Finding Large Cliques in Sparse Graphs
Lukas Gianinazzi, Maciej Besta, Yannick Schaffner +1
We present a parallel k-clique listing algorithm with improved work bounds (for the same depth) in sparse graphs with low degeneracy or arboricity. We achieve this by introducing a…
On the Parallel I/O Optimality of Linear Algebra Kernels: Near-Optimal Matrix Factorizations
Grzegorz Kwasniewski, Marko Kabić, Tal Ben-Nun +8
Matrix factorizations are among the most important building blocks of scientific computing. State-of-the-art libraries, however, are not communication-optimal, underutilizing curre…
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
Shigang Li, Torsten Hoefler
Training large deep learning models at scale is very challenging. This paper proposes Chimera, a novel pipeline parallelism scheme which combines bidirectional pipelines for effici…
Flare: Flexible In-Network Allreduce
Daniele De Sensi, Salvatore Di Girolamo, Saleh Ashkboos +2
The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation…
rFaaS: Enabling High Performance Serverless with RDMA and Leases
Marcin Copik, Konstantin Taranov, Alexandru Calotoiu +1
High performance is needed in many computing systems, from batch-managed supercomputers to general-purpose cloud platforms. However, scientific clusters lack elastic parallelism, w…
Learning Combinatorial Node Labeling Algorithms
Lukas Gianinazzi, Maximilian Fries, Nikoli Dryden +3
We present a novel neural architecture to solve graph optimization problems where the solution consists of arbitrary node labels, allowing us to solve hard problems like graph colo…
Motif Prediction with Graph Neural Networks
Maciej Besta, Raphael Grob, Cesare Miglioli +8
Link prediction is one of the central problems in graph mining. However, recent studies highlight the importance of higher-order network analysis, where complex structures called m…
Pebbles, Graphs, and a Pinch of Combinatorics: Towards Tight I/O Lower Bounds for Statically Analyzable Programs
Grzegorz Kwasniewski, Tal Ben-Nun, Lukas Gianinazzi +5
Determining I/O lower bounds is a crucial step in obtaining communication-efficient parallel algorithms, both across the memory hierarchy and between processors. Current approaches…
Distributed Quantum Computing with QMPI
Thomas Häner, Damian S. Steiger, Torsten Hoefler +1
Practical applications of quantum computers require millions of physical qubits and it will be challenging for individual quantum processors to reach such qubit numbers. It is ther…
EvalNet: A Practical Toolchain for Generation and Analysis of Extreme-Scale Interconnects
Maciej Besta, Patrick Iff, Marcel Schneider +10
The diversity of communication paths in a network, especially non-minimal paths, is a key enabler of performance at extreme scales. We present EvalNet, a toolchain for scalable gen…
SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems
Maciej Besta, Raghavendra Kanakagiri, Grzegorz Kwasniewski +15
Simple graph algorithms such as PageRank have been the target of numerous hardware accelerators. Yet, there also exist much more complex graph mining algorithms for problems such a…
GraphMineSuite: Enabling High-Performance and Programmable Graph Mining Algorithms with Set Algebra
Maciej Besta, Zur Vonarburg-Shmaria, Yannick Schaffner +16
We propose GraphMineSuite (GMS): the first benchmarking suite for graph mining that facilitates evaluating and constructing high-performance graph mining algorithms. First, GMS com…
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun +2
The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components. Similarly to their biol…
Enabling Dataflow Optimization for Quantum Programs
David Ittah, Thomas Häner, Vadym Kliuchnikov +1
We propose an IR for quantum computing that directly exposes quantum and classical data dependencies for the purpose of optimization. The Quantum Intermediate Representation for Op…
Clairvoyant Prefetching for Distributed Machine Learning I/O
Nikoli Dryden, Roman Böhringer, Tal Ben-Nun +1
I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments. Indeed, at large scale, I/O takes as much as 85% of training time. Addr…