483 citations · 3k across the 177 of their papers we have counts for
26 papers · 1 filter
A Data-Centric Approach to Extreme-Scale Ab initio Dissipative Quantum Transport Simulations
Alexandros Nikolaos Ziogas, Tal Ben-Nun, Guillermo Indalecio Fernández +3
The computational efficiency of a state of the art ab initio quantum transport (QT) solver, capable of revealing the coupled electro-thermal properties of atomically-resolved nano-…
Optimizing the Data Movement in Quantum Transport Simulations via Data-Centric Parallel Programming
Alexandros Nikolaos Ziogas, Tal Ben-Nun, Guillermo Indalecio Fernández +3
Designing efficient cooling systems for integrated circuits (ICs) relies on a deep understanding of the electro-thermal properties of transistors. To shed light on this issue in cu…
Predicting Weather Uncertainty with Deep Convnets
Peter Grönquist, Tal Ben-Nun, Nikoli Dryden +4
Modern weather forecast models perform uncertainty quantification using ensemble prediction systems, which collect nonparametric statistics based on multiple perturbed simulations.…
Practice of Streaming Processing of Dynamic Graphs: Concepts, Models, and Systems
Maciej Besta, Marc Fischer, Vasiliki Kalavri +2
Graph processing has become an important part of various areas of computing, including machine learning, medical applications, social network analysis, computational sciences, and…
Slim Fly: A Cost Effective Low-Diameter Network Topology
Maciej Besta, Torsten Hoefler
We introduce a high-performance cost-effective network topology called Slim Fly that approaches the theoretically optimal network diameter. Slim Fly is based on graphs that approxi…
Slim Graph: Practical Lossy Graph Compression for Approximate Graph Processing, Storage, and Analytics
Maciej Besta, Simon Weber, Lukas Gianinazzi +4
We propose Slim Graph: the first programming model and framework for practical lossy graph compression that facilitates high-performance approximate graph processing, storage, and…
Flexible Communication Avoiding Matrix Multiplication on FPGA with High-Level Synthesis
Johannes de Fine Licht, Grzegorz Kwasniewski, Torsten Hoefler
Data movement is the dominating factor affecting performance and energy in modern computing systems. Consequently, many algorithms have been developed to minimize the number of I/O…
A scalable weakly-synchronous algorithm for solving partial differential equations
Konduri Aditya, Tobias Gysi, Grzegorz Kwasniewski +3
Synchronization overheads pose a major challenge as applications advance towards extreme scales. In current large-scale algorithms, synchronization as well as data communication de…
Stream Semantic Registers: A Lightweight RISC-V ISA Extension Achieving Full Compute Utilization in Single-Issue Cores
Fabian Schuiki, Florian Zaruba, Torsten Hoefler +1
Single-issue processor cores are very energy efficient but suffer from the von Neumann bottleneck, in that they must explicitly fetch and issue the loads/storse necessary to feed t…
Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisons
Maciej Besta, Raghavendra Kanakagiri, Harun Mustafa +4
The Jaccard similarity index is an important measure of the overlap of two sets, widely used in machine learning, computational genomics, information retrieval, and many other area…
hlslib: Software Engineering for Hardware Design
Johannes de Fine Licht, Torsten Hoefler
High-level synthesis (HLS) tools have brought FPGA development into the mainstream, by allowing programmers to design architectures using familiar languages such as C, C++, and Ope…
Active Access: A Mechanism for High-Performance Distributed Data-Centric Computations
Maciej Besta, Torsten Hoefler
Remote memory access (RMA) is an emerging high-performance programming model that uses RDMA hardware directly. Yet, accessing remote memories cannot invoke activities at the target…
Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries
Maciej Besta, Robert Gerstenberger, Emanuel Peter +5
Graph processing has become an important part of multiple areas of computer science, such as machine learning, computational sciences, medical applications, social network analysis…
Mitigating Network Noise on Dragonfly Networks through Application-Aware Routing
Daniele De Sensi, Salvatore Di Girolamo, Torsten Hoefler
System noise can negatively impact the performance of HPC systems, and the interconnection network is one of the main factors contributing to this problem. To mitigate this effect,…
Streaming Message Interface: High-Performance Distributed Memory Programming on Reconfigurable Hardware
Tiziano De Matteis, Johannes de Fine Licht, Jakub Beránek +1
Distributed memory programming is the established paradigm used in high-performance computing (HPC) systems, requiring explicit communication between nodes and devices. When FPGAs…
Network-Accelerated Non-Contiguous Memory Transfers
Salvatore Di Girolamo, Konstantin Taranov, Andreas Kurth +7
Applications often communicate data that is non-contiguous in the send- or the receive-buffer, e.g., when exchanging a column of a matrix stored in row-major order. While non-conti…
Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency
Elad Hoffer, Berry Weinstein, Itay Hubara +3
Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model. Although trained on images of aspecific size, it is well…
Red-blue pebbling revisited: near optimal parallel matrix-matrix multiplication
Grzegorz Kwasniewski, Marko Kabić, Maciej Besta +3
We propose COSMA: a parallel matrix-matrix multiplication algorithm that is near communication-optimal for all combinations of matrix dimensions, processor counts, and memory sizes…
Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations
Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo +2
Load imbalance pervasively exists in distributed deep learning training systems, either caused by the inherent imbalance in learned tasks or by the system itself. Traditional synch…
FBLAS: Streaming Linear Algebra on FPGA
Tiziano De Matteis, Johannes de Fine Licht, Torsten Hoefler
Spatial computing architectures pose an attractive alternative to mitigate control and data movement overheads typical of load-store architectures. In practice, these devices are r…
FatPaths: Routing in Supercomputers and Data Centers when Shortest Paths Fall Short
Maciej Besta, Marcel Schneider, Karolina Cynk +5
We introduce FatPaths: a simple, generic, and robust routing architecture that enables state-of-the-art low-diameter topologies such as Slim Fly to achieve unprecedented performanc…
Graph Processing on FPGAs: Taxonomy, Survey, Challenges
Maciej Besta, Dimitri Stanojevic, Johannes De Fine Licht +2
Graph processing has become an important part of various areas, such as machine learning, computational sciences, medical applications, social network analysis, and many others. Va…
Stateful Dataflow Multigraphs: A Data-Centric Model for Performance Portability on Heterogeneous Architectures
Tal Ben-Nun, Johannes de Fine Licht, Alexandros Nikolaos Ziogas +2
The ubiquity of accelerators in high-performance computing has driven programming complexity beyond the skill-set of the average domain scientist. To maintain performance portabili…
Augment your batch: better training with larger batches
Elad Hoffer, Tal Ben-Nun, Itay Hubara +3
Large-batch SGD is important for scaling training of deep neural networks. However, without fine-tuning hyperparameter schedules, the generalization of the model may be hampered. W…