3.5k citations
Counting papers with at most 25 authors, so collaboration author lists do not stand in for the people here.
- Nvidia (United Kingdom)GB136 papers
- Lawrence Berkeley National LaboratoryUS32 papers
- Stanford UniversityUS30 papers
- University of California, BerkeleyUS29 papers
- California Institute of TechnologyUS26 papers
- Massachusetts Institute of TechnologyUS25 papers
- Carnegie Mellon UniversityUS24 papers
- University of Illinois Urbana-ChampaignUS24 papers
- University of TorontoCA24 papers
- University of WashingtonUS23 papers
- Harvard UniversityUS19 papers
- Georgia Institute of TechnologyUS17 papers
33 papers · 1 filter
NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL
Amos Goldman, Nimrod Boker, Maayan Sheraizin +15
Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, driving the development of specialized device-initiated communication libraries such…
GPU-Initiated Networking for NCCL
Khaled Hamidouche, John Bachan, Pak Markthub +6
Modern AI workloads, especially Mixture-of-Experts (MoE) architectures, increasingly demand low-latency, fine-grained GPU-to-GPU communication with device-side control. Traditional…
Guaranteed DGEMM Accuracy While Using Reduced Precision Tensor Cores Through Extensions of the Ozaki Scheme
Angelika Schwarz, Anton Anders, Cole Brower +8
The rapid growth of artificial intelligence (AI) has made low-precision formats such as FP16, FP8, and, most recently, block-scaled FP4 the primary focus of modern GPUs, where Tens…
Redesigning GROMACS Halo Exchange: Improving Strong Scaling with GPU-initiated NVSHMEM
Mahesh Doijade, Andrey Alekseenko, Ania Brown +2
Improving time-to-solution in molecular dynamics simulations often requires strong scaling due to fixed-sized problems. GROMACS is highly latency-sensitive, with peak iteration rat…
LAMMPS-KOKKOS: Performance Portable Molecular Dynamics Across Exascale Architectures
Anders Johansson, Evan Weinberg, Christian R. Trott +2
Since its inception in 1995, LAMMPS has grown to be a world-class molecular dynamics code, with thousands of users, over one million lines of code, and multi-scale simulation capab…
MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism
Zheng Zhang, Donglin Yang, Yaqi Xia +4
Recently, Mixture-of-Experts (MoE) has become one of the most popular techniques to scale pre-trained models to extraordinarily large sizes. Dynamic activation of experts allows fo…
Parallelizing the Approximate Minimum Degree Ordering Algorithm: Strategies and Evaluation
Yen-Hsiang Chang, Aydın Buluç, James Demmel
The approximate minimum degree algorithm is widely used before numerical factorization to reduce fill-in for sparse matrices. While considerable attention has been given to the num…
gpuPairHMM: High-speed Pair-HMM Forward Algorithm for DNA Variant Calling on GPUs
Bertil Schmidt, Felix Kallenborn, Alexander Wichmann +2
The continually increasing volume of DNA sequence data has resulted in a growing demand for fast implementations of core algorithms. Computation of pairwise alignments between cand…
Workflows Community Summit 2024: Future Trends and Challenges in Scientific Workflows
Rafael Ferreira da Silva, Deborah Bard, Kyle Chard +108
The Workflows Community Summit gathered 111 participants from 18 countries to discuss emerging trends and challenges in scientific workflows, focusing on six key areas: time-sensit…
Efficient N-to-M Checkpointing Algorithm for Finite Element Simulations
David A. Ham, Vaclav Hapla, Matthew G. Knepley +2
In this work, we introduce a new algorithm for N-to-M checkpointing in finite element simulations. This new algorithm allows efficient saving/loading of functions representing phys…
Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses
Jeongmin Brian Park, Vikram Sharma Mailthody, Zaid Qureshi +1
Graph Neural Networks (GNNs) are emerging as a powerful tool for learning from graph-structured data and performing sophisticated inference tasks in various application domains. Al…
Accelerating MPI Collectives with Process-in-Process-based Multi-object Techniques
Jiajun Huang, Kaiming Ouyang, Yujia Zhai +8
In the exascale computing era, optimizing MPI collective performance in high-performance computing (HPC) applications is critical. Current algorithms face performance degradation d…
Hector: An Efficient Programming and Compilation Framework for Implementing Relational Graph Neural Networks in GPU Architectures
Kun Wu, Mert Hidayetoğlu, Xiang Song +4
Relational graph neural networks (RGNNs) are graph neural networks with dedicated structures for modeling the different types of nodes and edges in heterogeneous graphs. While RGNN…
Parallelizing Maximal Clique Enumeration on GPUs
Mohammad Almasri, Yen-Hsiang Chang, Izzat El Hajj +3
We present a GPU solution for exact maximal clique enumeration (MCE) that performs a search tree traversal following the Bron-Kerbosch algorithm. Prior works on parallelizing MCE o…
Application Experiences on a GPU-Accelerated Arm-based HPC Testbed
Wael Elwasif, William Godoy, Nick Hagerty +28
This paper assesses and reports the experience of ten teams working to port,validate, and benchmark several High Performance Computing applications on a novel GPU-accelerated Arm t…
TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory
Hasan Al Maruf, Hao Wang, Abhishek Dhanotia +7
The increasing demand for memory in hyperscale applications has led to memory becoming a large portion of the overall datacenter spend. The emergence of coherent interfaces like CX…
GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture
Zaid Qureshi, Vikram Sharma Mailthody, Isaac Gelado +10
Graphics Processing Units (GPUs) have traditionally relied on the host CPU to initiate access to the data storage. This approach is well-suited for GPU applications with known data…
TEMPI: An Interposed MPI Library with a Canonical Representation of CUDA-aware Datatypes
Carl Pearson, Kun Wu, I-Hsin Chung +2
MPI derived datatypes are an abstraction that simplifies handling of non-contiguous data in MPI applications. These datatypes are recursively constructed at runtime from primitive…
mdspan in C++: A Case Study in the Integration of Performance Portable Features into International Language Standards
D. S. Hollman, Bryce Adelstein Lelbach, H. Carter Edwards +3
Multi-dimensional arrays are ubiquitous in high-performance computing (HPC), but their absence from the C++ language standard is a long-standing and well-known limitation of their…
Time-Based Roofline for Deep Learning Performance Analysis
Yunsong Wang, Charlene Yang, Steven Farrell +3
Deep learning applications are usually very compute-intensive and require a long run time for training and inference. This has been tackled by researchers from both hardware and so…
A Virtual Frame Buffer Abstraction for Parallel Rendering of Large Tiled Display Walls
Mengjiao Han, Ingo Wald, Will Usher +4
We present dw2, a flexible and easy-to-use software infrastructure for interactive rendering of large tiled display walls. Our library represents the tiled display wall as a single…
MLModelCI: An Automatic Cloud Platform for Efficient MLaaS
Huaizheng Zhang, Yuanming Li, Yizheng Huang +3
MLModelCI provides multimedia researchers and developers with a one-stop platform for efficient machine learning (ML) services. The system leverages DevOps techniques to optimize,…
EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal In GPUs
Seung Won Min, Vikram Sharma Mailthody, Zaid Qureshi +3
Modern analytics and recommendation systems are increasingly based on graph data that capture the relations between entities being analyzed. Practical graphs come in huge sizes, of…
Heterogeneous CPU/GPU co-execution of CFD simulations on the POWER9 architecture: Application to airplane aerodynamics
R. Borrell, D. Dosimont, M. Garcia-Gasulla +6
High fidelity Computational Fluid Dynamics simulations are generally associated with large computing requirements, which are progressively acute with each new generation of superco…