60 citations · 179 across the 33 of their papers we have counts for
27 papers · 1 filter
Multi-GPU Communication Schemes for Iterative Solvers: When CPUs are Not in Charge
Ismayil Ismayilov, Javid Baydamirli, Doğan Sağbili +2
This paper proposes a fully autonomous execution model for multi-GPU applications that completely excludes the involvement of the CPU beyond the initial kernel launch. In a typical…
An Efficient Out-of-Core Tomographic Imaging Framework for Edge Devices
Xuetao Chen, Cong Ma, Xiangyu Meng +10
Computed Tomography (CT) is an essential 3D imaging technology widely used in medical diagnostics and scientific research. However, performing CT imaging on edge devices is challen…
Scalable High-Fidelity Macromolecular Docking for GPU-Accelerated Supercomputers
Xiangyu Meng, Peng Chen, Mingzhen Li +7
Flexible macromolecular docking offers high-fidelity predictions of biomolecular interactions, but remains prohibitively expensive at scale. Among existing approaches, LightDock le…
SHIRO: Near-Optimal Communication Strategies for Distributed Sparse Matrix Multiplication
Chen Zhuang, Lingqi Zhang, Benjamin Brock +5
Distributed Sparse Matrix-Matrix Multiplication (SpMM) is a fundamental operation in high-performance computing and deep learning applications. The major performance bottleneck in…
RAPTOR: Practical Numerical Profiling of Scientific Applications
Faveo Hoerold, Ivan R. Ivanov, Akash Dhruv +4
The proliferation of low-precision units in modern high-performance architectures increasingly burdens domain scientists. Historically, the choice in HPC was easy: can we get away…
Paradigm Shift in Infrastructure Inspection Technology: Leveraging High-performance Imaging and Advanced AI Analytics to Inspect Road Infrastructure
Du Wu, Enzhi Zhang, Isaac Lyngaas +14
Effective road infrastructure management is crucial for modern society. Traditional manual inspection techniques remain constrained by cost, efficiency, and scalability, while came…
A Unifying Framework to Enable Artificial Intelligence in High Performance Computing Workflows
Jens Domke, Mohamed Wahib, Anshu Dubey +2
Current trends point to a future where large-scale scientific applications are tightly-coupled HPC/AI hybrids. Hence, we urgently need to invest in creating a seamless, scalable fr…
Balanced and Elastic End-to-end Training of Dynamic LLMs
Mohamed Wahib, Muhammed Abdullah Soyturk, Didem Unat
To reduce the computational and memory overhead of Large Language Models, various approaches have been proposed. These include a) Mixture of Experts (MoEs), where token routing aff…
Can Tensor Cores Benefit Memory-Bound Kernels? (No!)
Lingqi Zhang, Jiajun Huang, Sheng Di +2
Tensor cores are specialized processing units within GPUs that have demonstrated significant efficiency gains in compute-bound applications such as Deep Learning Training by accele…
Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
Chen Zhuang, Lingqi Zhang, Du Wu +8
Graph Convolutional Networks (GCNs), particularly for large-scale graphs, are crucial across numerous domains. However, training distributed full-batch GCNs on large-scale graphs s…
Ultra-Long Sequence Distributed Transformer
Xiao Wang, Isaac Lyngaas, Aristeidis Tsaris +7
Transformer models trained on long sequences often achieve higher accuracy than short sequences. Unfortunately, conventional transformers struggle with long sequence training due t…
KAKURENBO: Adaptively Hiding Samples in Deep Neural Network Training
Truong Thao Nguyen, Balazs Gerofi, Edgar Josafat Martinez-Noriega +2
This paper proposes a method for hiding the least-important samples during the training of deep neural networks to increase efficiency, i.e., to reduce the cost of training. Using…
Exploiting Scratchpad Memory for Deep Temporal Blocking: A case study for 2D Jacobian 5-point iterative stencil kernel (j2d5pt)
Lingqi Zhang, Mohamed Wahib, Peng Chen +4
General Purpose Graphics Processing Units (GPGPU) are used in most of the top systems in HPC. The total capacity of scratchpad memory has increased by more than 40 times in the las…
Revisiting Temporal Blocking Stencil Optimizations
Lingqi Zhang, Mohamed Wahib, Peng Chen +4
Iterative stencils are used widely across the spectrum of High Performance Computing (HPC) applications. Many efforts have been put into optimizing stencil GPU kernels, given the p…
Myths and Legends in High-Performance Computing
Satoshi Matsuoka, Jens Domke, Mohamed Wahib +2
In this thought-provoking article, we discuss certain myths and legends that are folklore among members of the high-performance computing community. We gathered these myths from co…
Preparing for the Future -- Rethinking Proxy Apps
Satoshi Matsuoka, Jens Domke, Mohamed Wahib +5
A considerable amount of research and engineering went into designing proxy applications, which represent common high-performance computing workloads, to co-design and evaluate the…
At the Locus of Performance: Quantifying the Effects of Copious 3D-Stacked Cache on HPC Workloads
Jens Domke, Emil Vatai, Balazs Gerofi +9
Over the last three decades, innovations in the memory subsystem were primarily targeted at overcoming the data movement bottleneck. In this paper, we focus on a specific market tr…
PERKS: a Locality-Optimized Execution Model for Iterative Memory-bound GPU Applications
Lingqi Zhang, Mohamed Wahib, Peng Chen +4
Iterative memory-bound solvers commonly occur in HPC codes. Typical GPU implementations have a loop on the host side that invokes the GPU kernel as much as time/algorithm steps the…
Performance Portable Back-projection Algorithms on CPUs: Agnostic Data Locality and Vectorization Optimizations
Peng Chen, Mohamed Wahib, Xiao Wang +4
Computed Tomography (CT) is a key 3D imaging technology that fundamentally relies on the compute-intense back-projection operation to generate 3D volumes. GPUs are typically used f…
An Oracle for Guiding Large-Scale Model/Hybrid Parallel Training of Convolutional Neural Networks
Albert Njoroge Kahira, Truong Thao Nguyen, Leonardo Bautista Gomez +3
Deep Neural Network (DNN) frameworks use distributed training to enable faster time to convergence and alleviate memory capacity limitations when training large models and/or using…
Matrix Engines for High Performance Computing:A Paragon of Performance or Grasping at Straws?
Jens Domke, Emil Vatai, Aleksandr Drozd +8
Matrix engines or units, in different forms and affinities, are becoming a reality in modern processors; CPUs and otherwise. The current and dominant algorithmic approach to Deep L…
Scaling Distributed Deep Learning Workloads beyond the Memory Capacity with KARMA
Mohamed Wahib, Haoyu Zhang, Truong Thao Nguyen +5
The dedicated memory of hardware accelerators can be insufficient to store all weights and/or intermediate states of large deep learning models. Although model parallelism is a via…
A Study of Single and Multi-device Synchronization Methods in Nvidia GPUs
Lingqi Zhang, Mohamed Wahib, Haoyu Zhang +1
GPUs are playing an increasingly important role in general-purpose computing. Many algorithms require synchronizations at different levels of granularity in a single GPU. Additiona…
AN5D: Automated Stencil Framework for High-Degree Temporal Blocking on GPUs
Kazuaki Matsumura, Hamid Reza Zohouri, Mohamed Wahib +2
Stencil computation is one of the most widely-used compute patterns in high performance computing applications. Spatial and temporal blocking have been proposed to overcome the mem…