8 papers
T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator
Keyvan Dadashzadeh, Yuehong Zhou, Minyu Cui +1
Large transformer-based models increasingly depend on multi-GPU execution, which requires frequent collective communication among GPUs. Existing communication libraries often rely…
Analytical Resource Management for Fine-grained MoE Computation-Communication Overlap
Hongyu Liu, Minyu Cui, Miquel Pericas
Fine-grained computation--communication overlap in distributed Mixture-of-Experts (MoE) inference allows communication to begin as partial compute results become ready. However, co…
FlashAttention for Scalable Vector Architectures
Sonia Rani Gupta, Nikela Papadopoulou, Miquel Pericàs
Inference with transformer models on CPUs is increasingly important, especially for Small Language Models (SLMs), where vector architectures are emerging as a promising execution s…
DEFT: Joint Task Placement and DVFS for Energy-Efficient Multi-GPU Runtimes
Jing Chen, Miquel Pericas
Energy efficiency has become a first-order concern in modern high-performance computing systems, as it directly determines achievable throughput under fixed power budgets. Although…
Resource-aware Computation-Communication Overlap for multi-GPU ML Workloads
Minyu Cui, Miquel Pericas
The rapid growth of large-scale machine learning (ML) has made distributed training across multiple GPUs a fundamental component of modern ML systems. As model sizes and computatio…
Challenges and Opportunities in the Co-design of Convolutions and RISC-V Vector Processors
Sonia Rani Gupta, Nikela Papadopoulou, Miquel Pericàs
The RISC-V "V" extension introduces vector processing to the RISC-V architecture. Unlike most SIMD extensions, it supports long vectors which can result in significant improvement…
Analysis and Characterization of Performance Variability for OpenMP Runtime
Minyu Cui, Nikela Papadopoulou, Miquel Pericàs
In the high performance computing (HPC) domain, performance variability is a major scalability issue for parallel computing applications with heavy synchronization and communicatio…
At the Locus of Performance: Quantifying the Effects of Copious 3D-Stacked Cache on HPC Workloads
Jens Domke, Emil Vatai, Balazs Gerofi +9
Over the last three decades, innovations in the memory subsystem were primarily targeted at overcoming the data movement bottleneck. In this paper, we focus on a specific market tr…