10 citations · 28 across the 19 of their papers we have counts for
10 papers · 1 filter
RATrain: A Resource-Aware Training Runtime for Large Language Models on Bandwidth-Constrained Heterogeneous Supercomputing Platforms
Yao Lu, Shiqing Ma, Zhongzhi Luan +5
Production heterogeneous supercomputing platforms are increasingly used to host large language model (LLM) training workloads. However, existing GPU-oriented training runtimes typi…
PRAGMA: A Profiling-Reasoned Multi-Agent Framework for Automatic Kernel Optimization
Kelun Lei, Hailong Yang, Huaitao Zhang +5
Designing high-performance kernels requires expert-level tuning and a deep understanding of hardware characteristics. Recent advances in large language models (LLMs) have enabled a…
LOw-cOst yet High-Performant Sparse Matrix-Matrix Multiplication on Arm SME Architectures
Kelun Lei, Hailong Yang, Kaige Zhang +8
Sparse matrix-dense matrix multiplication (SpMM) is a critical kernel in both scientific computing and emerging graph learning workloads. The recent Armv9 architecture introduces S…
Past-Future Scheduler for LLM Serving under SLA Guarantees
Ruihao Gong, Shihao Bai, Siyu Wu +5
The exploration and application of Large Language Models (LLMs) is thriving. To reduce deployment costs, continuous batching has become an essential feature in current service fram…
FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning
Yao Lu, Jiaxing QI, Zhongzhi Luan +4
Large language models (LLMs) have emerged as important components across various fields, yet their training requires substantial computation resources and abundant labeled data. It…
INSPIRIT: Optimizing Heterogeneous Task Scheduling through Adaptive Priority in Task-based Runtime Systems
Yiqing Wang, Xiaoyan Liu, Hailong Yang +5
As modern HPC computing platforms become increasingly heterogeneous, it is challenging for programmers to fully leverage the computation power of massive parallelism offered by suc…