9 citations · 18 across the 7 of their papers we have counts for
6 papers · 1 filter
RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
Zhan Zhao, Yuxin Wang, Amelie Chi Zhou
Large Language Models (LLMs) are transforming recommendation from ranking into a generative task, but industrial deployment remains limited by the high latency of processing long,…
When KV Meets Embeddings: Dynamic GPU Memory Allocation for Accelerating Generative Recommender Serving
Wenjun Yu, Shuguang Han, Amelie Chi Zhou
Generative Recommender (GR) inference places embedding hot caches (EMB) and KV caches in direct competition for limited GPU HBM: allocating more memory to one improves its efficien…
DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization
Zhenheng Tang, Zichen Tang, Junlin Huang +7
The growth of large language models (LLMs) increases challenges of accelerating distributed training across multiple GPUs in different data centers. Moreover, concerns about data p…
FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression
Zhenheng Tang, Xueze Kang, Yiming Yin +11
To alleviate hardware scarcity in training large deep neural networks (DNNs), particularly large language models (LLMs), we present FusionLLM, a decentralized training system desig…
Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning
Zichen Tang, Junlin Huang, Rudan Yan +5
Current data compression methods, such as sparsification in Federated Averaging (FedAvg), effectively enhance the communication efficiency of Federated Learning (FL). However, thes…
BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems
Yuxin Wang, Yuhan Chen, Zeyu Li +11
Serving systems for Large Language Models (LLMs) are often optimized to improve quality of service (QoS) and throughput. However, due to the lack of open-source LLM serving workloa…