17 citations · 19 across the 17 of their papers we have counts for
4 papers · 1 filter
Fine-grained MoE Load Balancing with Linear Programming
Chenqi Zhao, Wenfei Wu, Linhai Song +2
Mixture-of-Experts (MoE) has emerged as a promising approach to scale up deep learning models due to its significant reduction in computational resources. However, the dynamic natu…
XBOF: A Cost-Efficient CXL JBOF with Inter-SSD Compute Resource Sharing
Shushu Yi, Yuda An, Li Peng +11
Enterprise SSDs integrate numerous computing resources (e.g., ARM processor and onboard DRAM) to satisfy the ever-increasing performance requirements of I/O bursts. While these res…
Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
Kexin Chu, Zecheng Lin, Dawei Xiang +7
Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversari…
AllReduce Scheduling with Hierarchical Deep Reinforcement Learning
Yufan Wei, Mickel Liu, Wenfei Wu
AllReduce is a technique in distributed computing which saw use in many critical applications of deep learning. Existing methods of AllReduce scheduling oftentimes lack flexibility…