3 papers
cs.OS2026
Nixie: Efficient, Transparent Temporal Multiplexing for Consumer GPUs
Yechen Xu, Yifei Wang, Nathanael Ren +2
Consumer machines are increasingly running large ML workloads such as large language models (LLMs), text-to-image generation, and interactive image editing. Unlike datacenter GPUs,…
cs.LG2025
FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
Zishan Shao, Yixiao Wang, Qinsi Wang +6
Singular Value Decomposition (SVD) has recently seen a surge of interest as a simple yet powerful tool for large language models (LLMs) compression, with a growing number of works…
cs.LG2025
MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing
Haiyue Ma, Zhixu Du, Yiran Chen
In multi-GPU Mixture-of-Experts (MoE) network, experts are distributed across different GPUs, which creates load imbalance as each expert processes different number of tokens. Rece…