4 papers
Nixie: Efficient, Transparent Temporal Multiplexing for Consumer GPUs
Yechen Xu, Yifei Wang, Nathanael Ren +2
Consumer machines are increasingly running large ML workloads such as large language models (LLMs), text-to-image generation, and interactive image editing. Unlike datacenter GPUs,…
FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models
Zishan Shao, Yixiao Wang, Qinsi Wang +6
Singular Value Decomposition (SVD) has recently seen a surge of interest as a simple yet powerful tool for large language models (LLMs) compression, with a growing number of works…
MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing
Haiyue Ma, Zhixu Du, Yiran Chen
In multi-GPU Mixture-of-Experts (MoE) network, experts are distributed across different GPUs, which creates load imbalance as each expert processes different number of tokens. Rece…
CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion
Yiran Chen, Anyi Rao, Xuekun Jiang +5
With advancements in video generative AI models (e.g., SORA), creators are increasingly using these techniques to enhance video previsualization. However, they face challenges with…