ZeRO-Prefill: Zero Redundancy Overheads in MoE Prefill Serving

Zhaoyuan Su, Olatunji Ruwase, Karthik Ganesan +5

Production LLM workloads increasingly serve discriminative tasks, such as classification, recommendation, and verification, whose answers are read from the logits of a single prefi…