Spexis: Speculative Lookahead Scheduling for LLM Inference

Hyungyu Jung, Jaehyeok Yu, Hoonseo Choi +3

Spexis is a multi-GPU LLM inference framework that improves the efficiency of pipeline and tensor parallelism through speculative parallelism. Rather than using speculative decodin…