most citedLayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers

1 citations · 1 across the 5 of their papers we have counts for

collaborators
Showing cs.DCShow all

1 paper · 1 filter

cs.DC2025

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving

Rui Li, Zhaoning Zhang, Libo Zhang +3

Speculative decoding (SD) accelerates LLM inference by verifying draft tokens in parallel. However, this method presents a critical trade-off: it improves throughput in low-load, m…