Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Models
Eunyeong Cho, Jehyeon Bang, Ranggi Hwang +1
The emergence of reasoning-based LLMs leveraging Chain-of-Thought (CoT) inference introduces new serving challenges, as their extended reasoning phases delay user-visible output an…
cs.LG2024
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang, Jianyu Wei, Shijie Cao +4
Large language models (LLMs) based on transformers have made significant strides in recent years, the success of which is driven by scaling up their model size. Despite their high…