2 papers
cs.DC2025
Past-Future Scheduler for LLM Serving under SLA Guarantees
Ruihao Gong, Shihao Bai, Siyu Wu +5
The exploration and application of Large Language Models (LLMs) is thriving. To reduce deployment costs, continuous batching has become an essential feature in current service fram…
cs.CL2025
Pre: Enabling Deterministic Pushdown Automata for Faster Structured LLM Generation
Junyi Chen, Shihao Bai, Zaijun Wang +7
Extensive LLM applications demand efficient structured generations, particularly for LR(1) grammars, to produce outputs in specified formats (e.g., JSON). Existing methods primaril…