PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference

Niqi Lyu, Pengtao Shi, Wei Qiu +4

Large language models (LLMs) provide strong reasoning capabilities but are expensive to serve at scale, whereas small language models (SLMs) are cheaper but less reliable on diffic…