Cascadia: An Efficient Cascade Serving System for Large Language Models

Youhe Jiang, Fangcheng Fu, Wanru Zhao +4

Recent advances in large language models (LLMs) have intensified the need to deliver both rapid responses and high-quality outputs. More powerful models yield better results but in…