Compass: SLO-aware Query Planner for Compound AI Serving at Scale

Banruo Liu, Wei-Yu Lin, Minghao Fang +2

The rise of compound AI serving that integrates multiple operators in a pipeline enables end-user applications such as generative AI-powered meeting companions, autonomous driving,…