HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools

Aashna Garg, Siddharth Singha Roy, Jinu Jang +2

Production LLM deployments increasingly maintain heterogeneous model pools spanning order-of-magnitude cost differences. Existing routers make binary strong-vs-weak decisions and c…