Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference

Zheyuan Wang, Siyu Li, Peiqiao Song +3

Large language model (LLM) cascades answer easy requests with a small model and escalate selected requests to a larger model. Most routers prioritize examples on which the small mo…