False Floors: LLM Safety Routing Evaluations Break Under Distribution Shift

Amit Singh Bhatti, Vishal Vaddina

Safety routers send each request to one of several models and are judged against the best single model. A major routing benchmark picks that comparator on the evaluation data. In t…