Causal Judge Evaluation: Calibrated Surrogate Metrics for LLM Systems

Eddie Landesberg, Manjari Narayan

Measuring long-run LLM outcomes (user satisfaction, expert judgment, downstream KPIs) is expensive. Teams default to cheap LLM judges, but uncalibrated proxies can invert rankings…