Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents

Aaditya Khanal, Yangyang Tao, Junxiu Zhou

Existing benchmarks measure capability -- whether a model succeeds on a single attempt -- but production deployments require reliability -- consistent success across repeated attem…