What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training

Subham Rath, Raj Dandekar, Rajat Dandekar +1

pass@$k$, the fraction of problems a model solves within $k$ sampled attempts, is the field's default protocol for deciding whether reinforcement-learning (RL) post-training on ver…