The Statistical Benefits of Multiple Responses for Learning from Demonstrations

Chandramauli Chakraborty, Cong Ma

Many generative systems return multiple candidate responses and are evaluated according to the best one. Recent work shows that, when demonstrations are optimal, pass@$k$ can reduc…