Few Trials, Many Metrics: Beta Mixtures for Low-Budget AI Evaluation

Gustavo Tironi, Felipe Maia Polo, Diego Mesquita

Evaluating a stochastic AI system often requires many repeated samples per instance, especially for metrics defined over long horizons. Recent work has shown that modeling instance…