4 papers
Croissant Tasks: A Metadata Format for Reproducible Machine Learning Evaluations
Omar Benjelloun, Leonardo Martins Bianco, Isabelle Guyon +8
Reproducibility is fundamental to the scientific method, yet remains a critical challenge in machine learning. Contributing factors include underspecified execution details and bri…
Making the Discrete Continuous: Synthetic RAW Augmentations for Fine-Grained Evaluation of Person Detection Performance in Low Light
Valeria Pais, Malena Mendilaharzu, Daniele Faccio +3
Real-world deployment of AI vision models is both fueled and limited by the data available for training and testing. Real datasets are sparse and uneven: long-tailed or unbalanced…
Autoguided Online Data Curation for Diffusion Model Training
Valeria Pais, Luis Oala, Daniele Faccio +1
The costs of generative model compute rekindled promises and hopes for efficient data curation. In this work, we investigate whether recently developed autoguidance and online data…
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
Shaona Ghosh, Heather Frase, Adina Williams +99
The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehen…