Publications (6)
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
Alicia Parrish, Hannah Rose Kirk, Jessica Quaye +10
The generative AI revolution in recent years has been spurred by an expansion in compute power and data quantity, which together enable extensive pre-training of powerful text-to-i…
Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation
D. Sculley, Will Cukierski, Phil Culliton +8
In this position paper, we observe that empirical evaluation in Generative AI is at a crisis point since traditional ML evaluation and benchmarking strategies are insufficient to m…
Open Polymer Challenge: Post-Competition Report
Gang Liu, Sobin Alosious, Subhamoy Mahajan +9
Machine learning (ML) offers a powerful path toward discovering sustainable polymer materials, but progress has been limited by the lack of large, high-quality, and openly accessib…
Challenge design roadmap
Hugo Jair Escalante Balderas, Isabelle Guyon, Addison Howard +2
Challenges can be seen as a type of game that motivates participants to solve serious tasks. As a result, competition organizers must develop effective game rules. However, these r…
Exploring Human-AI Conceptual Alignment through the Prism of Chess
Semyon Lomasov, Judah Goldfeder, Mehmet Hamza Erol +5
Do AI systems truly understand human concepts or merely mimic surface patterns? We investigate this through chess, where human creativity meets precise strategic concepts. Analyzin…
A community-powered search of machine learning strategy space to find NMR property prediction models
Lars A. Bratholm, Will Gerrard, Brandon Anderson +26
The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions. For physical scientists wishing to apply ML st…