24 citations · 32 across the 6 of their papers we have counts for
7 papers
Game Arena: Strategic LLM Evaluation in Competitive Environments
Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu +59
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena…
Open Polymer Challenge: Post-Competition Report
Gang Liu, Sobin Alosious, Subhamoy Mahajan +9
Machine learning (ML) offers a powerful path toward discovering sustainable polymer materials, but progress has been limited by the lack of large, high-quality, and openly accessib…
Exploring Human-AI Conceptual Alignment through the Prism of Chess
Semyon Lomasov, Judah Goldfeder, Mehmet Hamza Erol +5
Do AI systems truly understand human concepts or merely mimic surface patterns? We investigate this through chess, where human creativity meets precise strategic concepts. Analyzin…
Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation
D. Sculley, Will Cukierski, Phil Culliton +8
In this position paper, we observe that empirical evaluation in Generative AI is at a crisis point since traditional ML evaluation and benchmarking strategies are insufficient to m…
Challenge design roadmap
Hugo Jair Escalante Balderas, Isabelle Guyon, Addison Howard +2
Challenges can be seen as a type of game that motivates participants to solve serious tasks. As a result, competition organizers must develop effective game rules. However, these r…
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
Alicia Parrish, Hannah Rose Kirk, Jessica Quaye +10
The generative AI revolution in recent years has been spurred by an expansion in compute power and data quantity, which together enable extensive pre-training of powerful text-to-i…
A community-powered search of machine learning strategy space to find NMR property prediction models
Lars A. Bratholm, Will Gerrard, Brandon Anderson +26
The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions. For physical scientists wishing to apply ML st…