Position: AI Evaluations Should be Grounded on a Theory of Capability

Nathanael Jo, Ashia Wilson

Evaluations of generative models are now ubiquitous, and their outcomes critically shape public and scientific expectations of AI's capabilities. Yet skepticism about their reliabi…