230 citations · 238 across the 6 of their papers we have counts for
6 papers
An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI
Ross Gruetzemacher, Alan Chan, Kevin Frazier +11
Given rapid progress toward advanced AI and risks from frontier AI systems (advanced AI systems pushing the boundaries of the AI capabilities frontier), the creation and implementa…
Predictable Artificial Intelligence
Lexin Zhou, Pablo A. Moreno-Casares, Fernando Martínez-Plumed +12
We introduce the fundamental ideas and challenges of Predictable AI, a nascent research area that explores the ways in which we can anticipate key validity indicators (e.g., perfor…
Inferring Capabilities from Task Performance with Bayesian Triangulation
John Burden, Konstantinos Voudouris, Ryan Burnell +3
As machine learning models become more general, we need to characterise them in richer, more meaningful ways. We describe a method to infer the cognitive profile of a system from d…
Compute and Energy Consumption Trends in Deep Learning Inference
Radosvet Desislavov, Fernando Martínez-Plumed, José Hernández-Orallo
The progress of some AI paradigms such as deep learning is said to be linked to an exponential growth in the number of parameters. There are many studies corroborating these trends…
Conditional Teaching Size
Manuel Garcia-Piqueras, José Hernández-Orallo
Recent research in machine teaching has explored the instruction of any concept expressed in a universal language. In this compositional context, new experimental results have show…
Between Progress and Potential Impact of AI: the Neglected Dimensions
Fernando Martínez-Plumed, Shahar Avin, Miles Brundage +3
We reframe the analysis of progress in AI by incorporating into an overall framework both the task performance of a system, and the time and resource costs incurred in the developm…