1 citations · 1 across the 4 of their papers we have counts for
3 papers · 1 filter
Using profiles of cognitive capability to assess AI suitability for workplace tasks
Jonathan Prunty, Marko Tešić, Patrick Quinn +2
Organisations deploying AI face a scoping problem: which tasks can be automated, which should remain with humans, and which are best shared between the two. Aggregate benchmark sco…
Visuospatial Perspective Taking in Multimodal Language Models
Jonathan Prunty, Seraphina Zhang, Patrick Quinn +3
As multimodal language models (MLMs) are increasingly used in social and collaborative settings, it is crucial to evaluate their perspective-taking abilities. Existing benchmarks l…
Capabilities Ain't All You Need: Measuring Propensities in AI
Daniel Romero-Alvarado, Fernando Martínez-Plumed, Lorenzo Pacchiardi +11
AI evaluation has primarily focused on measuring capabilities, with formal approaches inspired from Item Response Theory (IRT) being increasingly applied. Yet propensities - the te…