6 papers
Realist and Pluralist Conceptions of Intelligence and Their Implications on AI Research
Ninell Oldenburg, Ruchira Dhar, Anders Søgaard
In this paper, we argue that current AI research operates on a spectrum between two different underlying conceptions of intelligence: Intelligence Realism, which holds that intelli…
On the Measure of a Model: From Intelligence to Generality
Ruchira Dhar, Ninell Oldenburg, Anders Soegaard
Benchmarks such as ARC, Raven-inspired tests, and the Blackbird Task are widely used to evaluate the intelligence of large language models (LLMs). Yet, the concept of intelligence…
EvalCards: A Framework for Standardized Evaluation Reporting
Ruchira Dhar, Danae Sanchez Villegas, Antonia Karamolegkou +11
Evaluation has long been a central concern in NLP, and transparent reporting practices are more critical than ever in today's landscape of rapidly released open-access models. Draw…
Beyond Technocratic XAI: The Who, What & How in Explanation Design
Ruchira Dhar, Stephanie Brandl, Ninell Oldenburg +1
The field of Explainable AI (XAI) offers a wide range of techniques for making complex models interpretable. Yet, in practice, generating meaningful explanations is a context-depen…
Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users
Antonia Karamolegkou, Malvina Nikandrou, Georgios Pantazopoulos +5
This paper explores the effectiveness of Multimodal Large Language models (MLLMs) as assistive technologies for visually impaired individuals. We conduct a user survey to identify…
Defining Knowledge: Bridging Epistemology and Large Language Models
Constanza Fierro, Ruchira Dhar, Filippos Stamatiou +2
Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round? To address this question, we review sta…