7 citations · 9 across the 21 of their papers we have counts for
23 papers
Grounded Joint-Attention Other-Play for Zero-Shot Coordination
Giulia Benintendi, Constantin Ruhdorfer, Fabian Kögel +1
Joint attention - the human ability to share a common visual or cognitive focus with others - enables a meeting of minds that lets us coordinate even with unfamiliar partners. In t…
Amortized Bayesian Inference on Multilevel Models of Arbitrary Structure
Daniel Habermann, Andreas Bulling, Stefan T. Radev +1
We develop a general method for amortized Bayesian inference on multilevel models of arbitrary structure. Given a generative model specified as a directed acyclic graph, our method…
CARD: Diagnosing Belief to Action Routing Failures in Vision Language Models
Souptik Kumar Majumdar, Fabian Kögel, Andreas Bulling
Linear probes and activation steering have uncovered that vision-language models (VLMs) internally represent mental states such as agents' beliefs, knowledge, and intentions. Howev…
High entropy leads to symmetry-equivariant policies in Dec-POMDPs
Johannes Forkel, Constantin Ruhdorfer, Michael Beukman +2
We prove that in any Dec-POMDP, sufficiently high entropy regularization ensures that the policy gradient flow with tabular softmax parametrization always converges, for any initia…
ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions
Matteo Bortoletto, Constantin Ruhdorfer, Andreas Bulling
Most existing Theory of Mind (ToM) benchmarks for foundation models rely on variations of the Sally-Anne test, offering only a very limited perspective on ToM and neglecting the co…
ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback
Matteo Bortoletto, Yichao Zhou, Lance Ying +2
While humans are inherently social creatures, the challenge of identifying when and how to assist and collaborate with others - particularly when pursuing independent goals - can h…
The Yokai Learning Environment: Tracking Beliefs Over Space and Time
Constantin Ruhdorfer, Matteo Bortoletto, Johannes Forkel +2
The ability to cooperate with unknown partners is a central challenge in cooperative AI and widely studied in the form of zero-shot coordination (ZSC), which evaluates an algorithm…
Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
Constantin Ruhdorfer, Matteo Bortoletto, Victor Oei +2
We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly a…
VDial: Unification of Video and Visual Dialog via Multimodal Experts
Adnen Abdessaied, Anna Rohrbach, Marcus Rohrbach +1
We present VDial - a novel expert-based model specifically geared towards simultaneously handling image and video input data for multimodal conversational tasks. Current multim…
CLAD: Constrained Latent Action Diffusion for Vision-Language Procedure Planning
Lei Shi, Andreas Bulling
We propose CLAD, a Constrained Latent Action Diffusion model for vision-language procedure planning in instructional videos. Procedure planning is the challenging task of predictin…
MultiMediate'24: Multi-Domain Engagement Estimation
Philipp Müller, Michal Balazia, Tobias Baur +8
Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task…
Explicit Modelling of Theory of Mind for Belief Prediction in Nonverbal Social Interactions
Matteo Bortoletto, Constantin Ruhdorfer, Lei Shi +1
We propose MToMnet - a Theory of Mind (ToM) neural network for predicting beliefs and their dynamics during human social interactions from multimodal input. ToM is key for effectiv…
Amortized Bayesian Multilevel Models
Daniel Habermann, Marvin Schmitt, Lars Kühmichel +3
Multilevel models (MLMs) are a central building block of the Bayesian workflow. They enable joint, interpretable modeling of data across hierarchical levels and provide a fully pro…
Multi-Modal Video Dialog State Tracking in the Wild
Adnen Abdessaied, Lei Shi, Andreas Bulling
We present MST-MIXER - a novel video dialog model operating over a generic multi-modal state tracking scheme. Current models that claim to perform multi-modal state tracking fall s…
The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design
Constantin Ruhdorfer, Matteo Bortoletto, Anna Penzkofer +1
We introduce the Overcooked Generalisation Challenge (OGC) - a new benchmark for evaluating reinforcement learning (RL) agents on their ability to cooperate with unknown partners i…
Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models
Matteo Bortoletto, Constantin Ruhdorfer, Lei Shi +1
Despite growing interest in Theory of Mind (ToM) tasks for evaluating language models (LMs), little is known about how LMs internally represent mental states of self and others. Un…
Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition
Matteo Bortoletto, Constantin Ruhdorfer, Adnen Abdessaied +2
Recent work on dialogue-based collaborative plan acquisition (CPA) has suggested that Theory of Mind (ToM) modelling can improve missing knowledge prediction in settings with asymm…
VSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images
Anna Penzkofer, Lei Shi, Andreas Bulling
While Vector Symbolic Architectures (VSAs) are promising for modelling spatial cognition, their application is currently limited to artificially generated images and simple spatial…
ActionDiffusion: An Action-aware Diffusion Model for Procedure Planning in Instructional Videos
Lei Shi, Paul Bürkner, Andreas Bulling
We present ActionDiffusion -- a novel diffusion model for procedure planning in instructional videos that is the first to take temporal inter-dependencies between actions into acco…
OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog
Adnen Abdessaied, Manuel von Hochmeister, Andreas Bulling
We present the Object Language Video Transformer (OLViT) - a novel model for video dialog operating over a multi-modal attention-based dialog state tracker. Existing video dialog m…
Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI Research
Susanne Hindennach, Lei Shi, Filip Miletić +1
When users perceive AI systems as mindful, independent agents, they hold them responsible instead of the AI experts who created and designed these systems. So far, it has not been…
Neural Reasoning About Agents' Goals, Preferences, and Actions
Matteo Bortoletto, Lei Shi, Andreas Bulling
We propose the Intuitive Reasoning Network (IRENE) - a novel neural model for intuitive psychological reasoning about agents' goals, preferences, and actions that can generalise pr…
-: Boosting isual ialog with Cascaded Spatial-Temporal Multi-Modal aphs
Adnen Abdessaied, Lei Shi, Andreas Bulling
We propose - - a novel visual dialog model that combines pre-trained language models (LMs) with graph neural networks (GNNs). Prior works mainly focused o…