27 citations · 44 across the 22 of their papers we have counts for
23 papers
Towards VLA-Dreamer: Refining VLA Behavior Using World Models
Parsa Mastouri Kashani, Jan-Gerrit Habekost, Stefan Wermter
Vision-Language-Action models (VLAs), while showing strong potential for robot control, require massive amounts of high-quality imitation learning data. Moreover, the absence of an…
SmoLSTM: A Compact Vision-Language-Action Model with Recurrent Memory that Persists
Jan-Gerrit Habekost, Parsa Mastouri Kashani, Connor Gäde +5
Vision-language-action models often predict actions from only the current observation, which can leave tasks involving object occlusion or visually identical objects ambiguous with…
Latent Cluster Analysis for Vision-Language-Action Models
Theodor Wulff, Sergio Lanza, Tamara Bila +3
Vision-Language-Action (VLA) Models are increasingly used in robotics for their ability to ground language and perception into action, yet the internal representations driving thei…
When May I Help You? On The Effect of Proactivity on Group Human-Robot Collaboration
Thomas Vitry, Vanessa Maeder, Kieran von Valeburg +6
Robot initiative is a central challenge in multi-party human-robot collaboration. A robot that contributes without being addressed may provide timely support, but it may also disru…
Towards Learning a Generalizable 3D Scene Representation from 2D Observations
Martin Gromniak, Jan-Gerrit Habekost, Sebastian Kamp +2
We introduce a Generalizable Neural Radiance Field approach for predicting 3D workspace occupancy from egocentric robot observations. Unlike prior methods operating in camera-centr…
Robots with Attitudes: Influence of LLM-Driven Robot Personalities on Motivation and Performance
Dennis Becker, Kyra Ahrens, Connor Gäde +2
Large language models enable unscripted conversations while maintaining a consistent personality. One desirable personality trait in cooperative partners, known to improve task per…
Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
Hans Hergen Lehmann, Jae Hee Lee, Steven Schockaert +1
Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains…
Keypoint-based Diffusion for Robotic Motion Planning on the NICOL Robot
Lennart Clasmeier, Jan-Gerrit Habekost, Connor Gäde +2
We propose a novel diffusion-based action model for robotic motion planning. Commonly, established numerical planning approaches are used to solve general motion planning problems,…
Pointing-Guided Target Estimation via Transformer-Based Attention
Luca Müller, Hassan Ali, Philipp Allgeuer +2
Deictic gestures, like pointing, are a fundamental form of non-verbal communication, enabling humans to direct attention to specific objects or locations. This capability is essent…
Balancing long- and short-term dynamics for the modeling of saliency in videos
Theodor Wulff, Fares Abawi, Philipp Allgeuer +1
The role of long- and short-term dynamics towards salient object detection in videos is under-researched. We present a Transformer-based approach to learn a joint representation of…
Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?
Jae Hee Lee, Georgii Mikriukov, Gesina Schwalbe +2
Concept-based XAI (C-XAI) approaches to explaining neural vision models are a promising field of research, since explanations that refer to concepts (i.e., semantically meaningful…
Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation
Hassan Ali, Philipp Allgeuer, Carlo Mazzola +4
Large Language Models (LLMs) have been recently used in robot applications for grounding LLM common-sense reasoning with the robot's perception and physical abilities. In humanoid…
Unconstrained Open Vocabulary Image Classification: Zero-Shot Transfer from Text to Image via CLIP Inversion
Philipp Allgeuer, Kyra Ahrens, Stefan Wermter
We introduce NOVIC, an innovative real-time uNconstrained Open Vocabulary Image Classifier that uses an autoregressive transformer to generatively output classification labels as l…
When Robots Get Chatty: Grounding Multimodal Human-Robot Conversation and Collaboration
Philipp Allgeuer, Hassan Ali, Stefan Wermter
We investigate the use of Large Language Models (LLMs) to equip neural robotic agents with human-like social and cognitive competencies, for the purpose of open-ended human-robot c…
Unified Dynamic Scanpath Predictors Outperform Individually Trained Neural Models
Fares Abawi, Di Fu, Stefan Wermter
Previous research on scanpath prediction has mainly focused on group models, disregarding the fact that the scanpaths and attentional behaviors of individuals are diverse. The disr…
Inverse Kinematics for Neuro-Robotic Grasping with Humanoid Embodied Agents
Jan-Gerrit Habekost, Connor Gäde, Philipp Allgeuer +1
This paper introduces a novel zero-shot motion planning method that allows users to quickly design smooth robot motions in Cartesian space. A Bézier curve-based Cartesian plan is t…
Comparing Apples to Oranges: LLM-powered Multimodal Intention Prediction in an Object Categorization Task
Hassan Ali, Philipp Allgeuer, Stefan Wermter
Human intention-based systems enable robots to perceive and interpret user actions to interact with humans and adapt to their behavior proactively. Therefore, intention prediction…
Human Impression of Humanoid Robots Mirroring Social Cues
Di Fu, Fares Abawi, Philipp Allgeuer +1
Mirroring non-verbal social cues such as affect or movement can enhance human-human and human-robot interactions in the real world. The robotic platforms and control methods also i…
Causal State Distillation for Explainable Reinforcement Learning
Wenhao Lu, Xufeng Zhao, Thilo Fryen +4
Reinforcement learning (RL) is a powerful technique for training intelligent agents, but understanding why these agents make specific decisions can be quite challenging. This lack…
Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models
Kyra Ahrens, Hans Hergen Lehmann, Jae Hee Lee +1
We address the Continual Learning (CL) problem, wherein a model must learn a sequence of tasks from non-stationary distributions while preserving prior knowledge upon encountering…
Visually Grounded Continual Language Learning with Selective Specialization
Kyra Ahrens, Lennart Bengtson, Jae Hee Lee +1
A desirable trait of an artificial agent acting in the visual world is to continually learn a sequence of language-informed tasks while striking a balance between sufficiently spec…
From Neural Activations to Concepts: A Survey on Explaining Concepts in Neural Networks
Jae Hee Lee, Sergio Lanza, Stefan Wermter
In this paper, we review recent approaches for explaining concepts in neural networks. Concepts can act as a natural link between learning and reasoning: once the concepts are iden…
Neuro-Symbolic Spatio-Temporal Reasoning
Jae Hee Lee, Michael Sioutis, Kyra Ahrens +3
Knowledge about space and time is necessary to solve problems in the physical world: An AI agent situated in the physical world and interacting with objects often needs to reason a…