19 papers
Do Vision-Language-Action Models Understand Instructions? A Mechanistic Interpretability Study on Language Grounding
Theodor Wulff, Angelo Cangelosi
Vision-Language-Action models are designed to generalise across environments and task descriptions, raising the question of whether their action generation actually depends on the…
PHIRL: Aligning Learned Rewards with Task Progress for Inverse Reinforcement Learning
Hang Yu, James Staley, Cheng Xi Tsou +8
Human demonstrations provide dense policy-level information but sometimes lack local precision. Human feedback presents accurate local critiques, but offers sparse evaluations rath…
Latent Cluster Analysis for Vision-Language-Action Models
Theodor Wulff, Sergio Lanza, Tamara Bila +3
Vision-Language-Action (VLA) Models are increasingly used in robotics for their ability to ground language and perception into action, yet the internal representations driving thei…
Vision-Language Models are Fragile Multilingual Associators
Ritabrata Chakraborty, Rajatsubhra Chakraborty, Shivakumara Palaiahnakote +2
Vision-language models must associate visual entities with textual attributes. Whether these associations or concept bindings remain stable when the language of the input changes i…
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
Manith Adikari, Bei Peng, Samuele Vinanzi +1
Reinforcement Learning (RL) systems are typically trained using a single, well-specified scalar reward function. However, real-world decision-making tasks often involve multiple, c…
Grounding Hierarchical Vision-Language-Action Models Through Explicit Language-Action Alignment
Theodor Wulff, Federico Tavella, Rahul Singh Maharjan +2
Achieving robot transparency is a critical step toward effective human-robot collaboration. To be transparent, a robot's natural language communication must be consistent with its…
Hierarchical, Interpretable, Label-Free Concept Bottleneck Model
Haodong Xie, Yujun Cai, Rahul Singh Maharjan +3
Concept Bottleneck Models (CBMs) introduce interpretability to black-box deep learning models by predicting labels through human-understandable concepts. However, unlike humans, wh…
The Safety Challenge of World Models for Embodied AI Agents: A Review
Lorenzo Baraldi, Zifan Zeng, Chongzhe Zhang +8
The rapid progress in embodied artificial intelligence has highlighted the necessity for more advanced and integrated models that can perceive, interpret, and predict environmental…
DeGuV: Depth-Guided Visual Reinforcement Learning for Generalization and Interpretability in Manipulation
Tien Pham, Xinyun Chi, Khang Nguyen +2
Reinforcement learning (RL) agents can learn to solve complex tasks from visual inputs, but generalizing these learned skills to new environments remains a major challenge in RL ap…
Revisiting Data Attribution for Influence Functions
Hongbo Zhu, Angelo Cangelosi
The goal of data attribution is to trace the model's predictions through the learning algorithm and back to its training data. thereby identifying the most influential training sam…
Representation Understanding via Activation Maximization
Hongbo Zhu, Angelo Cangelosi
Understanding internal feature representations of deep neural networks (DNNs) is a fundamental step toward model interpretability. Inspired by neuroscience methods that probe biolo…
LHM-Humanoid: Long-Horizon Human Motion Control for Continuous Object Transport in Cluttered Scenes
Haozhuo Zhang, Jingkai Sun, Michele Caprio +5
Physics-based human motion control can make a simulated character walk, sit, and manipulate objects with high physical realism. Almost always, though, this happens in short, isolat…
Fake or Real, Can Robots Tell? Evaluating VLM Robustness to Domain Shift in Single-View Robotic Scene Understanding
Federico Tavella, Amber Drinkwater, Angelo Cangelosi
Robotic scene understanding increasingly relies on Vision-Language Models (VLMs) to generate natural language descriptions of the environment. In this work, we systematically evalu…
Joint Action Language Modelling for Transparent Policy Execution
Theodor Wulff, Rahul Singh Maharjan, Xinyun Chi +1
An agent's intention often remains hidden behind the black-box nature of embodied policies. Communication using natural language statements that describe the next action can provid…
Attributes-aware Visual Emotion Representation Learning
Rahul Singh Maharjan, Marta Romeo, Angelo Cangelosi
Visual emotion analysis or recognition has gained considerable attention due to the growing interest in understanding how images can convey rich semantics and evoke emotions in hum…
The ATTUNE model for Artificial Trust Towards Human Operators
Giannis Petousakis, Angelo Cangelosi, Rustam Stolkin +1
This paper presents a novel method to quantify Trust in HRI. It proposes an HRI framework for estimating the Robot Trust towards the Human in the context of a narrow and specified…
From Concrete to Abstract: A Multimodal Generative Approach to Abstract Concept Learning
Haodong Xie, Rahul Singh Maharjan, Federico Tavella +1
Understanding and manipulating concrete and abstract concepts is fundamental to human intelligence. Yet, they remain challenging for artificial agents. This paper introduces a mult…
Noise-Free Explanation for Driving Action Prediction
Hongbo Zhu, Theodor Wulff, Rahul Singh Maharjan +2
Although attention mechanisms have achieved considerable progress in Transformer-based architectures across various Artificial Intelligence (AI) domains, their inner workings remai…
LIPEx-Locally Interpretable Probabilistic Explanations-To Look Beyond The True Class
Hongbo Zhu, Angelo Cangelosi, Procheta Sen +1
In this work, we instantiate a novel perturbation-based multi-class explanation framework, LIPEx (Locally Interpretable Probabilistic Explanation). We demonstrate that LIPEx not on…