15 papers
Two routes to the Hadamard condition
Eleanor March
In quantum field theory on curved spacetime, the Hadamard condition is often presented as a necessary condition on 'physically reasonable', 'vacuum-like' states of linear quantum f…
Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks
Sophie Higham, Riccardo Andrea Izzo, Matteo Matteucci +1
Vision-Language-Action (VLA) models have achieved high task success rates on robot manipulation task benchmarks. More recently, there has been an emphasis on evaluating the robustn…
Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs
Riccardo Andrea Izzo, Rimvydas Rubavicius, Gianluca Bardaro +3
Flow-matching Vision-Language-Action (VLA) models have emerged as a potential solution for generalist robot control, designed by combining a pretrained Vision-Language Model (VLM)…
Investigating Knowledge Transfer Across Interactive Dialogue Games
Filippo Momentè, Mir Nafis Sharear Shopnil, Andrea de Varda +5
Dialogue games represent a challenging setting where complex cognitive skills are required to accomplish tasks while coordinating with other players. Considering that language repr…
GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes
Amit Parekh, Sabrina McCallum, Kareem Al-Hasan +3
Multimodal models are increasingly deployed to solve tasks collaboratively with humans or other artificial agents. While existing benchmarks show that they possess the fundamental…
Do Composed Image Retrieval Benchmarks Require Multimodal Composition?
Matteo Attimonelli, Alessandro De Bellis, Aryo Pradipta Gema +8
Composed Image Retrieval (CIR) is a multimodal retrieval task where a query consists of a reference image and a textual modification, and the goal is to retrieve a target image sat…
MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts
Chen Hu, Yintao Tai, Antonio Vergari +2
Pixel-based language models are gaining momentum as alternatives to traditional token-based approaches, promising to circumvent tokenization challenges; however, existing models re…
AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification
Hamza Mooraj, George Pantazopoulos, Alessandro Suglia
Reliable crop disease detection requires models that perform consistently across diverse acquisition conditions, yet existing evaluations often focus on single architectural famili…
VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models
Rohit Saxena, Alessandro Suglia, Pasquale Minervini
Vision-language models (VLMs) achieve strong performance on standard, high-quality datasets, but we still do not fully understand how they perform under real-world image distortion…
Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architectures
Georgios Pantazopoulos, Malvina Nikandrou, Ioannis Konstas +1
Transformers excel at in-context retrieval but suffer from quadratic complexity with sequence length, while State Space Models (SSMs) offer efficient linear-time processing but hav…
Same Answer, Different Representations: Hidden instability in VLMs
Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6
The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processi…
Improving Cooperation in Collaborative Embodied AI
Hima Jacob Leven Suprabha, Laxmi Nag Laxminarayan Nagesh, Ajith Nair +10
The integration of Large Language Models (LLMs) into multiagent systems has opened new possibilities for collaborative reasoning and cooperation with AI agents. This paper explores…
VoyagerVision: Investigating the Role of Multi-modal Information for Open-ended Learning Systems
Ethan Smyth, Alessandro Suglia
Open-endedness is an active field of research in the pursuit of capable Artificial General Intelligence (AGI), allowing models to pursue tasks of their own choosing. Simultaneously…
How Well Do Vision-Language Models Understand Movies?
Emmanouil Zaranis, António Farinhas, Saul Santos +28
Vision-language models (VLMs) can increasingly handle extended contexts and answer questions about long videos--but can they understand the narrative structure of movies as humans…
Playpen: An Environment for Exploring Learning Through Conversational Interaction
Nicola Horst, Davide Mazzaccara, Antonia Schmidt +13
Interaction between learner and feedback-giver has come into focus recently for post-training of Large Language Models (LLMs), through the use of reward models that judge the appro…