80 citations · 104 across the 24 of their papers we have counts for
28 papers
Task-Adaptive Grounded 3D-Programmers Using 2D VLMs
Arman Raayatsanati, Sombit Dey, Anna-Maria Halacheva +3
Recent vision-language models (VLMs) exhibit remarkable generalization and reasoning abilities, yet 3D understanding in these models is limited by data scale, training diversity, a…
AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models
Yutong Hu, Jan-Nico Zaech, Nikolay Nikolov +6
We propose a standalone autoregressive (AR) Action Expert that generates actions as a continuous causal sequence while conditioning on refreshable vision-language prefixes. In cont…
Advances and Innovations in the Multi-Agent Robotic System (MARS) Challenge
Li Kang, Heng Zhou, Xiufeng Song +41
Recent advancements in multimodal large language models and vision-languageaction models have significantly driven progress in Embodied AI. As the field transitions toward more com…
ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors
Liming Kuang, Yordanka Velikova, Mahdi Saleh +3
Object pose estimation is a fundamental task in computer vision and robotics, yet most methods require extensive, dataset-specific training. Concurrently, large-scale vision langua…
Autonomous Vehicle Path Planning by Searching With Differentiable Simulation
Asen Nachkov, Jan-Nico Zaech, Danda Pani Paudel +2
Planning allows an agent to safely refine its actions before executing them in the real world. In autonomous driving, this is crucial to avoid collisions and navigate in complex, d…
SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
Nikolay Nikolov, Giuliano Albanese, Sombit Dey +4
Robotic Foundation Models (RFMs) hold great promise as generalist, end-to-end systems for robot control. Yet their ability to generalize across new environments, tasks, and embodim…
LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentation
Yang Miao, Jan-Nico Zaech, Xi Wang +3
We propose LangHOPS, the first Multimodal Large Language Model (MLLM) based framework for open-vocabulary object-part instance segmentation. Given an image, LangHOPS can jointly de…
Generalist Robot Manipulation beyond Action Labeled Data
Alexander Spiridonov, Jan-Nico Zaech, Nikolay Nikolov +2
Recent advances in generalist robot manipulation leverage pre-trained Vision-Language Models (VLMs) and large-scale robot demonstrations to tackle diverse tasks in a zero-shot mann…
From Scan to Action: Leveraging Realistic Scans for Embodied Scene Understanding
Anna-Maria Halacheva, Jan-Nico Zaech, Sombit Dey +2
Real-world 3D scene-level scans offer realism and can enable better real-world generalizability for downstream applications. However, challenges such as data volume, diverse annota…
GaussianVLM: Scene-centric 3D Vision-Language Models using Language-aligned Gaussian Splats for Embodied Reasoning and Beyond
Anna-Maria Halacheva, Jan-Nico Zaech, Xi Wang +2
As multimodal language models advance, their application to 3D scene understanding is a fast-growing frontier, driving the development of 3D Vision-Language Models (VLMs). Current…
RhoDARTS: Differentiable Quantum Architecture Search with Density Matrix Simulations
Swagat Kumar, Jan-Nico Zaech, Colin Michael Wilmott +1
Variational Quantum Algorithms (VQAs) are a promising approach to leverage Noisy Intermediate-Scale Quantum (NISQ) computers. However, choosing optimal quantum circuits that effici…
Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms: Challenges and a Roadmap
Jianye Xu, Johannes Betz, Armin Mokhtarian +11
This article proposes a roadmap to address the current challenges in small-scale testbeds for Connected and Automated Vehicles (CAVs) and robot swarms. The roadmap is a joint effor…
Unlocking Efficient Vehicle Dynamics Modeling via Analytic World Models
Asen Nachkov, Danda Pani Paudel, Jan-Nico Zaech +2
Differentiable simulators represent an environment's dynamics as a differentiable function. Within robotics and autonomous driving, this property is used in Analytic Policy Gradien…
Occam's LGS: An Efficient Approach for Language Gaussian Splatting
Jiahuan Cheng, Jan-Nico Zaech, Luc Van Gool +1
TL;DR: Gaussian Splatting is a widely adopted approach for 3D scene representation, offering efficient, high-quality reconstruction and rendering. A key reason for its success is t…
Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Description
Anna-Maria Halacheva, Yang Miao, Jan-Nico Zaech +3
3D scene understanding is a long-standing challenge in computer vision and a key component in enabling mixed reality, wearable computing, and embodied AI. Providing a solution to t…
ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models
Sombit Dey, Jan-Nico Zaech, Nikolay Nikolov +2
Recent progress in large language models and access to large-scale robotic datasets has sparked a paradigm shift in robotics models transforming them into generalists able to adapt…
Probabilistic Sampling of Balanced K-Means using Adiabatic Quantum Computing
Jan-Nico Zaech, Martin Danelljan, Tolga Birdal +1
Adiabatic quantum computing (AQC) is a promising approach for discrete and often NP-hard optimization problems. Current AQCs allow to implement problems of research interest, which…
Adiabatic Quantum Computing for Multi Object Tracking
Jan-Nico Zaech, Alexander Liniger, Martin Danelljan +2
Multi-Object Tracking (MOT) is most often approached in the tracking-by-detection paradigm, where object detections are associated through time. The association step naturally lead…
Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Prediction
Edoardo Mello Rella, Jan-Nico Zaech, Alexander Liniger +1
Forecasting the future behavior of all traffic agents in the vicinity is a key task to achieve safe and reliable autonomous driving systems. It is a challenging problem as agents a…
Learnable Online Graph Representations for 3D Multi-Object Tracking
Jan-Nico Zaech, Dengxin Dai, Alexander Liniger +2
Tracking of objects in 3D is a fundamental task in computer vision that finds use in a wide range of applications such as autonomous driving, robotics or augmented reality. Most re…
Unsupervised Robust Domain Adaptation without Source Data
Peshal Agarwal, Danda Pani Paudel, Jan-Nico Zaech +1
We study the problem of robust domain adaptation in the context of unavailable target labels and source data. The considered robustness is against adversarial perturbations. This p…
Action Sequence Predictions of Vehicles in Urban Environments using Map and Social Context
Jan-Nico Zaech, Dengxin Dai, Alexander Liniger +1
This work studies the problem of predicting the sequence of future actions for surround vehicles in real-world driving scenarios. To this aim, we make three main contributions. The…
Semantic Understanding of Foggy Scenes with Purely Synthetic Data
Martin Hahner, Dengxin Dai, Christos Sakaridis +2
This work addresses the problem of semantic scene understanding under foggy road conditions. Although marked progress has been made in semantic scene understanding over the recent…
Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories
Jan-Nico Zaech, Cong Gao, Bastian Bier +4
Metal artifacts in computed tomography (CT) arise from a mismatch between physics of image formation and idealized assumptions during tomographic reconstruction. These artifacts ar…