103 citations · 275 across the 25 of their papers we have counts for
25 papers
Multi-Modal 3D Scene Graph Updater for Shared and Dynamic Environments
Emilio Olivastri, Jonathan Francis, Alberto Pretto +2
The advent of generalist Large Language Models (LLMs) and Large Vision Models (VLMs) have streamlined the construction of semantically enriched maps that can enable robots to groun…
Physically Embodied Gaussian Splatting: A Realtime Correctable World Model for Robotics
Jad Abou-Chakra, Krishan Rana, Feras Dayoub +1
For robots to robustly understand and interact with the physical world, it is highly beneficial to have a comprehensive representation - modelling geometry, physics, and visual obs…
RoboHop: Segment-based Topological Map Representation for Open-World Visual Navigation
Sourav Garg, Krishan Rana, Mehdi Hosseinzadeh +4
Mapping is crucial for spatial reasoning, planning and robot navigation. Existing approaches range from metric, which require precise geometry-based optimization, to purely topolog…
LHManip: A Dataset for Long-Horizon Language-Grounded Manipulation Tasks in Cluttered Tabletop Environments
Federico Ceola, Lorenzo Natale, Niko Sünderhauf +1
Instructing a robot to complete an everyday task within our homes has been a long-standing challenge for robotics. While recent progress in language-conditioned imitation learning…
Contrastive Language, Action, and State Pre-training for Robot Learning
Krishan Rana, Andrew Melnik, Niko Sünderhauf
In this paper, we introduce a method for unifying language, action, and state information in a shared embedding space to facilitate a range of downstream tasks in robot learning. O…
Addressing the Challenges of Open-World Object Detection
David Pershouse, Feras Dayoub, Dimity Miller +1
We address the challenging problem of open world object detection (OWOD), where object detectors must identify objects from known classes while also identifying and continually lea…
Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics
Krishan Rana, Ming Xu, Brendan Tidd +2
Skill-based reinforcement learning (RL) has emerged as a promising strategy to leverage prior knowledge for accelerated robot learning. Skills are typically extracted from expert d…
ParticleNeRF: A Particle-Based Encoding for Online Neural Radiance Fields
Jad Abou-Chakra, Feras Dayoub, Niko Sünderhauf
While existing Neural Radiance Fields (NeRFs) for dynamic scenes are offline methods with an emphasis on visual fidelity, our paper addresses the online use case that prioritises r…
Retrospectives on the Embodied AI Workshop
Matt Deitke, Dhruv Batra, Yonatan Bisk +36
We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped i…
SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detection
Samuel Wilson, Tobias Fischer, Feras Dayoub +2
We address the problem of out-of-distribution (OOD) detection for the task of object detection. We show that residual convolutional layers with batch normalisation produce Sensitiv…
Implicit Object Mapping With Noisy Data
Jad Abou-Chakra, Feras Dayoub, Niko Sünderhauf
Modelling individual objects in a scene as Neural Radiance Fields (NeRFs) provides an alternative geometric scene representation that may benefit downstream robotics tasks such as…
Zero-Shot Uncertainty-Aware Deployment of Simulation Trained Policies on Real-World Robots
Krishan Rana, Vibhavari Dasagi, Jesse Haviland +3
While deep reinforcement learning (RL) agents have demonstrated incredible potential in attaining dexterous behaviours for robotics, they tend to make errors when deployed in the r…
FSNet: A Failure Detection Framework for Semantic Segmentation
Quazi Marufur Rahman, Niko Sünderhauf, Peter Corke +1
Semantic segmentation is an important task that helps autonomous vehicles understand their surroundings and navigate safely. During deployment, even the most mature segmentation mo…
Evaluating the Impact of Semantic Segmentation and Pose Estimation on Dense Semantic SLAM
Suman Raj Bista, David Hall, Ben Talbot +3
Recent Semantic SLAM methods combine classical geometry-based estimation with deep learning-based object detection or semantic segmentation. In this paper we evaluate the quality o…
Probabilistic Appearance-Invariant Topometric Localization with New Place Awareness
Ming Xu, Tobias Fischer, Niko Sünderhauf +1
Probabilistic state-estimation approaches offer a principled foundation for designing localization systems, because they naturally integrate sequences of imperfect motion and exter…
Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Robotics
Krishan Rana, Vibhavari Dasagi, Jesse Haviland +3
We present Bayesian Controller Fusion (BCF): a hybrid control strategy that combines the strengths of traditional hand-crafted controllers and model-free deep reinforcement learnin…
Uncertainty for Identifying Open-Set Errors in Visual Object Detection
Dimity Miller, Niko Sünderhauf, Michael Milford +1
Deployed into an open world, object detectors are prone to open-set errors, false positive detections of object classes not present in the training dataset. We propose GMM-Det, a r…
Semantics for Robotic Mapping, Perception and Interaction: A Survey
Sourav Garg, Niko Sünderhauf, Feras Dayoub +9
For robots to navigate and interact more richly with the world around them, they will likely require a deeper understanding of the world in which they operate. In robotics and rela…
SWA Object Detection
Haoyang Zhang, Ying Wang, Feras Dayoub +1
Do you want to improve 1.0 AP for your object detector without any inference cost and any change to your detector? Let us tell you such a recipe. It is surprisingly simple: train y…
Online Monitoring of Object Detection Performance During Deployment
Quazi Marufur Rahman, Niko Sünderhauf, Feras Dayoub
During deployment, an object detector is expected to operate at a similar performance level reported on its testing dataset. However, when deployed onboard mobile robots that opera…
Per-frame mAP Prediction for Continuous Performance Monitoring of Object Detection During Deployment
Quazi Marufur Rahman, Niko Sünderhauf, Feras Dayoub
Performance monitoring of object detection is crucial for safety-critical applications such as autonomous vehicles that operate under varying and complex environmental conditions.…
BenchBot: Evaluating Robotics Research in Photorealistic 3D Simulation and on Real Robots
Ben Talbot, David Hall, Haoyang Zhang +4
We introduce BenchBot, a novel software suite for benchmarking the performance of robotics research across both photorealistic 3D simulations and real robot platforms. BenchBot pro…
VarifocalNet: An IoU-aware Dense Object Detector
Haoyang Zhang, Ying Wang, Feras Dayoub +1
Accurately ranking the vast number of candidate detections is crucial for dense object detectors to achieve high performance. Prior work uses the classification score or a combinat…
Class Anchor Clustering: a Loss for Distance-based Open Set Recognition
Dimity Miller, Niko Sünderhauf, Michael Milford +1
In open set recognition, deep neural networks encounter object classes that were unknown during training. Existing open set classifiers distinguish between known and unknown classe…