Publications (111)
Informed, Constrained, Aligned: A Field Analysis on Degeneracy-aware Point Cloud Registration in the Wild
Turcan Tuna, Julian Nubert, Patrick Pfreundschuh +3
The ICP registration algorithm has been a preferred method for LiDAR-based robot localization for nearly a decade. However, even in modern SLAM solutions, ICP can degrade and becom…
Empty Cities: Image Inpainting for a Dynamic-Object-Invariant Space
Berta Bescos, José Neira, Roland Siegwart +1
In this paper we present an end-to-end deep learning framework to turn images that show dynamic content, such as vehicles or pedestrians, into realistic static frames. This objecti…
Fast Traversability Estimation for Wild Visual Navigation
Jonas Frey, Matias Mattamala, Nived Chebrolu +3
Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes. In…
Holistic Fusion: Task- and Setup-Agnostic Robot Localization and State Estimation with Factor Graphs
Julian Nubert, Turcan Tuna, Jonas Frey +4
Seamless operation of mobile robots in challenging environments requires low-latency local motion estimation and accurate global localization. While most sensor-fusion approaches a…
Efficient Hierarchical Any-Angle Path Planning on Multi-Resolution 3D Grids
Victor Reijgwart, Cesar Cadena, Roland Siegwart +1
Hierarchical, multi-resolution volumetric mapping approaches are widely used to represent large and complex environments as they can efficiently capture their occupancy and connect…
Depth Based Semantic Scene Completion with Position Importance Aware Loss
Yu Liu, Jie Li, Xia Yuan +4
Semantic Scene Completion (SSC) refers to the task of inferring the 3D semantic segmentation of a scene while simultaneously completing the 3D shapes. We propose PALNet, a novel hy…
TartanGround: A Large-Scale Dataset for Ground Robot Perception and Navigation
Manthan Patel, Fan Yang, Yuheng Qiu +4
We present TartanGround, a large-scale, multi-modal dataset to advance the perception and autonomy of ground robots operating in diverse environments. This dataset, collected in va…
VIZARD: Reliable Visual Localization for Autonomous Vehicles in Urban Outdoor Environments
Mathias Bürki, Lukas Schaupp, Marcin Dymczyk +4
Changes in appearance is one of the main sources of failure in visual localization systems in outdoor environments. To address this challenge, we present VIZARD, a visual localizat…
Mesh Manifold based Riemannian Motion Planning for Omnidirectional Micro Aerial Vehicles
Michael Pantic, Lionel Ott, Cesar Cadena +2
This paper presents a novel on-line path planning method that enables aerial robots to interact with surfaces. We present a solution to the problem of finding trajectories that dri…
SAHA: Supervised Autonomous HArvester for selective forest thinning
Fang Nan, Meher Malladi, Qingqing Li +7
Forestry plays a vital role in our society, creating significant ecological, economic, and recreational value. Efficient forest management involves labor-intensive and complex oper…
Sampling-free obstacle gradients and reactive planning in Neural Radiance Fields (NeRF)
Michael Pantic, Cesar Cadena, Roland Siegwart +1
This work investigates the use of Neural implicit representations, specifically Neural Radiance Fields (NeRF), for geometrical queries and motion planning. We show that by adding t…
Spatially-Enhanced Recurrent Memory for Long-Range Mapless Navigation via End-to-End Reinforcement Learning
Fan Yang, Per Frivik, David Hoeller +3
Recent advancements in robot navigation, particularly with end-to-end learning approaches such as reinforcement learning (RL), have demonstrated strong performance. However, succes…
Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
Cesar Cadena, Luca Carlone, Henry Carrillo +5
Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving wit…
Freetures: Localization in Signed Distance Function Maps
Alexander Millane, Helen Oleynikova, Christian Lanegger +5
Localization of a robotic system within a previously mapped environment is important for reducing estimation drift and for reusing previously built maps. Existing techniques for ge…
X-View: Graph-Based Semantic Multi-View Localization
Abel Gawel, Carlo Del Don, Roland Siegwart +2
Global registration of multi-view robot data is a challenging task. Appearance-based global localization approaches often fail under drastic view-point changes, as representations…
Superquadric Object Representation for Optimization-based Semantic SLAM
Florian Tschopp, Juan Nieto, Roland Siegwart +1
Introducing semantically meaningful objects to visual Simultaneous Localization And Mapping (SLAM) has the potential to improve both the accuracy and reliability of pose estimates,…
A Unified Approach for Autonomous Volumetric Exploration of Large Scale Environments under Severe Odometry Drift
Lukas Schmid, Victor Reijgwart, Lionel Ott +3
Exploration is a fundamental problem in robot autonomy. A major limitation, however, is that during exploration robots oftentimes have to rely on on-board systems alone for state e…
Experimental Comparison of Visual-Aided Odometry Methods for Rail Vehicles
Florian Tschopp, Thomas Schneider, Andrew W. Palmer +4
Today, rail vehicle localization is based on infrastructure-side Balises (beacons) together with on-board odometry to determine whether a rail segment is occupied. Such a coarse lo…
From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots
Mark Pfeiffer, Michael Schaeuble, Juan Nieto +2
Learning from demonstration for motion planning is an ongoing research topic. In this paper we present a model that is able to learn the complex mapping from raw 2D-laser range fin…
Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End
Jin Jin, Chong Zhang, Jonas Frey +4
Autonomous robots must navigate reliably in unknown environments even under compromised exteroceptive perception, or perception failures. Such failures often occur when harsh envir…
NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping
Stefan Lionar, Lukas Schmid, Cesar Cadena +2
We present a novel 3D mapping method leveraging the recent progress in neural implicit representation for 3D reconstruction. Most existing state-of-the-art neural implicit represen…
Sight Guide: A Wearable Assistive Perception and Navigation System for the Vision Assistance Race in the Cybathlon 2024
Patrick Pfreundschuh, Giovanni Cioffi, Cornelius von Einem +6
Visually impaired individuals face significant challenges navigating and interacting with unknown situations, particularly in tasks requiring spatial awareness and semantic scene u…
Local and Global Information in Obstacle Detection on Railway Tracks
Matthias Brucker, Andrei Cramariuc, Cornelius von Einem +2
Reliable obstacle detection on railways could help prevent collisions that result in injuries and potentially damage or derail the train. Unfortunately, generic object detectors do…
LIME: Learning Intent-aware Camera Motion from Egocentric Video
Boyang Sun, Jiajie Li, Yung-Hsu Yang +6
Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vis…
Semantic Landmark Particle Filter for Robot Localisation in Vineyards
Rajitha de Silva, Jonathan Cox, James R. Heselden +3
Reliable localisation in vineyards is hindered by row-level perceptual aliasing: parallel crop rows produce nearly identical LiDAR observations, causing geometry-only and vision-ba…
Continuous-Time State Estimation Methods in Robotics: A Survey
William Talbot, Julian Nubert, Turcan Tuna +5
Accurate, efficient, and robust state estimation is more important than ever in robotics as the variety of platforms and complexity of tasks continue to grow. Historically, discret…
3D Multi-Robot Patrolling with a Two-Level Coordination Strategy
Luigi Freda, Mario Gianni, Fiora Pirri +4
Teams of UGVs patrolling harsh and complex 3D environments can experience interference and spatial conflicts with one another. Neglecting the occurrence of these events crucially h…
Keypoint Semantic Integration for Improved Feature Matching in Outdoor Agricultural Environments
Rajitha de Silva, Jonathan Cox, Marija Popovic +3
Robust robot navigation in outdoor environments requires accurate perception systems capable of handling visual challenges such as repetitive structures and changing appearances. V…
Spherical Multi-Modal Place Recognition for Heterogeneous Sensor Systems
Lukas Bernreiter, Lionel Ott, Juan Nieto +2
In this paper, we propose a robust end-to-end multi-modal pipeline for place recognition where the sensor systems can differ from the map building to the query. Our approach operat…
Self-Improving Semantic Perception for Indoor Localisation
Hermann Blum, Francesco Milano, René Zurbrügg +3
We propose a novel robotic system that can improve its perception during deployment. Contrary to the established approach of learning semantics from large datasets and deploying fi…
PHASER: a Robust and Correspondence-free Global Pointcloud Registration
Lukas Bernreiter, Lionel Ott, Juan Nieto +2
We propose PHASER, a correspondence-free global registration of sensor-centric pointclouds that is robust to noise, sparsity, and partial overlaps. Our method can seamlessly handle…
ForestLPR: LiDAR Place Recognition in Forests Attentioning Multiple BEV Density Images
Yanqing Shen, Turcan Tuna, Marco Hutter +2
Place recognition is essential to maintain global consistency in large-scale localization systems. While research in urban environments has progressed significantly using LiDARs or…
3D Registration of Aerial and Ground Robots for Disaster Response: An Evaluation of Features, Descriptors, and Transformation Estimation
Abel Gawel, Renaud Dubé, Hartmut Surmann +3
Global registration of heterogeneous ground and aerial mapping data is a challenging task. This is especially difficult in disaster response scenarios when we have no prior informa…
MOZARD: Multi-Modal Localization for Autonomous Vehicles in Urban Outdoor Environments
Lukas Schaupp, Patrick Pfreundschuh, Mathias Buerki +3
Visually poor scenarios are one of the main sources of failure in visual localization systems in outdoor environments. To address this challenge, we present MOZARD, a multi-modal l…
NaviTrace: Evaluating Embodied Navigation of Vision-Language Models
Tim Windecker, Manthan Patel, Moritz Reuss +5
Vision-language models demonstrate unprecedented performance and generalization across a wide range of tasks and scenarios. Integrating these foundation models into robotic navigat…
Enhancing Robotic Precision in Construction: A Modular Factor Graph-Based Framework to Deflection and Backlash Compensation Using High-Accuracy Accelerometers
Julien Kindle, Michael Loetscher, Andrea Alessandretti +2
Accurate positioning is crucial in the construction industry, where labor shortages highlight the need for automation. Robotic systems with long kinematic chains are required to re…
Voxgraph: Globally Consistent, Volumetric Mapping using Signed Distance Function Submaps
Victor Reijgwart, Alexander Millane, Helen Oleynikova +3
Globally consistent dense maps are a key requirement for long-term robot navigation in complex environments. While previous works have addressed the challenges of dense mapping and…
Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspection
Richard Kuhlmann, Jakob Wolfram, Boyang Sun +4
Autonomous inspection is a central problem in robotics, with applications ranging from industrial monitoring to search-and-rescue. Traditionally, inspection has often been reduced…
Precise Robot Localization in Architectural 3D Plans
Hermann Blum, Julian Stiefel, Cesar Cadena +2
This paper presents a localization system for mobile robots enabling precise localization in inaccurate building models. The approach leverages local referencing to counteract inhe…
Modular Sensor Fusion for Semantic Segmentation
Hermann Blum, Abel Gawel, Roland Siegwart +1
Sensor fusion is a fundamental process in robotic systems as it extends the perceptual range and increases robustness in real-world operations. Current multi-sensor deep learning b…
FrontierNet: Learning Visual Cues to Explore
Boyang Sun, Hanzhi Chen, Stefan Leutenegger +3
Exploration of unknown environments is crucial for autonomous robots; it allows them to actively reason and decide on what new data to acquire for different tasks, such as mapping,…
Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrations
Mark Pfeiffer, Samarth Shukla, Matteo Turchetta +4
This work presents a case study of a learning-based approach for target driven map-less navigation. The underlying navigation model is an end-to-end neural network which is trained…
Team CERBERUS Wins the DARPA Subterranean Challenge: Technical Overview and Lessons Learned
Marco Tranzatto, Mihir Dharmadhikari, Lukas Bernreiter +33
This article presents the CERBERUS robotic system-of-systems, which won the DARPA Subterranean Challenge Final Event in 2021. The Subterranean Challenge was organized by DARPA with…
See Yourself in Others: Attending Multiple Tasks for Own Failure Detection
Boyang Sun, Jiaxu Xing, Hermann Blum +2
Autonomous robots deal with unexpected scenarios in real environments. Given input images, various visual perception tasks can be performed, e.g., semantic segmentation, depth esti…
CalQNet -- Detection of Calibration Quality for Life-Long Stereo Camera Setups
Jiapeng Zhong, Zheyu Ye, Andrei Cramariuc +4
Many mobile robotic platforms rely on an accurate knowledge of the extrinsic calibration parameters, especially systems performing visual stereo matching. Although a number of accu…
Pixel-wise Anomaly Detection in Complex Driving Scenes
Giancarlo Di Biase, Hermann Blum, Roland Siegwart +1
The inability of state-of-the-art semantic segmentation methods to detect anomaly instances hinders them from being deployed in safety-critical and complex applications, such as au…
Free LSD: Prior-Free Visual Landing Site Detection for Autonomous Planes
Timo Hinzmann, Thomas Stastny, Cesar Cadena +2
Full autonomy for fixed-wing unmanned aerial vehicles (UAVs) requires the capability to autonomously detect potential landing sites in unknown and unstructured terrain, allowing fo…
GrandTour: A Legged Robotics Dataset in the Wild for Multi-Modal Perception and State Estimation
Turcan Tuna, Jonas Frey, Frank Fu +5
Accurate state estimation and multi-modal perception are prerequisites for autonomous legged robots in complex, large-scale environments. To date, no large-scale public legged-robo…
Observability-aware Self-Calibration of Visual and Inertial Sensors for Ego-Motion Estimation
Thomas Schneider, Mingyang Li, Cesar Cadena +2
External effects such as shocks and temperature variations affect the calibration of visual-inertial sensor systems and thus they cannot fully rely on factory calibrations. Re-cali…
DeFM: Learning Foundation Representations from Depth for Robotics
Manthan Patel, Jonas Frey, Mayank Mittal +5
Depth sensors are widely deployed across robotic platforms, and advances in fast, high-fidelity depth simulation have enabled robotic policies trained on depth observations to achi…
COIN-LIO: Complementary Intensity-Augmented LiDAR Inertial Odometry
Patrick Pfreundschuh, Helen Oleynikova, Cesar Cadena +2
We present COIN-LIO, a LiDAR Inertial Odometry pipeline that tightly couples information from LiDAR intensity with geometry-based point cloud registration. The focus of our work is…
A Framework for Collaborative Multi-Robot Mapping using Spectral Graph Wavelets
Lukas Bernreiter, Shehryar Khattak, Lionel Ott +3
The exploration of large-scale unknown environments can benefit from the deployment of multiple robots for collaborative mapping. Each robot explores a section of the environment a…
Boxi: Design Decisions in the Context of Algorithmic Performance for Robotics
Jonas Frey, Turcan Tuna, Lanke Frank Tarimo Fu +8
Achieving robust autonomy in mobile robots operating in complex and unstructured environments requires a multimodal sensor suite capable of capturing diverse and complementary info…
3D Gated Recurrent Fusion for Semantic Scene Completion
Yu Liu, Jie Li, Qingsen Yan +4
This paper tackles the problem of data fusion in the semantic scene completion (SSC) task, which can simultaneously deal with semantic labeling and scene completion. RGB images con…
CaRLi-V: Camera-RADAR-LiDAR Point-Wise 3D Velocity Estimation
Landson Guo, Andres M. Diaz Aguilar, William Talbot +3
Accurate point-wise velocity estimation in 3D is crucial for robot interaction with non-rigid dynamic agents, enabling robust performance in path planning, collision avoidance, and…
Event-based Civil Infrastructure Visual Defect Detection: ev-CIVIL Dataset and Benchmark
Udayanga G. W. K. N. Gamage, Xuanni Huo, Luca Zanatta +4
Small unmanned aerial vehicle (UAV)-based visual inspections are a more efficient alternative to manual methods for examining civil structural defects, offering safe access to haza…
DigiForest: Digital Analytics and Robotics for Sustainable Forestry
Marco Camurri, Enrico Tomelleri, MatÃas Mattamala +18
Covering one third of Earth's land surface, forests are vital to global biodiversity, climate regulation, and human well-being. In Europe, forests and woodlands reach approximately…
Large-Scale Autonomous Gas Monitoring for Volcanic Environments: A Legged Robot on Mount Etna
Julia Richter, Turcan Tuna, Manthan Patel +6
Volcanic gas emissions are key precursors of eruptive activity. Yet, obtaining accurate near-surface measurements remains hazardous and logistically challenging, motivating the nee…
Semantic-Aware Particle Filter for Reliable Vineyard Robot Localisation
Rajitha de Silva, Jonathan Cox, James R. Heselden +3
Accurate localisation is critical for mobile robots in structured outdoor environments, yet LiDAR-based methods often fail in vineyards due to repetitive row geometry and perceptua…
VersaVIS: An Open Versatile Multi-Camera Visual-Inertial Sensor Suite
Florian Tschopp, Michael Riner, Marius Fehr +7
Robust and accurate pose estimation is crucial for many applications in mobile robotics. Extending visual Simultaneous Localization and Mapping (SLAM) with other modalities such as…
This is not what I imagined: Error Detection for Semantic Segmentation through Visual Dissimilarity
David Haldimann, Hermann Blum, Roland Siegwart +1
There has been a remarkable progress in the accuracy of semantic segmentation due to the capabilities of deep learning. Unfortunately, these methods are not able to generalize much…
Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
Margarita Grinvald, Fadri Furrer, Tonci Novkovic +4
To autonomously navigate and plan interactions in real-world environments, robots require the ability to robustly perceive and map complex, unstructured surrounding scenes. Besides…
SphNet: A Spherical Network for Semantic Pointcloud Segmentation
Lukas Bernreiter, Lionel Ott, Roland Siegwart +1
Semantic segmentation for robotic systems can enable a wide range of applications, from self-driving cars and augmented reality systems to domestic robots. We argue that a spherica…
Tripody: An Overconstrained 3-SPR-like Parallel Robot for High-Reach Construction Tasks
Julien Kindle, Jakub Raczy, Riccardo Balbi +3
Many ceiling construction tasks still rely on heavy serial manipulators that are difficult to deploy in cluttered interiors, motivating lightweight, field-ready alternatives that r…
Robots Need More than VLA and World Models
Elis Karcini, Faisal Mehrban, Quang Nguyen +6
Generalist robot intelligence is often framed as a policy-scaling problem: collect more robot demonstrations, train larger Vision-Language-Action (VLA) models, and expect broader g…
SCIM: Simultaneous Clustering, Inference, and Mapping for Open-World Semantic Scene Understanding
Hermann Blum, Marcus G. Müller, Abel Gawel +2
In order to operate in human environments, a robot's semantic perception has to overcome open-world challenges such as novel objects and domain gaps. Autonomous deployment to such…
Unsupervised Continual Semantic Adaptation through Neural Rendering
Zhizheng Liu, Francesco Milano, Jonas Frey +3
An increasing amount of applications rely on data-driven models that are deployed for perception tasks across a sequence of scenes. Due to the mismatch between training and deploym…
Predicting Unobserved Space For Planning via Depth Map Augmentation
Marius Fehr, Tim Taubner, Yang Liu +2
Safe and efficient path planning is crucial for autonomous mobile robots. A prerequisite for path planning is to have a comprehensive understanding of the 3D structure of the robot…
Descriptellation: Deep Learned Constellation Descriptors
Chunwei Xing, Xinyu Sun, Andrei Cramariuc +5
Current descriptors for global localization often struggle under vast viewpoint or appearance changes. One possible improvement is the addition of topological information on semant…
Learning Multi-Agent Local Collision-Avoidance for Collaborative Carrying tasks with Coupled Quadrupedal Robots
Francesca Bray, Simone Tolomei, Andrei Cramariuc +2
Robotic collaborative carrying could greatly benefit human activities like warehouse and construction site management. However, coordinating the simultaneous motion of multiple rob…
Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization
Paul-Edouard Sarlin, Frédéric Debraine, Marcin Dymczyk +2
Many robotics applications require precise pose estimates despite operating in large and changing environments. This can be addressed by visual localization, using a pre-computed 3…
Map Management for Efficient Long-Term Visual Localization in Outdoor Environments
Mathias Bürki, Marcin Dymczyk, Igor Gilitschenski +3
We present a complete map management process for a visual localization system designed for multi-vehicle long- term operations in resource constrained outdoor environments. Outdoor…
SC-Explorer: Incremental 3D Scene Completion for Safe and Efficient Exploration Mapping and Planning
Lukas Schmid, Mansoor Nasir Cheema, Victor Reijgwart +3
Exploration of unknown environments is a fundamental problem in robotics and an essential component in numerous applications of autonomous systems. A major challenge in exploring u…
Learning Trajectories for Visual-Inertial System Calibration via Model-based Heuristic Deep Reinforcement Learning
Le Chen, Yunke Ao, Florian Tschopp +5
Visual-inertial systems rely on precise calibrations of both camera intrinsics and inter-sensor extrinsics, which typically require manually performing complex motions in front of…
Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Models
Mike Zhang, Kaixian Qu, Vaishakh Patil +2
Large Language Models (LLM) have emerged as a tool for robots to generate task plans using common sense reasoning. For the LLM to generate actionable plans, scene context must be p…
Path-Constrained State Estimation for Rail Vehicles
Cornelius von Einem, Andrei Cramariuc, Roland Siegwart +2
Globally rising demand for transportation by rail is pushing existing infrastructure to its capacity limits, necessitating the development of accurate, robust, and high-frequency p…
VIRUS-NeRF -- Vision, InfraRed and UltraSonic based Neural Radiance Fields
Nicolaj Schmid, Cornelius von Einem, Cesar Cadena +3
Autonomous mobile robots are an increasingly integral part of modern factory and warehouse operations. Obstacle detection, avoidance and path planning are critical safety-relevant…
ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning
Christopher E. Mower, Yuhui Wan, Hongzhan Yu +20
We present a framework for intuitive robot programming by non-experts, leveraging natural language prompts and contextual information from the Robot Operating System (ROS). Our sys…
Real-Time Frame- and Event-based Object Detection with Spiking Neural Networks on Edge Neuromorphic Hardware: Design, Deployment and Benchmark
Udayanga G. W. K. N. Gamage, Yan Zeng, Cesar Cadena +2
Real-time object detection on energy-constrained platforms is critical for applications such as UAV-based inspection, autonomous navigation, and mobile robotics. Spiking neural net…
Path-conditioned Reinforcement Learning-based Local Planning for Long-Range Navigation
Mateo Haro, Julia Richter, Fan Yang +2
Long-range navigation is commonly addressed through hierarchical pipelines in which a global planner generates a path, decomposed into waypoints, and followed sequentially by a loc…
MultiViPerFrOG: A Globally Optimized Multi-Viewpoint Perception Framework for Camera Motion and Tissue Deformation
Guido Caccianiga, Julian Nubert, Cesar Cadena +2
Reconstructing the 3D shape of a deformable environment from the information captured by a moving depth camera is highly relevant to surgery. The underlying challenge is the fact t…
Deep Learning-based Human Detection for UAVs with Optical and Infrared Cameras: System and Experiments
Timo Hinzmann, Tobias Stegemann, Cesar Cadena +1
In this paper, we present our deep learning-based human detection system that uses optical (RGB) and long-wave infrared (LWIR) cameras to detect, track, localize, and re-identify h…
An Efficient Beam Search Algorithm for Active Perception in Mobile Robotics
Kaixian Qu, Han Wang, Victor Klemm +2
Active perception is a fundamental problem in autonomous robotics in which the robot must decide where to move and what to sense in order to obtain the most informative observation…
SegMap: 3D Segment Mapping using Data-Driven Descriptors
Renaud Dubé, Andrei Cramariuc, Daniel Dugas +3
When performing localization and mapping, working at the level of structure can be advantageous in terms of robustness to environmental changes and differences in illumination. Thi…
The Hidden Uncertainty in a Neural Networks Activations
Janis Postels, Hermann Blum, Yannick Strümpler +4
The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution…
Flexible Trinocular: Non-rigid Multi-Camera-IMU Dense Reconstruction for UAV Navigation and Mapping
Timo Hinzmann, Cesar Cadena, Juan Nieto +1
In this paper, we propose a visual-inertial framework able to efficiently estimate the camera poses of a non-rigid trinocular baseline for long-range depth estimation on-board a fa…
Learning Densities in Feature Space for Reliable Segmentation of Indoor Scenes
Nicolas Marchal, Charlotte Moraldo, Roland Siegwart +3
Deep learning has enabled remarkable advances in scene understanding, particularly in semantic segmentation tasks. Yet, current state of the art approaches are limited to a closed…
Learning Camera Miscalibration Detection
Andrei Cramariuc, Aleksandar Petrov, Rohit Suri +3
Self-diagnosis and self-repair are some of the key challenges in deploying robotic platforms for long-term real-world applications. One of the issues that can occur to a robot is m…
Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision
MatÃas Mattamala, Jonas Frey, Piotr Libera +5
Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes. In…
Seeing Through the Grass: Semantic Pointcloud Filter for Support Surface Learning
Anqiao Li, Chenyu Yang, Jonas Frey +3
Mobile ground robots require perceiving and understanding their surrounding support surface to move around autonomously and safely. The support surface is commonly estimated based…
Continual Adaptation of Semantic Segmentation using Complementary 2D-3D Data Representations
Jonas Frey, Hermann Blum, Francesco Milano +2
Semantic segmentation networks are usually pre-trained once and not updated during deployment. As a consequence, misclassifications commonly occur if the distribution of the traini…
Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency
Lukas Schmid, Jeffrey Delmerico, Johannes Schönberger +4
For robotic interaction in environments shared with other agents, access to volumetric and semantic maps of the scene is crucial. However, such environments are inevitably subject…
iPlanner: Imperative Path Planning
Fan Yang, Chen Wang, Cesar Cadena +1
The problem of path planning has been studied for years. Classic planning pipelines, including perception, mapping, and path searching, can result in latency and compounding errors…
Multiple Hypothesis Semantic Mapping for Robust Data Association
Lukas Bernreiter, Abel Gawel, Hannes Sommer +3
In this paper, we present a semantic mapping approach with multiple hypothesis tracking for data association. As semantic information has the potential to overcome ambiguity in mea…
Collaborative Robot Mapping using Spectral Graph Analysis
Lukas Bernreiter, Shehryar Khattak, Lionel Ott +3
In this paper, we deal with the problem of creating globally consistent pose graphs in a centralized multi-robot SLAM framework. For each robot to act autonomously, individual onbo…
3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs
Samuel Looper, Javier Rodriguez-Puigvert, Roland Siegwart +2
Numerous applications require robots to operate in environments shared with other agents, such as humans or other robots. However, such shared scenes are typically subject to diffe…
maplab 2.0 -- A Modular and Multi-Modal Mapping Framework
Andrei Cramariuc, Lukas Bernreiter, Florian Tschopp +5
Integration of multiple sensor modalities and deep learning into Simultaneous Localization And Mapping (SLAM) systems are areas of significant interest in current research. Multi-m…
Embodied Active Domain Adaptation for Semantic Segmentation via Informative Path Planning
René Zurbrügg, Hermann Blum, Cesar Cadena +2
This work presents an embodied agent that can adapt its semantic segmentation network to new indoor environments in a fully autonomous way. Because semantic segmentation networks f…
SegMap: Segment-based mapping and localization using data-driven descriptors
Renaud Dubé, Andrei Cramariuc, Daniel Dugas +5
Precisely estimating a robot's pose in a prior, global map is a fundamental capability for mobile robotics, e.g. autonomous driving or exploration in disaster zones. This task, how…
Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation
Pascal Roth, Jonas Frey, Cesar Cadena +1
Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot`s capab…