Publications (223)
Learning Agile, Vision-based Drone Flight: from Simulation to Reality
Davide Scaramuzza, Elia Kaufmann
We present our latest research in learning deep sensorimotor policies for agile, vision-based quadrotor flight. We show methodologies for the successful transfer of such policies f…
Learning Quadrotor Control From Visual Features Using Differentiable Simulation
Johannes Heeg, Yunlong Song, Davide Scaramuzza
The sample inefficiency of reinforcement learning (RL) remains a significant challenge in robotics. RL requires large-scale simulation and can still cause long training times, slow…
Reducing the Sim-to-Real Gap for Event Cameras
Timo Stoffregen, Cedric Scheerlinck, Davide Scaramuzza +4
Event cameras are paradigm-shifting novel sensors that report asynchronous, per-pixel brightness changes called 'events' with unparalleled low latency. This makes them ideal for hi…
AirSim Drone Racing Lab
Ratnesh Madaan, Nicholas Gyde, Sai Vemprala +7
Autonomous drone racing is a challenging research problem at the intersection of computer vision, planning, state estimation, and control. We introduce AirSim Drone Racing Lab, a s…
Why does Deep Learning Improve Visual SLAM?
Giovanni Cioffi, Davide Scaramuzza
Visual SLAM is a well-established technology utilized in a wide range of real-world applications. However, its performance still degrades under challenging visual conditions, such…
Primal-Dual Mesh Convolutional Neural Networks
Francesco Milano, Antonio Loquercio, Antoni Rosinol +2
Recent works in geometric deep learning have introduced neural networks that allow performing inference tasks on three-dimensional geometric data by defining convolution, and somet…
State Space Models for Event Cameras
Nikola ZubiÄ, Mathias Gehrig, Davide Scaramuzza
Today, state-of-the-art deep neural networks that process event-camera data first convert a temporal window of events into dense, grid-like input representations. As such, they exh…
From Chaos Comes Order: Ordering Event Representations for Object Recognition and Detection
Nikola ZubiÄ, Daniel Gehrig, Mathias Gehrig +1
Today, state-of-the-art deep neural networks that process events first convert them into dense, grid-like input representations before using an off-the-shelf network. However, sele…
High Speed and High Dynamic Range Video with an Event Camera
Henri Rebecq, René Ranftl, Vladlen Koltun +1
Event cameras are novel sensors that report brightness changes in the form of a stream of asynchronous "events" instead of intensity frames. They offer significant advantages with…
Perturbed State Space Feature Encoders for Optical Flow with Event Cameras
Gokul Raju Govinda Raju, Nikola ZubiÄ, Marco Cannici +1
With their motion-responsive nature, event-based cameras offer significant advantages over traditional cameras for optical flow estimation. While deep learning has improved upon tr…
Autonomous Drone Racing: A Survey
Drew Hanover, Antonio Loquercio, Leonard Bauersfeld +6
Over the last decade, the use of autonomous drone systems for surveying, search and rescue, or last-mile delivery has increased exponentially. With the rise of these applications c…
Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
Cesar Cadena, Luca Carlone, Henry Carrillo +5
Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving wit…
The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
Elie Aljalbout, Jiaxu Xing, Angel Romero +9
Machine learning has facilitated significant advancements across various robotics domains, including navigation, locomotion, and manipulation. Many such achievements have been driv…
Continuous-Time vs. Discrete-Time Vision-based SLAM: A Comparative Study
Giovanni Cioffi, Titus Cieslewski, Davide Scaramuzza
Robotic practitioners generally approach the vision-based SLAM problem through discrete-time formulations. This has the advantage of a consolidated theory and very good understandi…
A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight
Elia Kaufmann, Leonard Bauersfeld, Davide Scaramuzza
Quadrotors are highly nonlinear dynamical systems that require carefully tuned controllers to be pushed to their physical limits. Recently, learning-based control policies have bee…
Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning
Yunlong Song, HaoChih Lin, Elia Kaufmann +2
Professional race-car drivers can execute extreme overtaking maneuvers. However, existing algorithms for autonomous overtaking either rely on simplified assumptions about the vehic…
Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms
Tim Salzmann, Elia Kaufmann, Jon Arrizabalaga +3
Model Predictive Control (MPC) has become a popular framework in embedded control for high-performance autonomous systems. However, to achieve good control performance using MPC, a…
A Monocular Event-Camera Motion Capture System
Leonard Bauersfeld, Davide Scaramuzza
Motion capture systems are a widespread tool in research to record ground-truth poses of objects. Commercial systems use reflective markers attached to the object and then triangul…
Microgravity induces overconfidence in perceptual decision-making
Leyla Loued-Khenissi, Christian Pfeiffer, Rupal Saxena +2
Does gravity affect decision-making? This question comes into sharp focus as plans for interplanetary human space missions solidify. In the framework of Bayesian brain theories, gr…
Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight
Angel Romero, Ashwin Shenai, Ismail Geles +2
Autonomous drone racing has risen as a challenging robotic benchmark for testing the limits of learning, perception, planning, and control. Expert human pilots are able to fly a dr…
Neural Events: Discrete Asynchronous Autoencoders for Event-Based Vision
Roberto Pellerito, Daniel Gehrig, Shintaro Shiba +1
Event cameras capture dynamic scenes with exceptional temporal fidelity by representing them as a continuous stream of microsecond resolution \textit{events}. Each individual event…
SIPs: Succinct Interest Points from Unsupervised Inlierness Probability Learning
Titus Cieslewski, Konstantinos G. Derpanis, Davide Scaramuzza
A wide range of computer vision algorithms rely on identifying sparse interest points in images and establishing correspondences between them. However, only a subset of the initial…
Continual Robot Policy Learning via Variational Neural Dynamics
Jiaxu Xing, Zhiyuan Zhu, Yunfan Ren +4
Robots deployed in the real world rarely operate under a single fixed dynamics model: wind changes, payloads vary, batteries drain, contacts shift, and hardware wears. Yet most lea…
Learning Depth With Very Sparse Supervision
Antonio Loquercio, Alexey Dosovitskiy, Davide Scaramuzza
Motivated by the astonishing capabilities of natural intelligent agents and inspired by theories from psychology, this paper explores the idea that perception gets coupled to 3D pr…
Semi-Dense 3D Reconstruction with a Stereo Event Camera
Yi Zhou, Guillermo Gallego, Henri Rebecq +3
Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision. Th…
ForesightNav: Learning Scene Imagination for Efficient Exploration
Hardik Shah, Jiaxu Xing, Nico Messikommer +3
Understanding how humans leverage prior knowledge to navigate unseen environments while making exploratory decisions is essential for developing autonomous robots with similar abil…
Perception-Aware Perching on Powerlines with Multirotors
Julio L. Paneque, Jose Ramiro MartÃnez de Dios, AnÃbal Ollero. Drew Hanover +3
Multirotor aerial robots are becoming widely used for the inspection of powerlines. To enable continuous, robust inspection without human intervention, the robots must be able to p…
Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction
Ivan Alberico, Marco Cannici, Giovanni Cioffi +1
In this paper, we present a real-time egocentric trajectory prediction system for table tennis using event cameras. Unlike standard cameras, which suffer from high latency and moti…
Humanity's Last Exam
Long Phan, Alice Gatti, Ziwen Han +1144
Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achi…
ESS: Learning Event-based Semantic Segmentation from Still Images
Zhaoning Sun, Nico Messikommer, Daniel Gehrig +1
Retrieving accurate semantic information in challenging high dynamic range (HDR) and high-speed conditions remains an open challenge for image-based algorithms due to severe image…
A Real-Time Game Theoretic Planner for Autonomous Two-Player Drone Racing
Riccardo Spica, Davide Falanga, Eric Cristofalo +3
To be successful in multi-player drone racing, a player must not only follow the race track in an optimal way, but also compete with other drones through strategic blocking, faking…
GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control
Mariam Hassan, Sebastian Stapf, Ahmad Rahimi +17
We present GEM, a Generalizable Ego-vision Multimodal world model that predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories. Hence, ou…
Visual-Inertial Odometry of Aerial Robots
Davide Scaramuzza, Zichao Zhang
Visual-Inertial odometry (VIO) is the process of estimating the state (pose and velocity) of an agent (e.g., an aerial robot) by using only the input of one or more cameras plus on…
Faster than FAST: GPU-Accelerated Frontend for High-Speed VIO
Balazs Nagy, Philipp Foehn, Davide Scaramuzza
The recent introduction of powerful embedded graphics processing units (GPUs) has allowed for unforeseen improvements in real-time computer vision applications. It has enabled algo…
Reinforcement Learning Meets Visual Odometry
Nico Messikommer, Giovanni Cioffi, Mathias Gehrig +1
Visual Odometry (VO) is essential to downstream mobile robotics and augmented/virtual reality tasks. Despite recent advances, existing VO methods still rely on heuristic design cho…
DSEC: A Stereo Event Camera Dataset for Driving Scenarios
Mathias Gehrig, Willem Aarents, Daniel Gehrig +1
Once an academic venture, autonomous driving has received unparalleled corporate funding in the last decade. Still, the operating conditions of current autonomous cars are mostly r…
Unsupervised Moving Object Detection via Contextual Information Separation
Yanchao Yang, Antonio Loquercio, Davide Scaramuzza +1
We propose an adversarial contextual model for detecting moving objects in images. A deep neural network is trained to predict the optical flow in a region using information from e…
Minimum-Time Quadrotor Waypoint Flight in Cluttered Environments
Robert Penicka, Davide Scaramuzza
We tackle the problem of planning a minimum-time trajectory for a quadrotor over a sequence of specified waypoints in the presence of obstacles while exploiting the full quadrotor…
Pushing the Limits of Asynchronous Graph-based Object Detection with Event Cameras
Daniel Gehrig, Davide Scaramuzza
State-of-the-art machine-learning methods for event cameras treat events as dense representations and process them with conventional deep neural networks. Thus, they fail to mainta…
Learning High-Level Policies for Model Predictive Control
Yunlong Song, Davide Scaramuzza
The combination of policy search and deep neural networks holds the promise of automating a variety of decision-making tasks. Model Predictive Control (MPC) provides robust solutio…
Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspection
Richard Kuhlmann, Jakob Wolfram, Boyang Sun +4
Autonomous inspection is a central problem in robotics, with applications ranging from industrial monitoring to search-and-rescue. Traditionally, inspection has often been reduced…
A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild
Lei Sun, Daniel Gehrig, Christos Sakaridis +7
Effective video frame interpolation hinges on the adept handling of motion in the input scene. Prior work acknowledges asynchronous event information for this, but often overlooks…
Video to Events: Recycling Video Datasets for Event Cameras
Daniel Gehrig, Mathias Gehrig, Javier Hidalgo-Carrió +1
Event cameras are novel sensors that output brightness changes in the form of a stream of asynchronous "events" instead of intensity frames. They offer significant advantages with…
Event-Based Angular Velocity Regression with Spiking Networks
Mathias Gehrig, Sumit Bam Shrestha, Daniel Mouritzen +1
Spiking Neural Networks (SNNs) are bio-inspired networks that process information conveyed as temporal spikes rather than numeric values. A spiking neuron of an SNN only produces a…
Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotor
Anish Bhattacharya, Marco Cannici, Nishanth Rao +4
We present the first static-obstacle avoidance method for quadrotors using just an onboard, monocular event camera. Quadrotors are capable of fast and agile flight in cluttered env…
LightPanel: Active Mobile Platform for Dense 3D Modelling
Jonas Schuler, Reza Sabzevari, Davide Scaramuzza
In this paper we introduce a novel platform for dense 3D modelling. This platform is an active image acquisition setup assisted with a set of light sources and a distance sensor. T…
Temporal Cascading of Planning and Control for Quadrotor MPC
Rudolf Reiter, Chao Qin, Leonard Bauersfeld +1
The paper introduces UNIQUE, a model predictive control framework that temporally cascades planning and control for quadrotors, integrating high‑ and low‑fidelity models within a s…
Human-Piloted Drone Racing: Visual Processing and Control
Christian Pfeiffer, Davide Scaramuzza
Humans race drones faster than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A bett…
A 5-Point Minimal Solver for Event Camera Relative Motion Estimation
Ling Gao, Hang Su, Daniel Gehrig +3
Event-based cameras are ideal for line-based motion estimation, since they predominantly respond to edges in the scene. However, accurately determining the camera displacement base…
Rethinking Trajectory Evaluation for SLAM: a Probabilistic, Continuous-Time Approach
Zichao Zhang, Davide Scaramuzza
Despite the existence of different error metrics for trajectory evaluation in SLAM, their theoretical justifications and connections are rarely studied, and few methods handle temp…
Time-Optimal Planning for Quadrotor Waypoint Flight
Philipp Foehn, Angel Romero, Davide Scaramuzza
Quadrotors are among the most agile flying robots. However, planning time-optimal trajectories at the actuation limit through multiple waypoints remains an open problem. This is cr…
PL-SLAM: a Stereo SLAM System through the Combination of Points and Line Segments
Ruben Gomez-Ojeda, David Zuñiga-Noël, Francisco-Angel Moreno +2
Traditional approaches to stereo visual SLAM rely on point features to estimate the camera trajectory and build a map of the environment. In low-textured environments, though, it i…
Exploring Event Camera-based Odometry for Planetary Robots
Florian Mahlknecht, Daniel Gehrig, Jeremy Nash +4
Due to their resilience to motion blur and high robustness in low-light and high dynamic range conditions, event cameras are poised to become enabling sensors for vision-based expl…
Performance, Precision, and Payloads: Adaptive Nonlinear MPC for Quadrotors
Drew Hanover, Philipp Foehn, Sihao Sun +2
Agile quadrotor flight in challenging environments has the potential to revolutionize shipping, transportation, and search and rescue applications. Nonlinear model predictive contr…
Event-based SLAM Benchmark for High-Speed Maneuvers
Sheng Zhong, Junkai Niu, Guillermo Gallego +7
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to ha…
Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars
Ana I. Maqueda, Antonio Loquercio, Guillermo Gallego +2
Event cameras are bio-inspired vision sensors that naturally capture the dynamics of a scene, filtering out redundant information. This paper presents a deep neural network approac…
Event-based Camera Pose Tracking using a Generative Event Model
Guillermo Gallego, Christian Forster, Elias Mueggler +1
Event-based vision sensors mimic the operation of biological retina and they represent a major paradigm shift from traditional cameras. Instead of providing frames of intensity mea…
Event-Aided Sharp Radiance Field Reconstruction for Fast-Flying Drones
Rong Zou, Marco Cannici, Davide Scaramuzza
Fast-flying aerial robots promise rapid inspection under limited battery constraints, with direct applications in infrastructure inspection, terrain exploration, and search and res…
Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features
Pietro Bonazzi, Marie-Julie Rakatosaona, Marco Cannici +2
Existing deep learning methods for the reconstruction and denoising of point clouds rely on small datasets of 3D shapes. We circumvent the problem by leveraging deep learning metho…
Learned Inertial Odometry for Autonomous Drone Racing
Giovanni Cioffi, Leonard Bauersfeld, Elia Kaufmann +1
Inertial odometry is an attractive solution to the problem of state estimation for agile quadrotor flight. It is inexpensive, lightweight, and it is not affected by perceptual degr…
Perception-Aware Time-Optimal Planning for Quadrotor Waypoint Flight
Chao Qin, Jiaxu Xing, Rudolf Reiter +4
Agile quadrotor flight pushes the limits of control, actuation, and onboard perception. While time-optimal trajectory planning has been extensively studied, existing approaches typ…
Asynchronous, Photometric Feature Tracking using Events and Frames
Daniel Gehrig, Henri Rebecq, Guillermo Gallego +1
We present a method that leverages the complementarity of event cameras and standard cameras to track visual features with low-latency. Event cameras are novel sensors that output…
Learning Acrobatic Flight from Preferences
Colin Merk, Ismail Geles, Jiaxu Xing +3
Preference-based reinforcement learning (PbRL) enables agents to learn control policies without requiring manually designed reward functions, making it well-suited for tasks where…
Revisiting Token Pruning for Object Detection and Instance Segmentation
Yifei Liu, Mathias Gehrig, Nico Messikommer +2
Vision Transformers (ViTs) have shown impressive performance in computer vision, but their high computational cost, quadratic in the number of tokens, limits their adoption in comp…
Deep Drone Racing: From Simulation to Reality with Domain Randomization
Antonio Loquercio, Elia Kaufmann, René Ranftl +3
Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges that limit the deployment of small autono…
A roadmap for AI in robotics
Aude Billard, Alin Albu-Schaeffer, Michael Beetz +8
AI technologies, including deep learning, large-language models have gone from one breakthrough to the other. As a result, we are witnessing growing excitement in robotics at the p…
Registration made easy -- standalone orthopedic navigation with HoloLens
Florentin Liebmann, Simon Roner, Marco von Atzigen +8
In surgical navigation, finding correspondence between preoperative plan and intraoperative anatomy, the so-called registration task, is imperative. One promising approach is to in…
S7: Selective and Simplified State Space Layers for Sequence Modeling
Taylan Soydan, Nikola ZubiÄ, Nico Messikommer +2
A central challenge in sequence modeling is efficiently handling tasks with extended contexts. While recent state-space models (SSMs) have made significant progress in this area, t…
Redesigning SLAM for Arbitrary Multi-Camera Systems
Juichung Kuo, Manasi Muglikar, Zichao Zhang +1
Adding more cameras to SLAM systems improves robustness and accuracy but complicates the design of the visual front-end significantly. Thus, most systems in the literature are tail…
FaVoR: Features via Voxel Rendering for Camera Relocalization
Vincenzo Polizzi, Marco Cannici, Davide Scaramuzza +1
Camera relocalization methods range from dense image alignment to direct camera pose regression from a query image. Among these, sparse feature matching stands out as an efficient,…
Towards Low-Latency High-Bandwidth Control of Quadrotors using Event Cameras
Rika Sugimoto Dimitrova, Mathias Gehrig, Dario Brescianini +1
Event cameras are a promising candidate to enable high speed vision-based control due to their low sensor latency and high temporal resolution. However, purely event-based feedback…
A General Framework for Uncertainty Estimation in Deep Learning
Antonio Loquercio, Mattia Segù, Davide Scaramuzza
Neural networks predictions are unreliable when the input sample is out of the training distribution or corrupted by noise. Being able to detect such failures automatically is fund…
Time-Optimal Online Replanning for Agile Quadrotor Flight
Angel Romero, Robert Penicka, Davide Scaramuzza
In this paper, we tackle the problem of flying a quadrotor using time-optimal control policies that can be replanned online when the environment changes or when encountering unknow…
Learning to Throw: Agile and Accurate Cable-Suspended Payload Delivery with a Quadrotor
Yifan Zhai, Elia Raimondi, Yunfan Ren +4
Quadrotors offer the agility needed to rapidly transport suspended payloads during time-critical applications, including search-and-rescue and medical delivery. While suspended-pay…
3D scene generation from scene graphs and self-attention
Pietro Bonazzi, Mengqi Wang, Diego Martin Arroyo +4
Synthesizing realistic and diverse indoor 3D scene layouts in a controllable fashion opens up applications in simulated navigation and virtual reality. As concise and robust repres…
Reading in the Dark with Foveated Event Vision
Carl Brander, Giovanni Cioffi, Nico Messikommer +1
Current smart glasses equipped with RGB cameras struggle to perceive the environment in low-light and high-speed motion scenarios due to motion blur and the limited dynamic range o…
Bridging Performance and Generalization in Reinforcement Learning for Agile Flight
Jonathan Green, Jiaxu Xing, Nico Messikommer +2
Autonomous drone racing is a fundamentally challenging regime for autonomous aerial robots, requiring time-optimal control while operating under persistent actuation saturation. Wh…
Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
Nikola ZubiÄ, Davide Scaramuzza
Selective State-Space Models (SSMs) such as Mamba have become central to long-sequence modeling. Still, their stability is poorly understood: their state-space coefficients are mod…
Generative Event Pretraining with Foundation Model Alignment
Jianwen Cao, Jiaxu Xing, Nico Messikommer +1
Event cameras provide robust visual signals under fast motion and challenging illumination conditions thanks to their microsecond latency and high dynamic range. However, their uni…
Beauty and the Beast: Optimal Methods Meet Learning for Drone Racing
Elia Kaufmann, Mathias Gehrig, Philipp Foehn +4
Autonomous micro aerial vehicles still struggle with fast and agile maneuvers, dynamic environments, imperfect sensing, and state estimation drift. Autonomous drone racing brings t…
Event-Based Motion Segmentation by Motion Compensation
Timo Stoffregen, Guillermo Gallego, Tom Drummond +2
In contrast to traditional cameras, whose pixels have a common exposure time, event-based cameras are novel bio-inspired sensors whose pixels work independently and asynchronously…
Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation
Jing Yuan Luo, Yunlong Song, Victor Klemm +3
First-order Policy Gradient (FoPG) algorithms such as Backpropagation through Time and Analytical Policy Gradients leverage local simulation physics to accelerate policy search, si…
Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories
Matthias Faessler, Antonio Franchi, Davide Scaramuzza
In this paper, we prove that the dynamical model of a quadrotor subject to linear rotor drag effects is differentially flat in its position and heading. We use this property to com…
Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing
Jiawei Fu, Yunlong Song, Yan Wu +2
Autonomous drones can operate in remote and unstructured environments, enabling various real-world applications. However, the lack of effective vision-based algorithms has been a s…
Exploration Without Global Consistency Using Local Volume Consolidation
Titus Cieslewski, Andreas Ziegler, Davide Scaramuzza
In exploration, the goal is to build a map of an unknown environment. Most state-of-the-art approaches use map representations that require drift-free state estimates to function p…
Dense Continuous-Time Optical Flow from Events and Frames
Mathias Gehrig, Manasi Muglikar, Davide Scaramuzza
We present a method for estimating dense continuous-time optical flow from event data. Traditional dense optical flow methods compute the pixel displacement between two images. Due…
Aggressive Quadrotor Flight through Narrow Gaps with Onboard Sensing and Computing using Active Vision
Davide Falanga, Elias Mueggler, Matthias Faessler +1
We address one of the main challenges towards autonomous quadrotor flight in complex environments, which is flight through narrow gaps. While previous works relied on off-board loc…
Environment as Policy: Learning to Race in Unseen Tracks
Hongze Wang, Jiaxu Xing, Nico Messikommer +1
Reinforcement learning (RL) has achieved outstanding success in complex robot control tasks, such as drone racing, where the RL agents have outperformed human champions in a known…
Event-Based De-Snowing for Autonomous Driving
Manasi Muglikar, Nico Messikommer, Marco Cannici +1
Adverse weather conditions, particularly heavy snowfall, pose significant challenges to both human drivers and autonomous vehicles. Traditional image-based de-snowing methods often…
Learning Perception-Aware Agile Flight in Cluttered Environments
Yunlong Song, Kexin Shi, Robert Penicka +1
Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minim…
Hilti SLAM Challenge 2023: Benchmarking Single + Multi-session SLAM across Sensor Constellations in Construction
Ashish Devadas Nair, Julien Kindle, Plamen Levchev +1
Simultaneous Localization and Mapping systems are a key enabler for positioning in both handheld and robotic applications. The Hilti SLAM Challenges organized over the past years h…
MPCC++: Model Predictive Contouring Control for Time-Optimal Flight with Safety Constraints
Maria Krinner, Angel Romero, Leonard Bauersfeld +3
Quadrotor flight is an extremely challenging problem due to the limited control authority encountered at the limit of handling. Model Predictive Contouring Control (MPCC) has emerg…
The Hilti SLAM Challenge Dataset
Michael Helmberger, Kristian Morin, Beda Berner +3
Research in Simultaneous Localization and Mapping (SLAM) has made outstanding progress over the past years. SLAM systems are nowadays transitioning from academic to real world appl…
Geometry-aware Compensation Scheme for Morphing Drones
Amedeo Fabris, Kevin Kleber, Davide Falanga +1
Morphing multirotors, such as the Foldable Drone , can increase the versatility of drones employing in-flight-adaptive-morphology. To further increase precision in their tasks, rec…
Hybrid Event Frame Sensors: Modeling, Calibration, and Simulation
Yunfan Lu, Nico Messikommer, Xiaogang Xu +5
Hybrid event-frame sensors integrate an Event Vision Sensor (EVS) and an Active Pixel Sensor (APS) within a single chip, combining the high dynamic range and low latency of the EVS…
Focus Is All You Need: Loss Functions For Event-based Vision
Guillermo Gallego, Mathias Gehrig, Davide Scaramuzza
Event cameras are novel vision sensors that output pixel-level brightness changes ("events") instead of traditional video frames. These asynchronous sensors offer several advantage…
Flightmare: A Flexible Quadrotor Simulator
Yunlong Song, Selim Naji, Elia Kaufmann +2
State-of-the-art quadrotor simulators have a rigid and highly-specialized structure: either are they really fast, physically accurate, or photo-realistic. In this work, we propose…
Event-aided Direct Sparse Odometry
Javier Hidalgo-Carrió, Guillermo Gallego, Davide Scaramuzza
We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in the blind time betwee…
Data-Efficient Collaborative Decentralized Thermal-Inertial Odometry
Vincenzo Polizzi, Robert Hewitt, Javier Hidalgo-Carrió +2
We propose a system solution to achieve data-efficient, decentralized state estimation for a team of flying robots using thermal images and inertial measurements. Each robot can fl…
Independent Motion Detection with Event-driven Cameras
Valentina Vasco, Arren Glover, Elias Mueggler +3
Unlike standard cameras that send intensity images at a constant frame rate, event-driven cameras asynchronously report pixel-level brightness changes, offering low latency and hig…