193 citations · 292 across the 24 of their papers we have counts for
26 papers
Cross-Embodiment Robot Foundation World Models with Latent Actions
Huang Huang, Sriram Yenamandra, Arjun Majumdar +9
The diversity of robot embodiments and action spaces makes it challenging to build robot world models that generalize across different embodiments. We introduce the Latent Action-C…
DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning
Hanchen Cui, Sergio Arnaud, Arjun Majumdar +5
Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness a…
Approximate Imitation Learning for Event-based Quadrotor Flight in Cluttered Environments
Nico Messikommer, Jiaxu Xing, Leonard Bauersfeld +3
Event cameras offer high temporal resolution and low latency, making them ideal sensors for high-speed robotic applications where conventional cameras suffer from motion blur. Howe…
The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
Elie Aljalbout, Jiaxu Xing, Angel Romero +9
Machine learning has facilitated significant advancements across various robotics domains, including navigation, locomotion, and manipulation. Many such achievements have been driv…
Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation
Jiahe Pan, Jiaxu Xing, Rudolf Reiter +3
Learning control policies in simulation enables rapid, safe, and cost-effective development of advanced robotic capabilities. However, transferring these policies to the real world…
Accelerating Model-Based Reinforcement Learning with State-Space World Models
Maria Krinner, Elie Aljalbout, Angel Romero +1
Reinforcement learning (RL) is a powerful approach for robot learning. However, model-free RL (MFRL) requires a large number of environment interactions to learn successful control…
Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight
Angel Romero, Ashwin Shenai, Ismail Geles +2
Autonomous drone racing has risen as a challenging robotic benchmark for testing the limits of learning, perception, planning, and control. Expert human pilots are able to fly a dr…
Multi-Task Reinforcement Learning for Quadrotors
Jiaxu Xing, Ismail Geles, Yunlong Song +2
Reinforcement learning (RL) has shown great effectiveness in quadrotor control, enabling specialized policies to develop even human-champion-level performance in single-task scenar…
GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control
Mariam Hassan, Sebastian Stapf, Ahmad Rahimi +17
We present GEM, a Generalizable Ego-vision Multimodal world model that predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories. Hence, ou…
Student-Informed Teacher Training
Nico Messikommer, Jiaxu Xing, Elie Aljalbout +1
Imitation learning with a privileged teacher has proven effective for learning complex control behaviors from high-dimensional inputs, such as images. In this framework, a teacher…
LIMT: Language-Informed Multi-Task Visual World Models
Elie Aljalbout, Nikolaos Sotirakis, Patrick van der Smagt +2
Most recent successes in robot reinforcement learning involve learning a specialized single-task agent. However, robots capable of performing multiple tasks can be much more valuab…
The Shortcomings of Force-from-Motion in Robot Learning
Elie Aljalbout, Felix Frank, Patrick van der Smagt +1
Robotic manipulation requires accurate motion and physical interaction control. However, current robot learning approaches focus on motion-centric action spaces that do not explici…
Guided Decoding for Robot On-line Motion Generation and Adaption
Nutan Chen, Botond Cseke, Elie Aljalbout +3
We present a novel motion generation approach for robot arms, with high degrees of freedom, in complex settings that can adapt online to obstacles or new via points. Learning from…
On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer
Elie Aljalbout, Felix Frank, Maximilian Karl +1
We study the choice of action space in robot manipulation learning and sim-to-real transfer. We define metrics that assess the performance, and examine the emerging properties in t…
Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight
Angel Romero, Elie Aljalbout, Yunlong Song +1
A key open challenge in agile quadrotor flight is how to combine the flexibility and task-level generality of model-free reinforcement learning (RL) with the structure and online r…
CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spaces
Elie Aljalbout, Maximilian Karl, Patrick van der Smagt
Multi-robot manipulation tasks involve various control entities that can be separated into dynamically independent parts. A typical example of such real-world tasks is dual-arm man…
Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space
Maximilian Ulmer, Elie Aljalbout, Sascha Schwarz +1
Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challeng…
Dual-Arm Adversarial Robot Learning
Elie Aljalbout
Robot learning is a very promising topic for the future of automation and machine intelligence. Future robots should be able to autonomously acquire skills, learn to represent thei…
Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning
Elie Aljalbout, Maximilian Ulmer, Rudolph Triebel
Vision-based reinforcement learning (RL) is a promising approach to solve control tasks involving images as the main observation. State-of-the-art RL algorithms still struggle in t…
Learning to Centralize Dual-Arm Assembly
Marvin Alles, Elie Aljalbout
Robotic manipulators are widely used in modern manufacturing processes. However, their deployment in unstructured environments remains an open problem. To deal with the variety, co…
Making Curiosity Explicit in Vision-based RL
Elie Aljalbout, Maximilian Ulmer, Rudolph Triebel
Vision-based reinforcement learning (RL) is a promising technique to solve control tasks involving images as the main observation. State-of-the-art RL algorithms still struggle in…
How to Make Deep RL Work in Practice
Nirnai Rao, Elie Aljalbout, Axel Sauer +1
In recent years, challenging control problems became solvable with deep reinforcement learning (RL). To be able to use RL for large-scale real-world applications, a certain degree…
Learning Vision-based Reactive Policies for Obstacle Avoidance
Elie Aljalbout, Ji Chen, Konstantin Ritt +2
In this paper, we address the problem of vision-based obstacle avoidance for robotic manipulators. This topic poses challenges for both perception and motion generation. While most…
Task-Independent Spiking Central Pattern Generator: A Learning-Based Approach
Elie Aljalbout, Florian Walter, Florian Röhrbein +1
Legged locomotion is a challenging task in the field of robotics but a rather simple one in nature. This motivates the use of biological methodologies as solutions to this problem.…