21 citations · 118 across the 15 of their papers we have counts for
16 papers
DreamFormer: Dream Imitation with a Transformer World Model for Language-Conditioned Robotic Manipulation
Mostafa Kotb, Cornelius Weber, Muhammad Burhan Hafez +1
We introduce DreamFormer, a model-based agent that acquires language-conditioned, multi-task skills by imitating expert demonstrations within the latent imagination of a learned wo…
REDACT: Robust Perceptive Locomotion under Unseen Visual Corruption
Natapat Kirdwichai, Tobias Driskell-Poole, Andrei Sontea +3
Depth-conditioned locomotion policies have demonstrated impressive agile maneuvers, but can be steered to unpredictable actions when observations are outside their training distrib…
Through the Lens of Doubt: Robust and Efficient Uncertainty Estimation for Visual Place Recognition
Emily Miller, Michael Milford, Muhammad Burhan Hafez +2
Visual Place Recognition (VPR) enables robots and autonomous vehicles to identify previously visited locations by matching current observations against a database of known places.…
Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation
Muhammad Burhan Hafez, Kerim Erekmen
Central to the development of universal learning systems is the ability to solve multiple tasks without retraining from scratch when new data arrives. This is crucial because each…
QT-TDM: Planning With Transformer Dynamics Model and Autoregressive Q-Learning
Mostafa Kotb, Cornelius Weber, Muhammad Burhan Hafez +1
Inspired by the success of the Transformer architecture in natural language processing and computer vision, we investigate the use of Transformers in Reinforcement Learning (RL), s…
Continual Robot Learning using Self-Supervised Task Inference
Muhammad Burhan Hafez, Stefan Wermter
Endowing robots with the human ability to learn a growing set of skills over the course of a lifetime as opposed to mastering single tasks is an open problem in robot learning. Whi…
Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning
Muhammad Burhan Hafez, Tilman Immisch, Tom Weber +1
Deep Reinforcement Learning agents often suffer from catastrophic forgetting, forgetting previously found solutions in parts of the input space when training on new data. Replay Me…
Model Predictive Control with Self-supervised Representation Learning
Jonas Matthies, Muhammad Burhan Hafez, Mostafa Kotb +1
Over the last few years, we have not seen any major developments in model-free or model-based learning methods that would make one obsolete relative to the other. In most cases, th…
Chat with the Environment: Interactive Multimodal Perception Using Large Language Models
Xufeng Zhao, Mengdi Li, Cornelius Weber +2
Programming robot behavior in a complex world faces challenges on multiple levels, from dextrous low-level skills to high-level planning and reasoning. Recent pre-trained Large Lan…
Learning Bidirectional Action-Language Translation with Limited Supervision and Incongruent Input
Ozan Özdemir, Matthias Kerzel, Cornelius Weber +4
Human infant learning happens during exploration of the environment, by interaction with objects, and by listening to and repeating utterances casually, which is analogous to unsup…
Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations
Xufeng Zhao, Cornelius Weber, Muhammad Burhan Hafez +1
Sound is one of the most informative and abundant modalities in the real world while being robust to sense without contacts by small and cheap sensors that can be placed on mobile…
Improving Model-Based Reinforcement Learning with Internal State Representations through Self-Supervision
Julien Scholz, Cornelius Weber, Muhammad Burhan Hafez +1
Using a model of the environment, reinforcement learning agents can plan their future moves and achieve superhuman performance in board games like Chess, Shogi, and Go, while remai…
Improving Robot Dual-System Motor Learning with Intrinsically Motivated Meta-Control and Latent-Space Experience Imagination
Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel +1
Combining model-based and model-free learning systems has been shown to improve the sample efficiency of learning to perform complex robotic tasks. However, dual-system approaches…
Efficient Intrinsically Motivated Robotic Grasping with Learning-Adaptive Imagination in Latent Space
Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel +1
Combining model-based and model-free deep reinforcement learning has shown great promise for improving sample efficiency on complex control tasks while still retaining high perform…
Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning
Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel +1
Recent success in deep reinforcement learning for continuous control has been dominated by model-free approaches which, unlike model-based approaches, do not suffer from representa…
Deep Intrinsically Motivated Continuous Actor-Critic for Efficient Robotic Visuomotor Skill Learning
Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel +1
In this paper, we present a new intrinsically motivated actor-critic algorithm for learning continuous motor skills directly from raw visual input. Our neural architecture is compo…