88 citations · 407 across the 41 of their papers we have counts for
27 papers · 1 filter
Cross-Embodiment Robot Foundation World Models with Latent Actions
Huang Huang, Sriram Yenamandra, Arjun Majumdar +9
The diversity of robot embodiments and action spaces makes it challenging to build robot world models that generalize across different embodiments. We introduce the Latent Action-C…
DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning
Hanchen Cui, Sergio Arnaud, Arjun Majumdar +5
Pretrained vision-language-action (VLA) policies show promising zero-shot generalization, but often fail under deployment-time distribution shift, leading to decreased robustness a…
Visuo-Tactile World Models
Carolina Higuera, Sergio Arnaud, Byron Boots +3
We introduce multi-task Visuo-Tactile World Models (VT-WM), which capture the physics of contact through touch reasoning. By complementing vision with tactile sensing, VT-WM better…
WorldPlanner: Monte Carlo Tree Search and MPC with Action-Conditioned Visual World Models
R. Khorrambakht, Joaquim Ortiz-Haro, Joseph Amigo +4
Robots must understand their environment from raw sensory inputs and reason about the consequences of their actions in it to solve complex tasks. Behavior Cloning (BC) leverages ta…
What do we learn from a large-scale study of pre-trained visual representations in sim and real environments?
Sneha Silwal, Karmesh Yadav, Tingfan Wu +10
We present a large empirical investigation on the use of pre-trained visual representations (PVRs) for training downstream policies that execute real-world tasks. Our study involve…
Neural Grasp Distance Fields for Robot Manipulation
Thomas Weng, David Held, Franziska Meier +1
We formulate grasp learning as a neural field and present Neural Grasp Distance Fields (NGDF). Here, the input is a 6D pose of a robot end effector and output is a distance to a co…
Differentiable and Learnable Robot Models
Franziska Meier, Austin Wang, Giovanni Sutanto +2
Building differentiable simulations of physical processes has recently received an increasing amount of attention. Specifically, some efforts develop differentiable robotic physics…
Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning
Todor Davchev, Sarah Bechtle, Subramanian Ramamoorthy +1
Inverse reinforcement learning is a paradigm motivated by the goal of learning general reward functions from demonstrated behaviours. Yet the notion of generality for learnt costs…
Leveraging Forward Model Prediction Error for Learning Control
Sarah Bechtle, Bilal Hammoud, Akshara Rai +2
Learning for model based control can be sample-efficient and generalize well, however successfully learning models and controllers that represent the problem at hand can be challen…
Learning Navigation Skills for Legged Robots with Learned Robot Embeddings
Joanne Truong, Denis Yarats, Tianyu Li +4
Recent work has shown results on learning navigation policies for idealized cylinder agents in simulation and transferring them to real wheeled robots. Deploying such navigation po…
Multi-Modal Learning of Keypoint Predictive Models for Visual Object Manipulation
Sarah Bechtle, Neha Das, Franziska Meier
Humans have impressive generalization capabilities when it comes to manipulating objects and tools in completely novel environments. These capabilities are, at least partially, a r…
Model-Based Inverse Reinforcement Learning from Visual Demonstrations
Neha Das, Sarah Bechtle, Todor Davchev +3
Scaling model-based inverse reinforcement learning (IRL) to real robotic manipulation tasks with unknown dynamics remains an open problem. The key challenges lie in learning good d…
Planning in Learned Latent Action Spaces for Generalizable Legged Locomotion
Tianyu Li, Roberto Calandra, Deepak Pathak +3
Hierarchical learning has been successful at learning generalizable locomotion skills on walking robots in a sample-efficient manner. However, the low-dimensional "latent" action u…
Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation
Todor Davchev, Kevin Sebastian Luck, Michael Burke +3
Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can…
Supervised Learning and Reinforcement Learning of Feedback Models for Reactive Behaviors: Tactile Feedback Testbed
Giovanni Sutanto, Katharina Rombach, Yevgen Chebotar +4
Robots need to be able to adapt to unexpected changes in the environment such that they can autonomously succeed in their tasks. However, hand-designing feedback models for adaptat…
Learning State-Dependent Losses for Inverse Dynamics Learning
Kristen Morse, Neha Das, Yixin Lin +3
Being able to quickly adapt to changes in dynamics is paramount in model-based control for object manipulation tasks. In order to influence fast adaptation of the inverse dynamics…
Encoding Physical Constraints in Differentiable Newton-Euler Algorithm
Giovanni Sutanto, Austin S. Wang, Yixin Lin +4
The recursive Newton-Euler Algorithm (RNEA) is a popular technique for computing the dynamics of robots. RNEA can be framed as a differentiable computational graph, enabling the dy…
Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
Tianyu Li, Nathan Lambert, Roberto Calandra +2
Learning to locomote to arbitrary goals on hardware remains a challenging problem for reinforcement learning. In this paper, we present a hierarchical learning framework that impro…
Curious iLQR: Resolving Uncertainty in Model-based RL
Sarah Bechtle, Yixin Lin, Akshara Rai +2
Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for lear…
Using Simulation to Improve Sample-Efficiency of Bayesian Optimization for Bipedal Robots
Akshara Rai, Rika Antonova, Franziska Meier +1
Learning for control can acquire controllers for novel robotic tasks, paving the path for autonomous agents. Such controllers can be expert-designed policies, which typically requi…
A New Data Source for Inverse Dynamics Learning
Daniel Kappler, Franziska Meier, Nathan Ratliff +1
Modern robotics is gravitating toward increasingly collaborative human robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, ada…
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control
Arunkumar Byravan, Felix Leeb, Franziska Meier +1
In this work, we present an approach to deep visuomotor control using structured deep dynamics models. Our deep dynamics model, a variant of SE3-Nets, learns a low-dimensional pose…
Learning Sensor Feedback Models from Demonstrations via Phase-Modulated Neural Networks
Giovanni Sutanto, Zhe Su, Stefan Schaal +1
In order to robustly execute a task under environmental uncertainty, a robot needs to be able to reactively adapt to changes arising in its environment. The environment changes are…
Real-time Perception meets Reactive Motion Generation
Daniel Kappler, Franziska Meier, Jan Issac +7
We address the challenging problem of robotic grasping and manipulation in the presence of uncertainty. This uncertainty is due to noisy sensing, inaccurate models and hard-to-pred…