93 citations · 706 across the 49 of their papers we have counts for
19 papers · 1 filter
MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations
Nicklas Hansen, Yixin Lin, Hao Su +3
Poor sample efficiency continues to be the primary challenge for deployment of deep Reinforcement Learning (RL) algorithms for real-world applications, and in particular for visuo-…
Policy Adaptation from Foundation Model Feedback
Yuying Ge, Annabella Macaluso, Li Erran Li +2
Recent progress on vision-language foundation models have brought significant advancement to building general-purpose robots. By using the pre-trained models to encode the scene an…
On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
Nicklas Hansen, Zhecheng Yuan, Yanjie Ze +5
In this paper, we examine the effectiveness of pre-training for visuo-motor control tasks. We revisit a simple Learning-from-Scratch (LfS) baseline that incorporates data augmentat…
Visual Reinforcement Learning with Self-Supervised 3D Representations
Yanjie Ze, Nicklas Hansen, Yinbo Chen +2
A prominent approach to visual Reinforcement Learning (RL) is to learn an internal state representation using self-supervised methods, which has the potential benefit of improved s…
Graph Inverse Reinforcement Learning from Diverse Videos
Sateesh Kumar, Jonathan Zamora, Nicklas Hansen +2
Research on Inverse Reinforcement Learning (IRL) from third-person videos has shown encouraging results on removing the need for manual reward design for robotic tasks. However, mo…
Temporal Difference Learning for Model Predictive Control
Nicklas Hansen, Xiaolong Wang, Hao Su
Data-driven model predictive control has two key advantages over model-free methods: a potential for improved sample efficiency through model learning, and better performance as co…
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
Yuzhe Qin, Yueh-Hua Wu, Shaowei Liu +4
While significant progress has been made on understanding hand-object interactions in computer vision, it is still very challenging for robots to perform complex dexterous manipula…
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Ruihan Yang, Minghao Zhang, Nicklas Hansen +2
We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimen…
Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
Nicklas Hansen, Hao Su, Xiaolong Wang
While agents trained by Reinforcement Learning (RL) can solve increasingly challenging tasks directly from visual observations, generalizing learned skills to novel environments re…
DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation
Minghao Zhang, Pingcheng Jian, Yi Wu +2
We address the problem of safely solving complex bimanual robot manipulation tasks with sparse rewards. Such challenging tasks can be decomposed into sub-tasks that are accomplisha…
Solving Compositional Reinforcement Learning Problems via Task Reduction
Yunfei Li, Yilin Wu, Huazhe Xu +2
We propose a novel learning paradigm, Self-Imitation via Reduction (SIR), for solving compositional reinforcement learning problems. SIR is based on two core ideas: task reduction…
BeBold: Exploration Beyond the Boundary of Explored Regions
Tianjun Zhang, Huazhe Xu, Xiaolong Wang +4
Efficient exploration under sparse rewards remains a key challenge in deep reinforcement learning. To guide exploration, previous work makes extensive use of intrinsic reward (IR).…
Generalization in Reinforcement Learning by Soft Data Augmentation
Nicklas Hansen, Xiaolong Wang
Extensive efforts have been made to improve the generalization ability of Reinforcement Learning (RL) methods via domain randomization and data augmentation. However, as more facto…
Multi-Agent Collaboration via Reward Attribution Decomposition
Tianjun Zhang, Huazhe Xu, Xiaolong Wang +4
Recent advances in multi-agent reinforcement learning (MARL) have achieved super-human performance in games like Quake 3 and Dota 2. Unfortunately, these techniques require orders-…
Self-Supervised Policy Adaptation during Deployment
Nicklas Hansen, Rishabh Jangir, Yu Sun +5
In most real world scenarios, a policy trained by reinforcement learning in one environment needs to be deployed in another, potentially quite different environment. However, gener…
Rethinking preventing class-collapsing in metric learning with margin-based losses
Elad Levi, Tete Xiao, Xiaolong Wang +1
Metric learning seeks perceptual embeddings where visually similar instances are close and dissimilar instances are apart, but learned representations can be sub-optimal when the d…
Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
Qian Long, Zihan Zhou, Abhibav Gupta +3
In multi-agent games, the complexity of the environment can grow exponentially as the number of agents increases, so it is particularly challenging to learn good policies when the…
Multi-Task Reinforcement Learning with Soft Modularization
Ruihan Yang, Huazhe Xu, Yi Wu +1
Multi-task learning is a very challenging problem in reinforcement learning. While training multiple tasks jointly allow the policies to share parameters across different tasks, th…
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
Yu Sun, Xiaolong Wang, Zhuang Liu +3
In this paper, we propose Test-Time Training, a general approach for improving the performance of predictive models when training and test data come from different distributions. W…