100 citations · 587 across the 34 of their papers we have counts for
34 papers
Evolutionary Curriculum Training for DRL-Based Navigation Systems
Max Asselmeier, Zhaoyi Li, Kelin Yu +1
In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising method for robot collision avoidance. However, such DRL models often come with limitations, such as ad…
Language-Guided Traffic Simulation via Scene-Level Diffusion
Ziyuan Zhong, Davis Rempe, Yuxiao Chen +5
Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development. However, current approaches for controlling…
Partial-View Object View Synthesis via Filtered Inversion
Fan-Yun Sun, Jonathan Tremblay, Valts Blukis +11
We propose Filtering Inversion (FINV), a learning framework and optimization process that predicts a renderable 3D object representation from one or few partial views. FINV address…
Zero-Shot Object Searching Using Large-scale Object Relationship Prior
Hongyi Chen, Ruinian Xu, Shuo Cheng +2
Home-assistant robots have been a long-standing research topic, and one of the biggest challenges is searching for required objects in housing environments. Previous object-goal na…
MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
Chen Wang, Linxi Fan, Jiankai Sun +5
Imitation learning from human demonstrations is a promising paradigm for teaching robots manipulation skills in the real world. However, learning complex long-horizon tasks often r…
Foundation Models for Semantic Novelty in Reinforcement Learning
Tarun Gupta, Peter Karkus, Tong Che +2
Effectively exploring the environment is a key challenge in reinforcement learning (RL). We address this challenge by defining a novel intrinsic reward based on a foundation model,…
Guided Conditional Diffusion for Controllable Traffic Simulation
Ziyuan Zhong, Davis Rempe, Danfei Xu +5
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based traffic models offer flexible control to make ve…
LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation
Shuo Cheng, Danfei Xu
To assist with everyday human activities, robots must solve complex long-horizon tasks and generalize to new settings. Recent deep reinforcement learning (RL) methods show promise…
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Ishika Singh, Valts Blukis, Arsalan Mousavian +6
Task planning can require defining myriad domain knowledge about the world in which a robot needs to act. To ameliorate that effort, large language models (LLMs) can be used to sco…
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Yulong Cao, Chaowei Xiao, Anima Anandkumar +2
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors. While many prior works aim to achieve higher prediction accuracy, few s…
BITS: Bi-level Imitation for Traffic Simulation
Danfei Xu, Yuxiao Chen, Boris Ivanovic +1
Simulation is the key to scaling up validation and verification for robotic systems such as autonomous vehicles. Despite advances in high-fidelity physics and sensor simulation, a…
Robust Trajectory Prediction against Adversarial Attacks
Yulong Cao, Danfei Xu, Xinshuo Weng +4
Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving (AD) systems. However, these methods are vulnerable to adversarial attacks,…
Generalizable Task Planning through Representation Pretraining
Chen Wang, Danfei Xu, Li Fei-Fei
The ability to plan for multi-step manipulation tasks in unseen situations is crucial for future home robots. But collecting sufficient experience data for end-to-end learning is o…
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
Ajay Mandlekar, Danfei Xu, Josiah Wong +7
Imitating human demonstrations is a promising approach to endow robots with various manipulation capabilities. While recent advances have been made in imitation learning and batch…
Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration
Chen Wang, Claudia Pérez-D'Arpino, Danfei Xu +3
We present a method for learning a human-robot collaboration policy from human-human collaboration demonstrations. An effective robot assistant must learn to handle diverse human b…
Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control
Chen Wang, Rui Wang, Ajay Mandlekar +3
Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. However, IL methods often fail to generalize to new scene configuratio…
Human-in-the-Loop Imitation Learning using Remote Teleoperation
Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín +3
Imitation Learning is a promising paradigm for learning complex robot manipulation skills by reproducing behavior from human demonstrations. However, manipulation tasks often conta…
Deep Affordance Foresight: Planning Through What Can Be Done in the Future
Danfei Xu, Ajay Mandlekar, Roberto Martín-Martín +3
Planning in realistic environments requires searching in large planning spaces. Affordances are a powerful concept to simplify this search, because they model what actions can be s…
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín +2
Imitation learning is an effective and safe technique to train robot policies in the real world because it does not depend on an expensive random exploration process. However, due…
Positive-Unlabeled Reward Learning
Danfei Xu, Misha Denil
Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from lear…
6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints
Chen Wang, Roberto Martín-Martín, Danfei Xu +5
We present 6-PACK, a deep learning approach to category-level 6D object pose tracking on RGB-D data. Our method tracks in real-time novel object instances of known object categorie…
Regression Planning Networks
Danfei Xu, Roberto Martín-Martín, De-An Huang +3
Recent learning-to-plan methods have shown promising results on planning directly from observation space. Yet, their ability to plan for long-horizon tasks is limited by the accura…
Situational Fusion of Visual Representation for Visual Navigation
Bokui Shen, Danfei Xu, Yuke Zhu +3
A complex visual navigation task puts an agent in different situations which call for a diverse range of visual perception abilities. For example, to "go to the nearest chair", the…
Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning
De-An Huang, Danfei Xu, Yuke Zhu +4
We address one-shot imitation learning, where the goal is to execute a previously unseen task based on a single demonstration. While there has been exciting progress in this direct…