105 citations · 471 across the 26 of their papers we have counts for
26 papers
Learning Multi-Stage Pick-and-Place with a Legged Mobile Manipulator
Haichao Zhang, Haonan Yu, Le Zhao +4
Quadruped-based mobile manipulation presents significant challenges in robotics due to the diversity of required skills, the extended task horizon, and partial observability. After…
SLIM: Sim-to-Real Legged Instructive Manipulation via Long-Horizon Visuomotor Learning
Haichao Zhang, Haonan Yu, Le Zhao +4
We present a low-cost legged mobile manipulation system that solves long-horizon real-world tasks, trained by reinforcement learning purely in simulation. This system is made possi…
VONet: Unsupervised Video Object Learning With Parallel U-Net Attention and Object-wise Sequential VAE
Haonan Yu, Wei Xu
Unsupervised video object learning seeks to decompose video scenes into structural object representations without any supervision from depth, optical flow, or segmentation. We pres…
Policy Expansion for Bridging Offline-to-Online Reinforcement Learning
Haichao Zhang, We Xu, Haonan Yu
Pre-training with offline data and online fine-tuning using reinforcement learning is a promising strategy for learning control policies by leveraging the best of both worlds in te…
Generative Planning for Temporally Coordinated Exploration in Reinforcement Learning
Haichao Zhang, Wei Xu, Haonan Yu
Standard model-free reinforcement learning algorithms optimize a policy that generates the action to be taken in the current time step in order to maximize expected future return.…
Do You Need the Entropy Reward (in Practice)?
Haonan Yu, Haichao Zhang, Wei Xu
Maximum entropy (MaxEnt) RL maximizes a combination of the original task reward and an entropy reward. It is believed that the regularization imposed by entropy, on both policy imp…
Towards Safe Reinforcement Learning with a Safety Editor Policy
Haonan Yu, Wei Xu, Haichao Zhang
We consider the safe reinforcement learning (RL) problem of maximizing utility with extremely low constraint violation rates. Assuming no prior knowledge or pre-training of the env…
TAAC: Temporally Abstract Actor-Critic for Continuous Control
Haonan Yu, Wei Xu, Haichao Zhang
We present temporally abstract actor-critic (TAAC), a simple but effective off-policy RL algorithm that incorporates closed-loop temporal abstraction into the actor-critic framewor…
MetaView: Few-shot Active Object Recognition
Wei Wei, Haonan Yu, Haichao Zhang +2
In robot sensing scenarios, instead of passively utilizing human captured views, an agent should be able to actively choose informative viewpoints of a 3D object as discriminative…
Hierarchical Reinforcement Learning By Discovering Intrinsic Options
Jesse Zhang, Haonan Yu, Wei Xu
We propose a hierarchical reinforcement learning method, HIDIO, that can learn task-agnostic options in a self-supervised manner while jointly learning to utilize them to solve spa…
Why Build an Assistant in Minecraft?
Arthur Szlam, Jonathan Gray, Kavya Srinet +11
In this document we describe a rationale for a research program aimed at building an open "assistant" in the game Minecraft, in order to make progress on the problems of natural la…
CraftAssist: A Framework for Dialogue-enabled Interactive Agents
Jonathan Gray, Kavya Srinet, Yacine Jernite +6
This paper describes an implementation of a bot assistant in Minecraft, and the tools and platform allowing players to interact with the bot and to record those interactions. The p…
EPNAS: Efficient Progressive Neural Architecture Search
Yanqi Zhou, Peng Wang, Sercan Arik +4
In this paper, we propose Efficient Progressive Neural Architecture Search (EPNAS), a neural architecture search (NAS) that efficiently handles large search space through a novel p…
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers
Ari S. Morcos, Haonan Yu, Michela Paganini +1
The success of lottery ticket initializations (Frankle and Carbin, 2019) suggests that small, sparsified networks can be trained so long as the network is initialized appropriately…
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP
Haonan Yu, Sergey Edunov, Yuandong Tian +1
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization…
Resource-Efficient Neural Architect
Yanqi Zhou, Siavash Ebrahimi, Sercan Ö. Arık +3
Neural Architecture Search (NAS) is a laborious process. Prior work on automated NAS targets mainly on improving accuracy, but lacks consideration of computational resource use. We…
Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents
Haonan Yu, Xiaochen Lian, Haichao Zhang +1
Recently there has been a rising interest in training agents, embodied in virtual environments, to perform language-directed tasks by deep reinforcement learning. In this paper, we…
Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game
Haichao Zhang, Haonan Yu, Wei Xu
Building intelligent agents that can communicate with and learn from humans in natural language is of great value. Supervised language learning is limited by the ability of capturi…
Interactive Grounded Language Acquisition and Generalization in a 2D World
Haonan Yu, Haichao Zhang, Wei Xu
We build a virtual agent for learning language in a 2D maze-like world. The agent sees images of the surrounding environment, listens to a virtual teacher, and takes actions to rec…
Listen, Interact and Talk: Learning to Speak via Interaction
Haichao Zhang, Haonan Yu, Wei Xu
One of the long-term goals of artificial intelligence is to build an agent that can communicate intelligently with human in natural language. Most existing work on natural language…
A Deep Compositional Framework for Human-like Language Acquisition in Virtual Environment
Haonan Yu, Haichao Zhang, Wei Xu
We tackle a task where an agent learns to navigate in a 2D maze-like environment called XWORLD. In each session, the agent perceives a sequence of raw-pixel frames, a natural langu…
Collecting and Annotating the Large Continuous Action Dataset
Daniel Paul Barrett, Ran Xu, Haonan Yu +1
We make available to the community a new dataset to support action-recognition research. This dataset is different from prior datasets in several key ways. It is significantly larg…
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
Haonan Yu, Jiang Wang, Zhiheng Huang +2
We present an approach that exploits hierarchical Recurrent Neural Networks (RNNs) to tackle the video captioning problem, i.e., generating one or multiple sentences to describe a…
Robot Language Learning, Generation, and Comprehension
Daniel Paul Barrett, Scott Alan Bronikowski, Haonan Yu +1
We present a unified framework which supports grounding natural-language semantics in robotic driving. This framework supports acquisition (learning grounded meanings of nouns and…