88 citations · 229 across the 15 of their papers we have counts for
16 papers
Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation
Annie Xie, Lisa Lee, Ted Xiao +1
What makes generalization hard for imitation learning in visual robotic manipulation? This question is difficult to approach at face value, but the environment from the perspective…
Barkour: Benchmarking Animal-level Agility with Quadruped Robots
Ken Caluwaerts, Atil Iscen, J. Chase Kew +41
Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biol…
Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models
Hao Liu, Xinyang Geng, Lisa Lee +4
Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing imp…
Instruction-Following Agents with Multimodal Transformer
Hao Liu, Lisa Lee, Kimin Lee +1
Humans are excellent at understanding language and vision to accomplish a wide range of tasks. In contrast, creating general instruction-following embodied agents remains a difficu…
Spectral Decomposition Representation for Reinforcement Learning
Tongzheng Ren, Tianjun Zhang, Lisa Lee +3
Representation learning often plays a critical role in reinforcement learning by managing the curse of dimensionality. A representative class of algorithms exploits a spectral deco…
Multimodal Masked Autoencoders Learn Transferable Representations
Xinyang Geng, Hao Liu, Lisa Lee +3
Building scalable models to learn from diverse, multimodal data remains an open challenge. For vision-language data, the dominant approaches are based on contrastive learning objec…
f-IRL: Inverse Reinforcement Learning via State Marginal Matching
Tianwei Ni, Harshit Sikchi, Yufei Wang +3
Imitation learning is well-suited for robotic tasks where it is difficult to directly program the behavior or specify a cost for optimal control. In this work, we propose a method…
Weakly-Supervised Reinforcement Learning for Controllable Behavior
Lisa Lee, Benjamin Eysenbach, Ruslan Salakhutdinov +2
Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks. However, in many settings, an agent must winnow down the inconceivably large space…
Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling
Yaqiong Li, Xuhui Fan, Ling Chen +3
The Dirichlet Belief Network~(DirBN) has been recently proposed as a promising approach in learning interpretable deep latent representations for objects. In this work, we leverage…
Efficient Exploration via State Marginal Matching
Lisa Lee, Benjamin Eysenbach, Emilio Parisotto +3
Exploration is critical to a reinforcement learning agent's performance in its given environment. Prior exploration methods are often based on using heuristic auxiliary predictions…
Embodied Multimodal Multitask Learning
Devendra Singh Chaplot, Lisa Lee, Ruslan Salakhutdinov +2
Recent efforts on training visual navigation agents conditioned on language using deep reinforcement learning have been successful in learning policies for different multimodal tas…
On the Complexity of Exploration in Goal-Driven Navigation
Maruan Al-Shedivat, Lisa Lee, Ruslan Salakhutdinov +1
Building agents that can explore their environments intelligently is a challenging open problem. In this paper, we make a step towards understanding how a hierarchical design of th…
Gated Path Planning Networks
Lisa Lee, Emilio Parisotto, Devendra Singh Chaplot +2
Value Iteration Networks (VINs) are effective differentiable path planning modules that can be used by agents to perform navigation while still maintaining end-to-end differentiabi…
Dual Motion GAN for Future-Flow Embedded Video Prediction
Xiaodan Liang, Lisa Lee, Wei Dai +1
Future frame prediction in videos is a promising avenue for unsupervised video representation learning. Video frames are naturally generated by the inherent pixel flows from preced…
Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection
Xiaodan Liang, Lisa Lee, Eric P. Xing
Despite progress in visual perception tasks such as image classification and detection, computers still struggle to understand the interdependency of objects in the scene as a whol…
Robotic Search & Rescue via Online Multi-task Reinforcement Learning
Lisa Lee
Reinforcement learning (RL) is a general and well-known method that a robot can use to learn an optimal control policy to solve a particular task. We would like to build a versatil…