24 citations · 75 across the 22 of their papers we have counts for
23 papers
Retriever: Composing the Perception-Reasoning-Action Loop for Long-Horizon Manipulation
Linfeng Zhao, Haojie Huang, Jiayuan Mao +3
Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with…
Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Maps
Linfeng Zhao, Lawson L. S. Wong
Learning navigation capabilities in different environments has long been one of the major challenges in decision-making. In this work, we focus on zero-shot navigation ability usin…
On-Robot Reinforcement Learning with Goal-Contrastive Rewards
Ondrej Biza, Thomas Weng, Lingfeng Sun +6
Reinforcement Learning (RL) has the potential to enable robots to learn from their own actions in the real world. Unfortunately, RL can be prohibitively expensive, in terms of on-r…
E(2)-Equivariant Graph Planning for Navigation
Linfeng Zhao, Hongyu Li, Taskin Padir +2
Learning for robot navigation presents a critical and challenging task. The scarcity and costliness of real-world datasets necessitate efficient learning approaches. In this letter…
Can Euclidean Symmetry be Leveraged in Reinforcement Learning and Planning?
Linfeng Zhao, Owen Howell, Jung Yeon Park +3
In robotic tasks, changes in reference frames typically do not influence the underlying physical properties of the system, which has been known as invariance of physical laws.These…
One-shot Imitation Learning via Interaction Warping
Ondrej Biza, Skye Thompson, Kishore Reddy Pagidi +7
Imitation learning of robot policies from few demonstrations is crucial in open-ended applications. We propose a new method, Interaction Warping, for learning SE(3) robotic manipul…
The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetry
Dian Wang, Jung Yeon Park, Neel Sortur +3
Extensive work has demonstrated that equivariant neural networks can significantly improve sample efficiency and generalization by enforcing an inductive bias in the network archit…
Robust Imitation of a Few Demonstrations with a Backwards Model
Jung Yeon Park, Lawson L. S. Wong
Behavior cloning of expert demonstrations can speed up learning optimal policies in a more sample-efficient way over reinforcement learning. However, the policy cannot extrapolate…
Scaling up and Stabilizing Differentiable Planning with Implicit Differentiation
Linfeng Zhao, Huazhe Xu, Lawson L. S. Wong
Differentiable planning promises end-to-end differentiability and adaptivity. However, an issue prevents it from scaling up to larger-scale problems: they need to differentiate thr…
Integrating Symmetry into Differentiable Planning with Steerable Convolutions
Linfeng Zhao, Xupeng Zhu, Lingzhi Kong +2
We study how group symmetry helps improve data efficiency and generalization for end-to-end differentiable planning algorithms when symmetry appears in decision-making tasks. Motiv…
Binding Actions to Objects in World Models
Ondrej Biza, Robert Platt, Jan-Willem van de Meent +2
We study the problem of binding actions to objects in object-factored world models using action-attention mechanisms. We propose two attention mechanisms for binding actions to obj…
Toward Compositional Generalization in Object-Oriented World Modeling
Linfeng Zhao, Lingzhi Kong, Robin Walters +1
Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compo…
Factored World Models for Zero-Shot Generalization in Robotic Manipulation
Ondrej Biza, Thomas Kipf, David Klee +3
World models for environments with many objects face a combinatorial explosion of states: as the number of objects increases, the number of possible arrangements grows exponentiall…
Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding
Seth Pate, Wei Xu, Ziyi Yang +3
To enable robots to instruct humans in collaborations, we identify several aspects of language processing that are not commonly studied in this context. These include location, pla…
Bad-Policy Density: A Measure of Reinforcement Learning Hardness
David Abel, Cameron Allen, Dilip Arumugam +3
Reinforcement learning is hard in general. Yet, in many specific environments, learning is easy. What makes learning easy in one environment, but difficult in another? We address t…
Hierarchical Robot Navigation in Novel Environments using Rough 2-D Maps
Chengguang Xu, Christopher Amato, Lawson L. S. Wong
In robot navigation, generalizing quickly to unseen environments is essential. Hierarchical methods inspired by human navigation have been proposed, typically consisting of a high-…
Action Priors for Large Action Spaces in Robotics
Ondrej Biza, Dian Wang, Robert Platt +2
In robotics, it is often not possible to learn useful policies using pure model-free reinforcement learning without significant reward shaping or curriculum learning. As a conseque…
Deep Imitation Learning for Bimanual Robotic Manipulation
Fan Xie, Alexander Chowdhury, M. Clara De Paolis Kaluza +3
We present a deep imitation learning framework for robotic bimanual manipulation in a continuous state-action space. A core challenge is to generalize the manipulation skills to ob…
Learning Discrete State Abstractions With Deep Variational Inference
Ondrej Biza, Robert Platt, Jan-Willem van de Meent +1
Abstraction is crucial for effective sequential decision making in domains with large state spaces. In this work, we propose an information bottleneck method for learning approxima…
A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions
Siddharth Karamcheti, Edward C. Williams, Dilip Arumugam +4
Robots operating alongside humans in diverse, stochastic environments must be able to accurately interpret natural language commands. These instructions often fall into one of two…
Modeling Latent Attention Within Neural Networks
Christopher Grimm, Dilip Arumugam, Siddharth Karamcheti +3
Deep neural networks are able to solve tasks across a variety of domains and modalities of data. Despite many empirical successes, we lack the ability to clearly understand and int…
Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularities
Dilip Arumugam, Siddharth Karamcheti, Nakul Gopalan +2
Humans can ground natural language commands to tasks at both abstract and fine-grained levels of specificity. For instance, a human forklift operator can be instructed to perform a…
Object-based World Modeling in Semi-Static Environments with Dependent Dirichlet-Process Mixtures
Lawson L. S. Wong, Thanard Kurutach, Leslie Pack Kaelbling +1
To accomplish tasks in human-centric indoor environments, robots need to represent and understand the world in terms of objects and their attributes. We refer to this attribute-bas…