5 citations · 11 across the 11 of their papers we have counts for
13 papers
COWBOY: A Scalable Sim-to-Real Framework for Learning Contextual Whole-Body Manipulation
Jiahui Yang, Jason Jingzhou Liu, Ray Muxin Liu +4
Mobile manipulators must adapt not only to objects, but also to the spatial context in which they appear: where to position the base, how to manipulate, and where to look while nav…
FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning
Steven Oh, Jason Jingzhou Liu, Tony Tao +5
Contact-rich manipulation requires force sensitivity, but many robot arms lack dedicated force sensors due to their high cost. We present Neural External Torque Estimation (NEXT),…
Response to Promises and Pitfalls of Deep Kernel Learning
Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov +1
This note responds to "Promises and Pitfalls of Deep Kernel Learning" (Ober et al., 2021). The marginal likelihood of a Gaussian process can be compartmentalized into a data fit te…
AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents
Arman Zharmagambetov, Chuan Guo, Ivan Evtimov +3
Autonomous AI agents that can follow instructions and perform complex multi-step tasks have tremendous potential to boost human productivity. However, to perform many of these task…
Self-Regulation and Requesting Interventions
So Yeon Min, Yue Wu, Jimin Sun +4
Human intelligence involves metacognitive abilities like self-regulation, recognizing limitations, and seeking assistance only when needed. While LLM Agents excel in many domains,…
FACTR: Force-Attending Curriculum Training for Contact-Rich Policy Learning
Jason Jingzhou Liu, Yulong Li, Kenneth Shaw +3
Many contact-rich tasks humans perform, such as box pickup or rolling dough, rely on force feedback for reliable execution. However, this force information, which is readily availa…
Local Policies Enable Zero-shot Long-horizon Manipulation
Murtaza Dalal, Min Liu, Walter Talbott +4
Sim2real for robotic manipulation is difficult due to the challenges of simulating complex contacts and generating realistic task distributions. To tackle the latter problem, we in…
Neural MP: A Generalist Neural Motion Planner
Murtaza Dalal, Jiahui Yang, Russell Mendonca +3
The current paradigm for motion planning generates solutions from scratch for every new problem, which consumes significant amounts of time and computational resources. For complex…
Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation
Quanting Xie, So Yeon Min, Pengliang Ji +7
There is no limit to how much a robot might explore and learn, but all of that knowledge needs to be searchable and actionable. Within language research, retrieval augmented genera…
Situated Instruction Following
So Yeon Min, Xavi Puig, Devendra Singh Chaplot +6
Language is never spoken in a vacuum. It is expressed, comprehended, and contextualized within the holistic backdrop of the speaker's history, actions, and environment. Since human…
Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks
Murtaza Dalal, Tarun Chiruvolu, Devendra Chaplot +1
Large Language Models (LLMs) have been shown to be capable of performing high-level planning for long-horizon robotics tasks, yet existing methods require access to a pre-defined s…
AgentKit: Structured LLM Reasoning with Dynamic Graphs
Yue Wu, Yewen Fan, So Yeon Min +6
We propose an intuitive LLM prompting framework (AgentKit) for multifunctional agents. AgentKit offers a unified framework for explicitly constructing a complex "thought process" f…
Confronting Reward Model Overoptimization with Constrained RLHF
Ted Moskovitz, Aaditya K. Singh, DJ Strouse +4
Large language models are typically aligned with human preferences by optimizing (RMs) fitted to human feedback. However, human preferences are multi-facet…