5 citations · 8 across the 12 of their papers we have counts for
13 papers
Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory
Minghao Guo, Qingyue Jiao, Xiujin Liu +77
Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities…
Hydra-0: Action Flow for Generalist World Modeling and Control
Hongyu Li, Bowen Wen, Xinghao Zhu +8
We introduce Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface enables generalist world mod…
Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration
Xinghao Zhu, Zixi Liu, Shalin Jain +18
Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. We present Contact…
Vesta: A Generalist Embodied Reasoning Model
Johan Bjorck, Zhiqi Li, Yunze Man +29
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at indiv…
SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation
Nadun Ranawaka, Josiah Wong, Wei-Lin Pai +15
Training and evaluating robot policies in the real world is costly and difficult to scale. We introduce SimFoundry, a modular and automated system for zero-shot real-to-sim scene c…
Refinery: Active Fine-tuning and Deployment-time Optimization for Contact-Rich Policies
Bingjie Tang, Iretiayo Akinola, Jie Xu +6
Simulation-based learning has enabled policies for precise, contact-rich tasks (e.g., robotic assembly) to reach high success rates (~80%) under high levels of observation noise an…
SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation
Cheng-Chun Hsu, Bowen Wen, Jie Xu +5
We introduce SPOT, an object-centric imitation learning framework. The key idea is to capture each task by an object-centric representation, specifically the SE(3) object pose traj…
SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Garrett, Ajay Mandlekar, Bowen Wen +1
Imitation learning from human demonstrations is an effective paradigm for robot manipulation, but acquiring large datasets is costly and resource-intensive, especially for long-hor…
FORGE: Force-Guided Exploration for Robust Contact-Rich Manipulation under Uncertainty
Michael Noseworthy, Bingjie Tang, Bowen Wen +7
We present FORGE, a method for sim-to-real transfer of force-aware manipulation policies in the presence of significant pose uncertainty. During simulation-based policy learning, F…
AutoMate: Specialist and Generalist Assembly Policies over Diverse Geometries
Bingjie Tang, Iretiayo Akinola, Jie Xu +7
Robotic assembly for high-mixture settings requires adaptivity to diverse parts and poses, which is an open challenge. Meanwhile, in other areas of robotics, large models and sim-t…
NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows
Zhenggang Tang, Zhongzheng Ren, Xiaoming Zhao +4
We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the…
Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects
Yijia Weng, Bowen Wen, Jonathan Tremblay +4
We address the problem of building digital twins of unknown articulated objects from two RGBD scans of the object at different articulation states. We decompose the problem into tw…
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Ajay Mandlekar, Soroush Nasiriany, Bowen Wen +5
Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents. However, the demonstrations can be extremely c…