59 citations · 104 across the 15 of their papers we have counts for
15 papers · 1 filter
GS-Agent: Creating 4D Physical Worlds With Generative Simulation
Hongxin Zhang, Chunru Lin, Junyan Li +3
Creating dynamic and physically realistic 4D worlds from natural language descriptions is both fascinating and challenging. Traditional computer graphics methods rely on manual cre…
One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs
Jianren Wang, Kangni Liu, Dingkun Guo +2
Learning to manipulate dynamic and deformable objects from a single demonstration video holds great promise in terms of scalability. Previous approaches have predominantly focused…
Thin-Shell Object Manipulations With Differentiable Physics Simulations
Yian Wang, Juntian Zheng, Zhehuan Chen +4
In this work, we aim to teach robots to manipulate various thin-shell materials. Prior works studying thin-shell object manipulation mostly rely on heuristic policies or learn poli…
DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
Zilin Si, Gu Zhang, Qingwei Ben +4
We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In…
RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
Yufei Wang, Zhanyi Sun, Jesse Zhang +4
Reward engineering has long been a challenge in Reinforcement Learning (RL) research, as it often requires extensive human effort and iterative processes of trial-and-error to desi…
RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation
Yufei Wang, Zhou Xian, Feng Chen +6
We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. RoboGen leverages the latest advancements in fou…
Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models
Pushkal Katara, Zhou Xian, Katerina Fragkiadaki
Generalist robot manipulators need to learn a wide variety of manipulation skills across diverse environments. Current robot training pipelines rely on humans to provide kinestheti…
Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulation
Theophile Gervet, Zhou Xian, Nikolaos Gkanatsios +1
3D perceptual representations are well suited for robot manipulation as they easily encode occlusions and simplify spatial reasoning. Many manipulation tasks require high spatial p…
Towards Generalist Robots: A Promising Paradigm via Generative Simulation
Zhou Xian, Theophile Gervet, Zhenjia Xu +3
This document serves as a position paper that outlines the authors' vision for a potential pathway towards generalist robots. The purpose of this document is to share the excitemen…
Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement
Nikolaos Gkanatsios, Ayush Jain, Zhou Xian +3
Language is compositional; an instruction can express multiple relation constraints to hold among objects in a scene that a robot is tasked to rearrange. Our focus in this work is…
SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments
Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg +5
While significant research progress has been made in robot learning for control, unique challenges arise when simultaneously co-optimizing morphology. Existing work has typically b…
FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation
Zhou Xian, Bo Zhu, Zhenjia Xu +4
Humans manipulate various kinds of fluids in their everyday life: creating latte art, scooping floating objects from water, rolling an ice cream cone, etc. Using robots to augment…
RoboNinja: Learning an Adaptive Cutting Policy for Multi-Material Objects
Zhenjia Xu, Zhou Xian, Xingyu Lin +4
We introduce RoboNinja, a learning-based cutting system for multi-material objects (i.e., soft objects with rigid cores such as avocados or mangos). In contrast to prior works usin…
HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworks
Zhou Xian, Shamit Lal, Hsiao-Yu Tung +2
We propose HyperDynamics, a dynamics meta-learning framework that conditions on an agent's interactions with the environment and optionally its visual observations, and generates t…
Closed-Chain Manipulation of Large Objects by Multi-Arm Robotic Systems
Xian Zhou, Puttichai Lertkultanon, Quang-Cuong Pham
Closed kinematic chains are created whenever multiple robot arms concurrently manipulate a single object. The closed-chain constraint, when coupled with robot joint limits, dramati…