169 citations · 661 across the 22 of their papers we have counts for
22 papers
DFields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangement
Yixuan Wang, Mingtong Zhang, Zhuoran Li +5
Scene representation is a crucial design choice in robotic manipulation systems. An ideal representation is expected to be 3D, dynamic, and semantic to meet the demands of diverse…
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
Wenlong Huang, Chen Wang, Ruohan Zhang +3
Large language models (LLMs) are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation in the form of reasoning and planning. Despite the p…
Dynamic-Resolution Model Learning for Object Pile Manipulation
Yixuan Wang, Yunzhu Li, Katherine Driggs-Campbell +2
Dynamics models learned from visual observations have shown to be effective in various robotic manipulation tasks. One of the key questions for learning such dynamics models is wha…
Multi-Object Manipulation via Object-Centric Neural Scattering Functions
Stephen Tian, Yancheng Cai, Hong-Xing Yu +5
Learned visual dynamics models have proven effective for robotic manipulation tasks. Yet, it remains unclear how best to represent scenes involving multi-object interactions. Curre…
RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools
Haochen Shi, Huazhe Xu, Samuel Clarke +2
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often reg…
The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects
Ruohan Gao, Yiming Dou, Hao Li +5
We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for multisensory object-centric learning, centered around object recognition, reconstruction, and manipulatio…
Partial-View Object View Synthesis via Filtered Inversion
Fan-Yun Sun, Jonathan Tremblay, Valts Blukis +11
We propose Filtering Inversion (FINV), a learning framework and optimization process that predicts a renderable 3D object representation from one or few partial views. FINV address…
Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?
Lirui Wang, Kaiqing Zhang, Yunzhu Li +2
Decentralized learning has been advocated and widely deployed to make efficient use of distributed datasets, with an extensive focus on supervised learning (SL) problems. Unfortuna…
Reinforcement Learning with Neural Radiance Fields
Danny Driess, Ingmar Schubert, Pete Florence +2
It is a long-standing problem to find effective representations for training reinforcement learning (RL) agents. This paper demonstrates that learning state representations with su…
RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks
Haochen Shi, Huazhe Xu, Zhiao Huang +2
Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, roll…
ComPhy: Compositional Physical Reasoning of Objects and Events from Videos
Zhenfang Chen, Kexin Yi, Yunzhu Li +4
Objects' motions in nature are governed by complex interactions and their properties. While some properties, such as shape and material, can be identified via the object's visual a…
DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools
Xingyu Lin, Zhiao Huang, Yunzhu Li +3
We consider the problem of sequential robotic manipulation of deformable objects using tools. Previous works have shown that differentiable physics simulators provide gradients to…
Learning Multi-Object Dynamics with Compositional Neural Radiance Fields
Danny Driess, Zhiao Huang, Yunzhu Li +2
We present a method to learn compositional multi-object dynamics models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural…
Visual Grounding of Learned Physical Models
Yunzhu Li, Toru Lin, Kexin Yi +5
Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. The abilities to perform physical…
Learning Compositional Koopman Operators for Model-Based Control
Yunzhu Li, Hao He, Jiajun Wu +2
Finding an embedding space for a linear approximation of a nonlinear dynamical system enables efficient system identification and control synthesis. The Koopman operator theory lay…
CLEVRER: CoLlision Events for Video REpresentation and Reasoning
Kexin Yi, Chuang Gan, Yunzhu Li +4
The ability to reason about temporal and causal events from videos lies at the core of human intelligence. Most video reasoning benchmarks, however, focus on pattern recognition fr…
Connecting Touch and Vision via Cross-Modal Prediction
Yunzhu Li, Jun-Yan Zhu, Russ Tedrake +1
Humans perceive the world using multi-modal sensory inputs such as vision, audition, and touch. In this work, we investigate the cross-modal connection between vision and touch. Th…
Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
Yunzhu Li, Jiajun Wu, Russ Tedrake +2
Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigi…
Propagation Networks for Model-Based Control Under Partial Observation
Yunzhu Li, Jiajun Wu, Jun-Yan Zhu +3
There has been an increasing interest in learning dynamics simulators for model-based control. Compared with off-the-shelf physics engines, a learnable simulator can quickly adapt…
InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
Yunzhu Li, Jiaming Song, Stefano Ermon
The goal of imitation learning is to mimic expert behavior without access to an explicit reward signal. Expert demonstrations provided by humans, however, often show significant va…
Skin Cancer Detection and Tracking using Data Synthesis and Deep Learning
Yunzhu Li, Andre Esteva, Brett Kuprel +3
Dense object detection and temporal tracking are needed across applications domains ranging from people-tracking to analysis of satellite imagery over time. The detection and track…
Face Detection with End-to-End Integration of a ConvNet and a 3D Model
Yunzhu Li, Benyuan Sun, Tianfu Wu +1
This paper presents a method for face detection in the wild, which integrates a ConvNet and a 3D mean face model in an end-to-end multi-task discriminative learning framework. The…