6 papers
Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
Guanxiong Chen, Qianjun Xia, Jiawei Peng +24
Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover…
LychSim: A Controllable and Interactive Simulation Framework for Vision Research
Wufei Ma, Chloe Wang, Siyi Chen +3
While self-supervised pretraining has reduced vision systems' reliance on synthetic data, simulation remains an indispensable tool for closed-loop optimization and rigorous out-of-…
Perceptual Taxonomy: Evaluating and Guiding Hierarchical Scene Reasoning in Vision-Language Models
Jonathan Lee, Xingrui Wang, Jiawei Peng +9
We propose Perceptual Taxonomy, a structured process of scene understanding that first recognizes objects and their spatial configurations, then infers task-relevant properties suc…
4D-Animal: Freely Reconstructing Animatable 3D Animals from Videos
Shanshan Zhong, Jiawei Peng, Zehan Zheng +6
Existing methods for reconstructing animatable 3D animals from videos typically rely on sparse semantic keypoints to fit parametric models. However, obtaining such keypoints is lab…
PartInstruct: Part-level Instruction Following for Fine-grained Robot Manipulation
Yifan Yin, Zhengtao Han, Shivam Aarya +6
Fine-grained robot manipulation, such as lifting and rotating a bottle to display the label on the cap, requires robust reasoning about object parts and their relationships with in…
Learning Part Segmentation from Synthetic Animals
Jiawei Peng, Ju He, Prakhar Kaushik +3
Semantic part segmentation provides an intricate and interpretable understanding of an object, thereby benefiting numerous downstream tasks. However, the need for exhaustive annota…