9 papers
PackLab: A Comprehensive Framework for Developing, Training, and Evaluating MLLMs in Robotic Bin Packing
Donghao Zhou, Jia-Hui Pan, Fan Zhang +6
Robotic bin packing requires long-horizon sequential decision-making, as each object placement affects the available space for subsequent packing. Existing methods primarily rely o…
SkelGen4D: Weakly-Supervised Skeleton-Based 4D Generation for Text-Driven Mesh Animation
Hao Feng, Zhi Zuo, Jia-Hui Pan +6
We study 4D generation to synthesize temporally coherent sequences of 3D geometry for animation and content creation. In contrast to existing SDS-based optimization methods and vid…
coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation
Chunhan Li, Qifeng Wu, Jia-Hui Pan +7
Text-to-image generation has advanced rapidly, but existing models still struggle with faithfully composing multiple objects and preserving their attributes in complex scenes. We p…
Rethinking Intermediate Representation for VLM-based Robot Manipulation
Weiliang Tang, Jialin Gao, Jia-Hui Pan +6
Vision-Language Model (VLM) is an important component to enable robust robot manipulation. Yet, using it to translate human instructions into an action-resolvable intermediate repr…
CollaBot: Vision-Language Guided Simultaneous Collaborative Manipulation
Kun Song, Gaoming Chen, Shentao Ma +5
One central goal of robotics is to enable robots to interact with the physical world. Traditional manipulation studies primarily focus on single robots and relatively small objects…
OPA-Pack: Object-Property-Aware Robotic Bin Packing
Jia-Hui Pan, Yeok Tatt Cheah, Zhengzhe Liu +5
Robotic bin packing aids in a wide range of real-world scenarios such as e-commerce and warehouses. Yet, existing works focus mainly on considering the shape of objects to optimize…
Incentivizing Multimodal Reasoning in Large Models for Direct Robot Manipulation
Weiliang Tang, Dong Jing, Jia-Hui Pan +5
Recent Large Multimodal Models have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems and realizing accurate spatial perception. O…
GeoManip: Geometric Constraints as General Interfaces for Robot Manipulation
Weiliang Tang, Jia-Hui Pan, Yun-Hui Liu +4
We present GeoManip, a framework to enable generalist robots to leverage essential conditions derived from object and part relationships, as geometric constraints, for robot manipu…
Embodiment-Agnostic Action Planning via Object-Part Scene Flow
Weiliang Tang, Jia-Hui Pan, Wei Zhan +6
Observing that the key for robotic action planning is to understand the target-object motion when its associated part is manipulated by the end effector, we propose to generate the…