10 papers
HODAgent: Towards On-Demand, Responsive Humanoids for Physical World Human Interaction
Wang Warren Chen, Jiahao Zhang, Zhenjiang Li +6
We propose HODAgent, a System-2 embodied agent for humanoid robots in service settings, addressing situated intent, responsive execution, task revision, and outcome verification. I…
Engagement Process: Rethinking the Temporal Interface of Action and Observation
Jialian Li, Yuchen Cao, Junhong Liu +5
Task completion in digital and physical environments increasingly involves complex temporal interaction, where actions and observations unfold over different time scales rather tha…
Bayesian Gated Non-Negative Contrastive Learning
Peng Cui, Jiahao Zhang, Lijie Hu
While Contrastive Learning (CL) has revolutionized self-supervised representation learning, its latent representations remain highly entangled and opaque, limiting their interpreta…
RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generation
Jiahao Zhang, Joseph Liu, Young-Yoon Lee +9
Success in generative modeling across language, image, and video demonstrates that large, well-curated datasets are the key driver for building capable models. 3D Human motion, how…
From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning
Haodong Wu, Jiahao Zhang, Lijie Hu +1
Supervised fine-tuning (SFT) data selection is commonly formulated as instance ranking: score each example and retain a top- subset. However, effective SFT training subsets are…
AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects
Danrui Li, Jiahao Zhang, Bernhard Egger +4
Assembling objects from parts requires understanding multimodal instructions, linking them to 3D components, and predicting physically plausible 6-DoF motions for each assembly ste…
Manual-PA: Learning 3D Part Assembly from Instruction Diagrams
Jiahao Zhang, Anoop Cherian, Cristian Rodriguez +2
Assembling furniture amounts to solving the discrete-continuous optimization task of selecting the furniture parts to assemble and estimating their connecting poses in a physically…
Temporally Grounding Instructional Diagrams in Unconstrained Videos
Jiahao Zhang, Frederic Z. Zhang, Cristian Rodriguez +3
We study the challenging problem of simultaneously localizing a sequence of queries in the form of instructional diagrams in a video. This requires understanding not only the indiv…
GoferBot: A Visual Guided Human-Robot Collaborative Assembly System
Zheyu Zhuang, Yizhak Ben-Shabat, Jiahao Zhang +2
The current transformation towards smart manufacturing has led to a growing demand for human-robot collaboration (HRC) in the manufacturing process. Perceiving and understanding th…
Aligning Step-by-Step Instructional Diagrams to Video Demonstrations
Jiahao Zhang, Anoop Cherian, Yanbin Liu +3
Multimodal alignment facilitates the retrieval of instances from one modality when queried using another. In this paper, we consider a novel setting where such an alignment is betw…