6 citations · 11 across the 16 of their papers we have counts for
17 papers
WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation
Peterson Co, Sicheng Hu, Chunxuan Jiao +17
Action-conditioned world models (ACWMs) promise to provide embodied AI with scalable predictive simulators for planning, policy evaluation, and data generation. Realizing this prom…
Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models
Xiangkai Ma, Yue Ma, Junjie Wang +6
World Action Models (WAMs) aim to construct a unified architecture capable of understanding world state evolution and guiding to generative motion planning. However, existing visua…
JoyAI-Sim: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid
Peidong Liu, Yongce Liu, Songyan Guo +34
Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstra…
Towards Valid Student Simulation with Large Language Models
Zhihao Yuan, Yunze Xiao, Ming Li +4
This paper presents a conceptual and methodological framework for large language model (LLM) based student simulation in educational settings. The authors identify a core failure m…
Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents
Yueqi Song, Ketan Ramaneti, Zaid Sheikh +18
Public research results on large-scale supervised finetuning of AI agents remain relatively rare, since the collection of agent training data presents unique challenges. In this wo…
See the Forest and the Trees: A Synergistic Reasoning Framework for Knowledge-Based Visual Question Answering
Junjie Wang, Yunhan Tang, Yijie Wang +4
Multimodal Large Language Models (MLLMs) have pushed the frontiers of Knowledge-Based Visual Question Answering (KBVQA), yet their reasoning is fundamentally bottlenecked by a reli…
Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning with Lightweight Annotations
Junjie Wang, Zhihao Yuan, Shuyi Jiang +5
Large language models are increasingly being applied to 3D scene understanding. However, existing 3D-aware LLMs largely operate as black boxes: they produce bounding boxes or textu…
Empowering Large Language Models with 3D Situation Awareness
Zhihao Yuan, Yibo Peng, Jinke Ren +7
Driven by the great success of Large Language Models (LLMs) in the 2D image domain, their applications in 3D scene understanding has emerged as a new trend. A key difference betwee…
PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Models
Zilu Guo, Hongbin Lin, Zhihao Yuan +6
3D Multimodal Large Language Models (MLLMs) have recently made substantial advancements. However, their potential remains untapped, primarily due to the limited quantity and subopt…
Generative Semantic Communication for Text-to-Speech Synthesis
Jiahao Zheng, Jinke Ren, Peng Xu +5
Semantic communication is a promising technology to improve communication efficiency by transmitting only the semantic information of the source data. However, traditional semantic…
Instance-free Text to Point Cloud Localization with Relative Position Awareness
Lichao Wang, Zhihao Yuan, Jinke Ren +2
Text-to-point-cloud cross-modal localization is an emerging vision-language task critical for future robot-human collaboration. It seeks to localize a position from a city-scale po…
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance
Haiming Zhang, Zhihao Yuan, Chaoda Zheng +6
Although existing speech-driven talking face generation methods achieve significant progress, they are far from real-world application due to the avatar-specific training demand an…
Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding
Zhihao Yuan, Jinke Ren, Chun-Mei Feng +3
3D Visual Grounding (3DVG) aims at localizing 3D object based on textual descriptions. Conventional supervised methods for 3DVG often necessitate extensive annotations and a predef…
Toward Explainable and Fine-Grained 3D Grounding through Referring Textual Phrases
Zhihao Yuan, Xu Yan, Zhuo Li +4
Recent progress in 3D scene understanding has explored visual grounding (3DVG) to localize a target object through a language description. However, existing methods only consider t…
X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning
Zhihao Yuan, Xu Yan, Yinghong Liao +4
3D dense captioning aims to describe individual objects by natural language in 3D scenes, where 3D scenes are usually represented as RGB-D scans or point clouds. However, only expl…
Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation
Xu Yan, Zhihao Yuan, Yuhao Du +4
Visual Question Answering on 3D Point Cloud (VQA-3D) is an emerging yet challenging field that aims at answering various types of textual questions given an entire point cloud scen…
InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring
Zhihao Yuan, Xu Yan, Yinghong Liao +4
Compared with the visual grounding on 2D images, the natural-language-guided 3D object localization on point clouds is more challenging. In this paper, we propose a new model, name…