57 citations · 202 across the 49 of their papers we have counts for
35 papers · 1 filter
GeoReader: Learning Latent Geometry for Spatial Reasoning in Vision-Language Models
Rong Li, Shijie Li, Jiangyong Huang +7
Vision-language models (VLMs) struggle with metric and multi-view relationships that remain implicit in RGB observations. Geometry foundation models recover this structure from RGB…
LARA: Latent Action Representation Alignment for Vision-Language-Action Models
Mengya Liu, Baoxiong Jia, Jiangyong Huang +2
Visual-language action (VLA) models enable robots to predict actions directly from observations and language instructions, but their performance depends on large-scale, high-qualit…
Lifting Unlabeled Internet-level Data for 3D Scene Understanding
Yixin Chen, Yaowei Zhang, Huangyue Yu +9
Annotated 3D scene data is scarce and expensive to acquire, while abundant unlabeled videos are readily available on the internet. In this paper, we demonstrate that carefully desi…
ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models
Yanpeng Zhao, Wentao Ding, Hongtao Li +2
A recent trend in vision-language models (VLMs) has been to enhance their spatial cognition for embodied domains. Despite progress, existing evaluations have been limited both in p…
3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding
Xiongkun Linghu, Jiangyong Huang, Baoxiong Jia +1
Reinforcement Learning with Verifiable Rewards ( RLVR ) has emerged as a transformative paradigm for enhancing the reasoning capabilities of Large Language Models ( LLMs), yet its…
SceneCOT: Eliciting Grounded Chain-of-Thought Reasoning in 3D Scenes
Xiongkun Linghu, Jiangyong Huang, Ziyu Zhu +2
Existing research on 3D Large Language Models (LLMs) still struggles to achieve grounded question-answering, primarily due to the under-exploration of the mechanism of human-like s…
VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Video
Yu Liu, Baoxiong Jia, Ruijie Lu +5
Building digital twins of articulated objects from monocular video presents an essential challenge in computer vision, which requires simultaneous reconstruction of object geometry…
Spatial-Temporal Multi-Scale Quantization for Flexible Motion Generation
Zan Wang, Jingze Zhang, Yixin Chen +3
Despite significant advancements in human motion generation, current motion representations, typically formulated as discrete frame sequences, still face two critical limitations:…
Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation
Ziyu Zhu, Xilin Wang, Yixuan Li +9
Embodied scene understanding requires not only comprehending visual-spatial information that has been observed but also determining where to explore next in the 3D physical world.…
LEO-VL: Efficient Scene Representation for Scalable 3D Vision-Language Learning
Jiangyong Huang, Xiaojian Ma, Xiongkun Linghu +6
Developing vision-language models (VLMs) capable of understanding 3D scenes has been a longstanding research goal. Despite recent progress, 3D VLMs still struggle with spatial reas…
MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans
Huangyue Yu, Baoxiong Jia, Yixin Chen +9
Embodied AI (EAI) research requires high-quality, diverse 3D scenes to effectively support skill acquisition, sim-to-real transfer, and generalization. Achieving these quality stan…
Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understanding
Yan Wang, Baoxiong Jia, Ziyu Zhu +1
Open-vocabulary 3D scene understanding is pivotal for enhancing physical intelligence, as it enables embodied agents to interpret and interact dynamically within real-world environ…
Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis
Jiangyong Huang, Baoxiong Jia, Yan Wang +5
Existing 3D vision-language (3D-VL) benchmarks fall short in evaluating 3D-VL models, creating a "mist" that obscures rigorous insights into model capabilities and 3D-VL tasks. Thi…
ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting
Yu Liu, Baoxiong Jia, Ruijie Lu +3
Building articulated objects is a key challenge in computer vision. Existing methods often fail to effectively integrate information across different object states, limiting the ac…
ARFlow: Human Action-Reaction Flow Matching with Physical Guidance
Wentao Jiang, Jingya Wang, Kaiyang Ji +3
Human action-reaction synthesis, a fundamental challenge in modeling causal human interactions, plays a critical role in applications ranging from virtual reality to social robotic…
MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes
Ruijie Lu, Yixin Chen, Junfeng Ni +5
Repurposing pre-trained diffusion models has been proven to be effective for NVS. However, these methods are mostly limited to a single object; directly applying such methods to co…
Multi-modal Situated Reasoning in 3D Scenes
Xiongkun Linghu, Jiangyong Huang, Xuesong Niu +3
Situation awareness is essential for understanding and reasoning about 3D scenes in embodied AI agents. However, existing datasets and benchmarks for situated understanding are lim…
SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields
Yu Liu, Baoxiong Jia, Yixin Chen +1
The ability to distill object-centric abstractions from intricate visual scenes underpins human-level generalization. Despite the significant progress in object-centric learning me…
PhysPart: Physically Plausible Part Completion for Interactable Objects
Rundong Luo, Haoran Geng, Congyue Deng +5
Interactable objects are ubiquitous in our daily lives. Recent advances in 3D generative models make it possible to automate the modeling of these objects, benefiting a range of ap…
Task-oriented Sequential Grounding and Navigation in 3D Scenes
Zhuofan Zhang, Ziyu Zhu, Junhao Li +8
Grounding natural language in 3D environments is a critical step toward achieving robust 3D vision-language alignment. Current datasets and models for 3D visual grounding predomina…
Unifying 3D Vision-Language Understanding via Promptable Queries
Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma +6
A unified model for 3D vision-language (3D-VL) understanding is expected to take various scene representations and perform a wide range of tasks in a 3D scene. However, a considera…
PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI
Yandan Yang, Baoxiong Jia, Peiyuan Zhi +1
With recent developments in Embodied Artificial Intelligence (EAI) research, there has been a growing demand for high-quality, large-scale interactive scene generation. While prior…
Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance
Zan Wang, Yixin Chen, Baoxiong Jia +7
Despite significant advancements in text-to-motion synthesis, generating language-guided human motion within 3D environments poses substantial challenges. These challenges stem pri…
SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
Baoxiong Jia, Yixin Chen, Huangyue Yu +5
3D vision-language grounding, which focuses on aligning language with the 3D physical environment, stands as a cornerstone in the development of embodied agents. In comparison to r…