18 papers
GeoReader: Learning Latent Geometry for Spatial Reasoning in Vision-Language Models
Rong Li, Shijie Li, Jiangyong Huang +7
Vision-language models (VLMs) struggle with metric and multi-view relationships that remain implicit in RGB observations. Geometry foundation models recover this structure from RGB…
Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
Qinchuan Cheng, Zhantao Gong, Pengzhan Sun +2
Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself. We introduce Ego2World,…
EgoVine: Post-Training with Branching Experience Compiled from Real-World Videos
Qinchuan Cheng, Bowen Lyu, Zhantao Gong +3
Real-world egocentric videos capture multi-step household procedures, but their recorded trajectories leave alternative actions and recovery from the learner's own mistakes largely…
Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift
Pengzhan Sun, Shiu-hong Kao, Shijie Li +4
This paper studies \textbf{thinking--answer consistency} in vision-language models. We focus on Visual Intention Grounding, where a model infers a target object based on a human in…
MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning
Zirui Cheng, Xun Xu, Tiankai Chen +7
Few-shot in-context learning (ICL) with multi-modal large language models (MLLMs) enables task adaptation without parameter updates, but its performance is highly sensitive to the…
Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
Qinchuan Cheng, Zhantao Gong, Pengzhan Sun +3
Embodied agents in household environments must plan under partial observation: they need to remember objects, track state changes, and recover when actions fail. Existing benchmark…
Grounding by Remembering: Cross-Scene and In-Scene Memory for 3D Functional Affordances
Qirui Wang, Jingyi He, Yining Pan +2
Functional affordance grounding requires more than recognizing an object: an agent must localize the specific region that supports an interaction, such as the handle to pull or the…
PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments
Yunn Kang Lim, Pengzhan Sun, Ziyi Bai +4
When an LLM-based embodied agent fails at a household task, the culprit could be misidentified objects, forgotten sub-goals, or poor action sequencing -- yet existing benchmarks re…
PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Driving
Yining Pan, Shijie Li, Yuchen Wu +2
This paper presents the first study on Unsupervised Domain Adaptation (UDA) for multimodal 3D panoptic segmentation (mm-3DPS), aiming to improve generalization under domain shifts…
Perception-Aware Multimodal Spatial Reasoning from Monocular Images
Yanchun Cheng, Rundong Wang, Xulei Yang +4
Spatial reasoning from monocular images is essential for autonomous driving, yet current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception, partic…
MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images
Qirui Wang, Jingyi He, Yining Pan +3
Spatial reasoning (SR), the ability to infer 3D spatial information from 2D inputs, is essential for real-world applications such as embodied AI and autonomous driving. However, ex…
Thinking Ahead: Foresight Intelligence in MLLMs and World Model
Zhantao Gong, Liaoyuan Fan, Qing Guo +3
In this work, we define Foresight Intelligence as the capability to anticipate and interpret future events-an ability essential for applications such as autonomous driving, yet lar…
Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs
Yongyi Su, Haojie Zhang, Shijie Li +11
Multimodal large language models (MLLMs) have advanced rapidly in recent years. However, existing approaches for vision tasks often rely on indirect representations, such as genera…
DiffPCN: Latent Diffusion Model Based on Multi-view Depth Images for Point Cloud Completion
Zijun Li, Hongyu Yan, Shijie Li +4
Latent diffusion models (LDMs) have demonstrated remarkable generative capabilities across various low-level vision tasks. However, their potential for point cloud completion remai…
Zero-Shot 3D Visual Grounding from Vision-Language Models
Rong Li, Shijie Li, Lingdong Kong +2
3D Visual Grounding (3DVG) seeks to locate target objects in 3D scenes using natural language descriptions, enabling downstream applications such as augmented reality and robotics.…
Multi-View Industrial Anomaly Detection with Epipolar Constrained Cross-View Fusion
Yifan Liu, Xun Xu, Shijie Li +2
Multi-camera systems provide richer contextual information for industrial anomaly detection. However, traditional methods process each view independently, disregarding the compleme…
Future-Aware Interaction Network For Motion Forecasting
Shijie Li, Xun Xu, Si Yong Yeo +1
Motion forecasting is a crucial component of autonomous driving systems, enabling the generation of accurate and smooth future trajectories to ensure safe navigation to the destina…
SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding
Rong Li, Shijie Li, Lingdong Kong +2
3D Visual Grounding (3DVG) aims to locate objects in 3D scenes based on textual descriptions, essential for applications like augmented reality and robotics. Traditional 3DVG appro…