18 papers
Understanding NVIDIA Grace-Blackwell Unified Memory Through Placement-Controlled Microbenchmarks
Nathan Graddon, Aaron Jarmusch, Sunita Chandrasekaran
NVIDIA Grace--Blackwell systems make CPU--GPU memory increasingly unified at the programming level, but unified addressability does not make the memory system uniform. A CUDA manag…
Geometry Conditioning in an Embodied SLM: Training Controls and Robustness Diagnostics in a 0.8B Hybrid Model
Hao Li, Haofei Sun, Lin He
We study how physical-state inputs affect a 0.8B hybrid language model adapted for manipulation with 6.2M trainable parameters. Six conditions are trained on three LIBERO-Spatial t…
GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding
Hao Li, Han Fang, Zixin Pan +8
Although multimodal large language models (MLLMs) have achieved remarkable progress, understanding 3D spatial relationships from 2D images remains a critical challenge. Existing me…
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Tianyi Gao, Han Fang, Tianyi Ding +9
Multimodal Large Language Models have made great progress in grounding tasks, yet existing methods still struggle to unify precise localization and complex reasoning. For one thing…
DrivingDepth: Sparse-Prompted Pixel-wise Scale Correction for Driving Depth Estimation
Chi Huang, Wenhao Zhang, Hang Yin +5
Dense depth estimation for autonomous driving faces a geometry-scale conflict: depth foundation models deliver pixel-aligned dense visual geometry without reliable metric scale, wh…
Monte Carlo Energy Aggregation for Mobile 3D Gaussian Splatting
Xiaobiao Du, YuAn Wang, Hao Li +3
Recent advances in 3D Gaussian Splatting have demonstrated unprecedented success in novel view synthesis. However, the substantial inference and storage overhead driven by high-ord…
Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test
Chun-Kai Fan, Xiaowei Chi, Xiaozhu Ju +18
As world models gain momentum in Embodied AI, an increasing number of works explore using video foundation models as predictive world models for downstream embodied tasks like 3D p…
Envision: Embodied Visual Planning via Goal-Imagery Video Diffusion
Yuming Gu, Yizhi Wang, Yining Hong +9
Embodied visual planning aims to enable manipulation tasks by imagining how a scene evolves toward a desired goal and using the imagined trajectories to guide actions. Video diffus…
RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension
Tianyi Gao, Hao Li, Han Fang +8
Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description. Existing REC benchmarks primarily evaluate…
FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gain
YuAn Wang, Xiaofan Li, Chi Huang +5
In controllable driving-scene reconstruction and 3D scene generation, maintaining geometric fidelity while synthesizing visually plausible appearance under large viewpoint shifts i…
Any4D: Open-Prompt 4D Generation from Natural Language and Images
Hao Li, Qiao Sun
While video-generation-based embodied world models have gained increasing attention, their reliance on large-scale embodied interaction data remains a key bottleneck. The scarcity,…
WoW: Towards a World omniscient World model Through Embodied Interaction
Xiaowei Chi, Peidong Jia, Chun-Kai Fan +33
Humans develop an understanding of intuitive physics through active interaction with the world. This approach is in stark contrast to current video models, such as Sora, which rely…
Character Mixing for Video Generation
Tingting Liao, Chongjian Ge, Guangyi Liu +2
Imagine Mr. Bean stepping into Tom and Jerry--can we generate videos where characters interact naturally across different worlds? We study inter-character interaction in text-to-vi…
MatDecompSDF: High-Fidelity 3D Shape and PBR Material Decomposition from Multi-View Images
Chengyu Wang, Isabella Bennett, Henry Scott +4
We present MatDecompSDF, a novel framework for recovering high-fidelity 3D shapes and decomposing their physically-based material properties from multi-view images. The core challe…
Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing
Xiangyu Zhao, Peiyuan Zhang, Kexian Tang +10
Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but they still face challenges in General Visual Editing, particularly in…
Seeing Sound, Hearing Sight: Uncovering Modality Bias and Conflict of AI models in Sound Localization
Yanhao Jia, Ji Xie, S Jivaganesh +3
Imagine hearing a dog bark and turning toward the sound only to see a parked car, while the real, silent dog sits elsewhere. Such sensory conflicts test perception, yet humans reli…
Unified Reward Model for Multimodal Understanding and Generation
Yibin Wang, Yuhang Zang, Hao Li +2
Recent advances in human preference alignment have significantly improved multimodal generation and understanding. A key approach is to train reward models that provide supervision…
UniPLV: Towards Label-Efficient Open-World 3D Scene Understanding by Regional Visual Language Supervision
Yuru Wang, Pei Liu, Songtao Wang +7
Open-world 3D scene understanding is a critical challenge that involves recognizing and distinguishing diverse objects and categories from 3D data, such as point clouds, without re…