6 papers
Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMs
Haoyuan Li, Yanpeng Zhou, Yufei Gao +7
Remarkable progress in 2D Vision-Language Models (VLMs) has spurred interest in extending them to 3D settings for tasks like 3D Question Answering, Dense Captioning, and Visual Gro…
Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models?
Kun Xiang, Zhili Liu, Zihao Jiang +13
In this paper, we address the challenging task of multimodal mathematical reasoning by incorporating the ability of "slow thinking" into multimodal large language models (MLLMs). O…
UniGS: Unified Language-Image-3D Pretraining with Gaussian Splatting
Haoyuan Li, Yanpeng Zhou, Tao Tang +5
Recent advancements in multi-modal 3D pre-training methods have shown promising efficacy in learning joint representations of text, images, and point clouds. However, adopting poin…
TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba
Xiuwei Chen, Wentao Hu, Xiao Dong +7
Transformer-based architectures have become the backbone of both uni-modal and multi-modal foundation models, largely due to their scalability via attention mechanisms, resulting i…
VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation
Youpeng Wen, Junfan Lin, Yi Zhu +4
Recent advancements utilizing large-scale video data for learning video generation models demonstrate significant potential in understanding complex physical dynamics. It suggests…
EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation
Cong Wang, Jiaxi Gu, Panwen Hu +5
Following the advancements in text-guided image generation technology exemplified by Stable Diffusion, video generation is gaining increased attention in the academic community. Ho…