9 papers
HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers
Guozhen Zhang, Xuerui Qiu, Yutao Cui +11
Holistic visual tokenizers are fundamental to unified multimodal models (UMMs) as they map diverse visual inputs into a unified representation space. In this paper, we present HYDR…
4DPChat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping
Xindan Zhang, Weilong Yan, Yufei Shi +5
Point clouds provide a compact and expressive representation of 3D objects, and have recently been integrated into multimodal large language models (MLLMs). However, existing metho…
FaVChat: Hierarchical Prompt-Query Guided Facial Video Understanding with Data-Efficient GRPO
Fufangchen Zhao, Songbai Tan, Xuerui Qiu +7
Existing video large language models (VLLMs) primarily leverage prompt agnostic visual encoders, which extract untargeted facial representations without awareness of the queried in…
Why Do DiT Editors Drift? Plug-and-Play Low Frequency Alignment in VAE Latent Space
Xiaoce Wang, Sifan Zhou, Kaifei Wang +4
Recent advances in diffusion transformers (DiTs) have enabled promising single-turn image editing capabilities. However, multi-turn editing often leads to progressive semantic drif…
HYDRA: Unifying Multi-modal Generation and Understanding via Representation-Harmonized Tokenization
Xuerui Qiu, Yutao Cui, Guozhen Zhang +9
Unified Multimodal Models struggle to bridge the fundamental gap between the abstract representations needed for visual understanding and the detailed primitives required for gener…
Omni-View: Unlocking How Generation Facilitates Understanding in Unified 3D Model based on Multiview images
JiaKui Hu, Shanshan Zhao, Qing-Guo Chen +6
This paper presents Omni-View, which extends the unified multimodal understanding and generation to 3D scenes based on multiview images, exploring the principle that "generation fa…