13 papers
VGGA: Geometry-Grounded Autoencoder for Video Generation
Chaofan Gan, Yian Zhao, Shijie Li +4
Autoencoders are the core component of latent video generative models, compressing high-dimensional videos into compact latent spaces for downstream generation. However, existing v…
Awakening Diffusion Transformers: Eliciting Stronger Generation and Understanding via Massive Activation Modulation
Chaofan Gan, Zicheng Zhao, Yuanpeng Tu +6
Massive Activations (MAs) have been widely observed in Transformer-based models, yet their structure and functional roles in Diffusion Transformers (DiTs) remain insufficiently und…
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
Shijie Li, Yilin Gao, Siyuan Yang +7
Multimodal Large Language Models (MLLMs) are often constrained by a language-space bottleneck, forcing complex visual reasoning into discrete tokens which can lose perceptual nuanc…
GRACE: Boosting Video MLLMs with Grounded Action-Centric Evidence for Viewer Sentiment Prediction
Ruoxuan Yang, Tieyuan Chen, Xiaofeng Huang +6
Viewer sentiment prediction in video advertisements aims to infer the latent affective response evoked in the audience. To bridge the gap between what is shown and what is felt, mo…
From Priors to Perception: Grounding Video-LLMs in Physical Reality
Zicheng Zhao, Chaofan Gan, Shijie Li +1
Video Large Language Models (Video-LLMs) excel in general video understanding but often base physical judgments on event expectations rather than observations. We find that they no…
VidLaDA: Bidirectional Diffusion Large Language Models for Efficient Video Understanding
Zhihao He, Tieyuan Chen, Kangyu Wang +6
Current Video Large Language Models (Video LLMs) typically encode frames via a vision encoder and employ an autoregressive (AR) LLM for understanding and generation. However, this…
Massive Activations are the Key to Local Detail Synthesis in Diffusion Transformers
Chaofan Gan, Zicheng Zhao, Yuanpeng Tu +5
Diffusion Transformers (DiTs) have recently emerged as a powerful backbone for visual generation. Recent observations reveal \emph{Massive Activations} (MAs) in their internal feat…
DND: Boosting Large Language Models with Dynamic Nested Depth
Tieyuan Chen, Xiaodong Chen, Haoxing Chen +3
We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs by selecting critical tokens to reprocess in a nested depth manner. Specifi…
CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
Kangyu Wang, Zhiyun Jiang, Haibo Feng +5
Diffusion large language models (dLLMs) generate text through iterative denoising. In commonly adopted parallel decoding schemes, each step confirms only high-confidence positions…
Enhancing Video Large Language Models with Structured Multi-Video Collaborative Reasoning
Zhihao He, Tianyao He, Yun Xu +5
Despite the prosperity of the video language model, the current pursuit of comprehensive video reasoning is thwarted by the inherent spatio-temporal incompleteness within individua…
MCA: 2D-3D Retrieval with Noisy Labels via Multi-level Adaptive Correction and Alignment
Gui Zou, Chaofan Gan, Chern Hong Lim +2
With the increasing availability of 2D and 3D data, significant advancements have been made in the field of cross-modal retrieval. Nevertheless, the existence of imperfect annotati…
CogStream: Context-guided Streaming Video Question Answering
Zicheng Zhao, Kangyu Wang, Shijie Li +3
Despite advancements in Video Large Language Models (Vid-LLMs) improving multimodal understanding, challenges persist in streaming video reasoning due to its reliance on contextual…
Looking Beyond Visible Cues: Implicit Video Question Answering via Dual-Clue Reasoning
Tieyuan Chen, Huabin Liu, Yi Wang +8
Video Question Answering (VideoQA) aims to answer natural language questions based on the given video, with prior work primarily focusing on identifying the duration of relevant se…