13 papers
FlowAct-R2: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning
Ziyao Huang, Zhengkun Rong, Shiyang Qin +5
We present FlowAct-R2, a framework for interactive humanoid video generation that combines continuous multimodal control with proactive agent planning. Our method consists of two c…
Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools
Xiuwei Chen, Quanlin Chen, Wentao Hu +8
Recent multimodal large language models (MLLMs) have made remarkable progress on fine-grained perception tasks under the "Thinking with Images" (TwI) paradigm by iteratively perfor…
Latent Visual States for Efficient Multimodal Reasoning
Xiuwei Chen, Wentao Hu, Yongxin Wang +8
The integration of visual evidence has significantly enhanced the capabilities of large multimodal models. However, this integration predominantly relies on generating discrete out…
Regret Pre-training: Bridging Prior and Posterior Views for Enhanced Knowledge Grounding
Mingkuan Zhao, Xiayu Sun, Wentao Hu +5
Causal language models factorize sequence probabilities using only preceding context, leaving future information unexploited during training despite its availability in the trainin…
Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization
Mingkuan Zhao, Wentao Hu, Tianchen Huang +6
Hallucination in Large Language Models (LLMs), characterized by the generation of content inconsistent with contextual facts or logical constraints -- remains a persistent challeng…
Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time
Mingkuan Zhao, Yide Gao, Wentao Hu +6
Large Language Models (LLMs) frequently exhibit "contextual disregard" when faced with input evidence that conflicts with their internal parametric memory, leading to persistent fa…
WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark
Wang Lin, Feng Wang, Majun Zhang +7
Recent advances in image editing models have demonstrated remarkable capabilities in executing explicit instructions, such as attribute manipulation, style transfer, and pose synth…
Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis
Yikang Ding, Jiwen Liu, Wenyuan Zhang +11
Recent advances in audio-driven avatar video generation have significantly enhanced audio-visual realism. However, existing methods treat instruction conditioning merely as low-lev…
VIR-Bench: Evaluating Geospatial and Temporal Understanding of MLLMs via Travel Video Itinerary Reconstruction
Hao Wang, Eiki Murata, Lingfang Zhang +11
Recent advances in multimodal large language models (MLLMs) have significantly enhanced video understanding capabilities, opening new possibilities for practical applications. Yet…
SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning
Xiuwei Chen, Wentao Hu, Hanhui Li +9
Recent advances in multimodal large language models (MLLMs) have shown impressive reasoning capabilities. However, further enhancing existing MLLMs necessitates high-quality vision…
Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning
Bohan Wang, Zhongqi Yue, Fengda Zhang +15
We completely discard the conventional spatial prior in image representation and introduce a novel discrete visual tokenizer: Self-consistency Tokenizer (Selftok). At its design co…
On Path to Multimodal Generalist: General-Level and General-Bench
Hao Fei, Yuan Zhou, Juncheng Li +29
The Multimodal Large Language Model (MLLM) is currently experiencing rapid growth, driven by the advanced capabilities of LLMs. Unlike earlier specialists, existing MLLMs are evolv…
Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learning
Wang Lin, Liyu Jia, Wentao Hu +6
Despite recent progress in video generation, producing videos that adhere to physical laws remains a significant challenge. Traditional diffusion-based methods struggle to extrapol…