29 papers
FrameMorrow: Future-guided Frame Selection with Prospective Tokens for Long-Horizon Video Generation
Bo Yin, Xiaobin Hu, Jiaqi Zhao +1
Long-horizon video generation requires models to effectively leverage an increasingly long generation history. As the generated history grows, retaining all previous content become…
EgoTSR++: Egocentric Spatiotemporal Reasoning for Task Progress Understanding
Xiaoda Yang, Can Wang, Yuxiang Liu +3
Vision-Language Models (VLMs) have advanced rapidly in static visual understanding, yet remain unreliable when judging how an egocentric task is progressing. Given a task instructi…
QuantWM: Temporally Consistent 2-Bit KV Cache Quantization for Video World Models
Jiaqi Zhao, Xiaobin Hu, Bo Yin +3
Video world models achieve long-range temporal consistency by storing KV cache during generation, but the growing cache makes KV cache memory a major deployment bottleneck, which m…
TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation
Xiaoda Yang, Yuxiang Liu, Kaiwen Zheng +12
Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal computation of large models…
ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
Jiahao Zhao, Xiaomin Yu, Zhongxiang Sun +5
Text-to-image (T2I) models can produce visually compelling images, yet they remain limited on open-world tasks that require complex semantic understanding, multi-step reasoning, an…
MambaADv2: Evolving Duality-enhanced State Space Model for Unsupervised Anomaly Detection
Xiaobin Hu, Haoyang He, Bo Yin +5
While recent advancements in anomaly detection have demonstrated the efficacy of CNN- and Transformer-based approaches, these architectures face inherent limitations: CNNs struggle…
SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMs
Bo Yin, Xiaobin Hu, Chengming Xu +6
Vision-language models (VLMs) often underperform on evidence intensive tasks because decisive visual evidence are small, localized, and easy to overlook, leading to failures in evi…
JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
Yinan Chen, Chuming Lin, Zhennan Chen +12
While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge t…
TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation
Kailin Lyu, Di Wu, Pengwei Zhang +12
Touch is a key modality for embodied agents to understand the physical world. Although recent work has incorporated tactile signals into language systems for tactile commonsense re…
Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation
Zhifei Xie, Kaiyu Pang, Haobin Zhang +4
Despite rapid advances in automatic speech recognition (ASR) and large audio-language models, robust recognition in real-world environments remains limited by an "acoustic robustne…
Anisotropic Modality Align
Xiaomin Yu, Yijiang Li, Yuhui Zhang +8
Training multimodal large language models has long been limited by the scarcity of high-quality paired multimodal data. Recent studies show that the shared representation space of…
Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generation
Fangda Ye, Zhifei Xie, Yuxin Hu +5
Recent agentic search frameworks enable deep research via iterative planning and retrieval, reducing hallucinations and enhancing factual grounding. However, they remain text-centr…
Evolution of Optimization Methods: Algorithms, Scenarios, and Evaluations
Tong Zhang, Jiangning Zhang, Zhucun Xue +9
Balancing convergence speed, generalization capability, and computational efficiency remains a core challenge in deep learning optimization. First-order gradient descent methods, e…
WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents
Yulin Chen, Tri Cao, Haoran Li +7
Web agents powered by vision-language models (VLMs) enable autonomous interaction with web environments by perceiving and acting on both visual and textual webpage content to accom…
From Perception to Planning: Evolving Ego-Centric Task-Oriented Spatiotemporal Reasoning via Curriculum Learning
Xiaoda Yang, Yuxiang Liu, Shenzhou Gao +8
Modern vision-language models achieve strong performance in static perception, but remain limited in the complex spatiotemporal reasoning required for embodied, egocentric tasks. A…
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
Xiaomin Yu, Yi Xin, Yuhui Zhang +12
Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of d…
Dual Latent Memory for Visual Multi-agent System
Xinlei Yu, Chengming Xu, Zhangquan Chen +8
While Visual Multi-Agent Systems (VMAS) promise to enhance comprehensive abilities through inter-agent collaboration, empirical evidence reveals a counter-intuitive "scaling wall":…
Multi-Dimensional Knowledge Profiling with Large-Scale Literature Database and Hierarchical Retrieval
Zhucun Xue, Jiangning Zhang, Juntao Jiang +6
The rapid expansion of research across machine learning, vision, and language has produced a volume of publications that is increasingly difficult to synthesize. Traditional biblio…
M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
Juntao Jiang, Jiangning Zhang, Yali Bi +7
Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models by encouraging step-by-step intermediate reasoning, and recent advances have extended this…
IRPO: Boosting Image Restoration via Post-training GRPO
Haoxuan Xu, Yi Liu, Tianfu Li +7
Post-training has become effective for high-level generation, but its role in low-level vision remains underexplored. Existing image restoration methods often rely on fixed pixel-w…
Boosting Reasoning in Large Multimodal Models via Activation Replay
Yun Xing, Xiaobin Hu, Qingdong He +4
Recently, Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as an effective approach to incentivizing reasoning capability in Large Multimodal Models (LMMs), while…
FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning
Bo Yin, Xiaobin Hu, Xingyu Zhou +7
Diffusion models have achieved remarkable success in generative modeling, yet how to effectively adapt large pretrained models to new tasks remains challenging. We revisit the reco…
VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models
Xinlei Yu, Chengming Xu, Guibin Zhang +7
Despite the remarkable success of Vision-Language Models (VLMs), their performance on a range of complex visual tasks is often hindered by a "visual processing bottleneck": a prope…
TokenAR: Multiple Subject Generation via Autoregressive Token-level enhancement
Haiyue Sun, Qingdong He, Jinlong Peng +5
Autoregressive Model (AR) has shown remarkable success in conditional image generation. However, these approaches for multiple reference generation struggle with decoupling differe…