34 papers
Xiaomi-OCR-0 Technical Report
Xin Chen, Anan Du, Feng Feng +7
Compact OCR-specific vision-language models achieve strong document parsing performance, but often rely on costly supervision and focus primarily on visual-text reconstruction. We…
Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models
Xiaoyi Yu, Enver Sangineto, Pei Fu +6
Diffusion Large Language Models (dLLMs) have emerged as an efficient alternative to autoregressive models, yet aligning them via Reinforcement Learning (RL) requires likelihood sur…
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
Yiyang Fang, Pei Fu, Jinjie Li +7
Multimodal Large Language Models (MLLMs) often follow a fixed Think-then-Answer paradigm, which is inefficient in heterogeneous multitask settings because simple inputs may not req…
DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models
Pengjie Wang, Linger Deng, Zujia Zhang +6
Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visua…
ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval
Yuhan Liu, Pei Fu, Hang Li +8
Leveraging Multimodal Large Language Models (MLLMs) via contrastive learning has become a mainstream paradigm for improving the performance of Universal Multimodal Retrieval (UMR).…
UniTranslator: A Unified Multi-modal Framework for End-to-end In-Image Machine Translation
Jiahao Lyu, Pei Fu, Zhenhang Li +6
In-Image Machine Translation (IIMT) aims to translate scene text in an image and render the translated text back into the original regions while preserving the overall visual appea…
RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation
Feifei Bian, Zhimin Zheng, Wei Deng +2
Recent years have witnessed remarkable progress in image generation and editing, particularly regarding instruction following and visual fidelity. However, when handling ambiguous…
STAR: SpatioTemporal Adaptive Reward Allocation for Text-to-Image RL Post-Training
Jinjie Shen, Wei Deng, Xian Hu +2
Existing RL post-training methods for text-to-image generation usually convert the final-image reward into a single scalar advantage and apply it with the same strength to the enti…
Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization
Shizhe Xiang, Ke An, Wenlong Yu +5
Recent post-training methods, particularly Reinforcement Learning with Verifiable Rewards (RLVR), have significantly enhanced the reasoning ability of Large Vision-Language Models…
Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment
Yuchen Sun, Pei Fu, Shaojie Zhang +6
Test-Time Scaling (TTS), which samples multiple candidate actions and ranks them via a Critic Model, has emerged as a promising paradigm for generalist GUI agents. Its efficacy thu…
StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video
Ao Li, Zihan Xiao, Zihao Yue +7
Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to respond. This naturally introdu…
PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media
Fuhao Li, Shaofeng You, Jiagao Hu +6
Evaluating object removal in images and videos remains challenging because the task is inherently one-to-many, yet existing metrics frequently disagree with human perception. Full-…
Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models
Longwei Xu, Feng Feng, Shaojie Zhang +7
Optical Character Recognition (OCR) is increasingly regarded as a foundational capability for modern vision-language models (VLMs), enabling them not only to read text in images bu…
Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA
Yuanlei Zheng, Pei Fu, Hang Li +9
Multi-page Document Visual Question Answering requires reasoning over semantics, layouts, and visual elements in long, visually dense documents. Existing OCR-free methods face a tr…
IMTBench: A Multi-Scenario Cross-Modal Collaborative Evaluation Benchmark for In-Image Machine Translation
Jiahao Lyu, Pei Fu, Zhenhang Li +7
End-to-end In-Image Machine Translation (IIMT) aims to convert text embedded within an image into a target language while preserving the original visual context, layout, and render…
DashengTokenizer: One layer is enough for unified audio understanding and generation
Heinrich Dinkel, Xingwei Sun, Gang Li +8
This paper introduces DashengTokenizer, a continuous audio tokenizer engineered for joint use in both understanding and generation tasks. Unlike conventional approaches, which trai…
ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding
Yiran Guan, Sifan Tu, Dingkang Liang +6
Omni-modal reasoning is essential for intelligent systems to understand and draw inferences from diverse data sources. While existing omni-modal large language models (OLLM) excel…
PatchCue: Enhancing Vision-Language Model Reasoning with Patch-Based Visual Cues
Yukun Qi, Pei Fu, Hang Li +5
Vision-Language Models (VLMs) have achieved remarkable progress on a wide range of challenging multimodal understanding and reasoning tasks. However, existing reasoning paradigms,…
CoME: Empowering Channel-of-Mobile-Experts with Informative Hybrid-Capabilities Reasoning
Yuxuan Liu, Weikai Xu, Kun Huang +9
Mobile Agents can autonomously execute user instructions, which requires hybrid-capabilities reasoning, including screen summary, subtask planning, action decision and action funct…
Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
Yiran Guan, Liang Yin, Dingkang Liang +5
Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synch…
MSJoE: Jointly Evolving MLLM and Sampler for Efficient Long-Form Video Understanding
Wenhui Tan, Xiaoyi Yu, Jiaze Li +5
Efficiently understanding long-form videos remains a fundamental challenge for multimodal large language models (MLLMs). In this paper, we present MLLM-Sampler Joint Evolution (MSJ…
Federated Balanced Learning
Jiaze Li, Haoran Xu, Wanyi Wu +9
Federated learning is a paradigm of joint learning in which clients collaborate by sharing model parameters instead of data. However, in the non-iid setting, the global model exper…
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
Yiyang Fang, Wenke Huang, Pei Fu +5
Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of…
Visual Para-Thinker: Divide-and-Conquer Reasoning for Visual Comprehension
Haoran Xu, Hongyu Wang, Jiaze Li +5
Existing LLM test-time scaling laws emphasize the emergence of self-reflective behaviors through extended reasoning length. Nevertheless, this vertical scaling strategy often encou…