8 papers
Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval
Zelong Sun, Jun Wang, Kaicheng Yang +3
Unified multimodal retrieval aims to identify candidates that satisfy complex user intent expressed through heterogeneous inputs. Although Large Vision-Language Model (LVLM)-based…
UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards
Jun Wang, Shuo Tan, Zelong Sun +5
Retrieval-Augmented Generation (RAG) extends Large Vision-Language Models (LVLMs) with external visual knowledge. However, existing visual RAG systems typically rely on generic ret…
DanQing: An Up-to-Date Large-Scale Chinese Vision-Language Pre-training Dataset
Hengyu Shen, Tiancheng Gu, Bin Qin +10
Vision-Language Pre-training (VLP) models have achieved remarkable success by leveraging large-scale image-text pairs. While English-centric models like CLIP and SigLIP benefit fro…
UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning
Tiancheng Gu, Kaicheng Yang, Kaichen Zhang +6
Universal multimodal embedding models are foundational to various tasks. Existing approaches typically employ in-batch negative mining by measuring the similarity of query-candidat…
Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs
Tiancheng Gu, Kaicheng Yang, Ziyong Feng +6
The Contrastive Language-Image Pre-training (CLIP) framework has become a widely used approach for multimodal representation learning, particularly in image-text retrieval and clus…
ViCToR: Improving Visual Comprehension via Token Reconstruction for Pretraining LMMs
Yin Xie, Kaicheng Yang, Peirou Liang +7
Large Multimodal Models (LMMs) often face a modality representation gap during pretraining: while language embeddings remain stable, visual representations are highly sensitive to…