11 citations · 19 across the 18 of their papers we have counts for
20 papers
MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework
Haowen Xiang, Yibo Yan, Jiahao Huo +4
Multi-vector visual document retrievers achieve strong fine-grained matching by representing each page with multiple vectors from deep Vision-Language Models (VLMs), but this desig…
SAMark: A Self-Anchored Text Watermarking with Paragraph-Level Paraphrase Robustness
Jiahao Huo, Wenjie Qu, Yibo Yan +5
Semantic-level watermarking (SWM) improves robustness against text modifications by treating sentences as the basic unit. However, robustness to paragraph-level paraphrasing remain…
Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval
Yibo Yan, Mingdong Ou, Yi Cao +5
Multi-vector models dominate Visual Document Retrieval (VDR) due to their fine-grained matching capabilities, but their high storage and computational costs present a major barrier…
Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations
Yibo Yan, Mingdong Ou, Yi Cao +6
Harnessing the full potential of visually-rich documents requires retrieval systems that understand not just text, but intricate layouts, a core challenge in Visual Document Retrie…
EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next
Ye Pan, Chi Kit Wong, Yuanhuiyi Lyu +8
Egocentric video provides a natural modality for studying human behavior, but conventional visual understanding captures mainly observable scenes, objects, and actions rather than…
Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval
Yibo Yan, Jiahao Huo, Guanbo Feng +12
With the rapid proliferation of multimodal information, Visual Document Retrieval (VDR) has emerged as a critical frontier in bridging the gap between unstructured visually rich da…
Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework
Yibo Yan, Mingdong Ou, Yi Cao +5
Visual Document Retrieval (VDR), which aims to retrieve relevant pages within vast corpora of visually-rich documents, is of significance in current multimodal retrieval applicatio…
Memory in the Age of AI Agents
Yuyang Hu, Shichun Liu, Yanwei Yue +44
Memory has emerged, and will continue to remain, a core capability of foundation model-based agents. As research on agent memory rapidly expands and attracts unprecedented attentio…
CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding
Jiahao Huo, Yu Huang, Yibo Yan +7
Although Multimodal Large Language Models (MLLMs) have shown remarkable potential in Visual Document Retrieval (VDR) through generating high-quality multi-vector embeddings, the su…
Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models
NaHyeon Park, Na Min An, Kunhee Kim +3
Text-to-image (T2I) systems increasingly rely on Large Language Model (LLM)-based text conditioning to interpret and expand user prompts. While this improves prompt understanding a…
PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraints
Jiahao Huo, Shuliang Liu, Bin Wang +5
Semantic-level watermarking (SWM) for large language models (LLMs) enhances watermarking robustness against text modifications and paraphrasing attacks by treating the sentence as…
Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis
Haoming Huang, Yibo Yan, Jiahao Huo +4
Large Language Models (LLMs), despite their remarkable capabilities, are hampered by hallucinations. A particularly challenging variant, knowledge overshadowing, occurs when one pi…
MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection
Yibo Yan, Shen Wang, Jiahao Huo +3
Mathematical error detection in educational settings presents a significant challenge for Multimodal Large Language Models (MLLMs), requiring a sophisticated understanding of both…
EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models
Jiamin Su, Yibo Yan, Fangteng Fu +6
Automated Essay Scoring (AES) plays a crucial role in educational assessment by providing scalable and consistent evaluations of writing tasks. However, traditional AES systems fac…
MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models
Jiahao Huo, Yibo Yan, Xu Zheng +4
Recent progress in Machine Unlearning (MU) has introduced solutions for the selective removal of private or sensitive information encoded within deep neural networks. Nonetheless,…
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning
Yibo Yan, Shen Wang, Jiahao Huo +7
Scientific reasoning, the process through which humans apply logic, evidence, and critical thinking to explore and interpret scientific phenomena, is essential in advancing knowled…
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey
Yunkai Dang, Kaichen Huang, Jiahao Huo +11
The rapid development of Artificial Intelligence (AI) has revolutionized numerous fields, with large language models (LLMs) and computer vision (CV) systems driving advancements in…
MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models
Kaichen Huang, Jiahao Huo, Yibo Yan +3
In recent years, multimodal large language models (MLLMs) have significantly advanced, integrating more modalities into diverse applications. However, the lack of explainability re…
ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection
Yibo Yan, Shen Wang, Jiahao Huo +13
As the field of Multimodal Large Language Models (MLLMs) continues to evolve, their potential to revolutionize artificial intelligence is particularly promising, especially in addr…
MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model
Jiahao Huo, Yibo Yan, Boren Hu +2
Projecting visual features into word embedding space has become a significant fusion strategy adopted by Multimodal Large Language Models (MLLMs). However, its internal mechanisms…