53 citations · 119 across the 20 of their papers we have counts for
23 papers
MM-ContextFold: Context Folding for Multimodal Agentic Retrieval
Yang Tian, Fan Liu, Jingyuan Zhang +3
Multimodal Agentic Retrieval (MAR) requires agents to solve complex information-seeking tasks by iteratively invoking external tools. Typical frameworks such as ReAct maintain raw…
ChartLens: A Dual-Branch Framework for Chart Data Correction and Factual Summary Refinement
Hao Liu, Ruping Cao, Kun Wang +4
In this report, we present our champion solution for the DataMFM Challenge Track 2: Chart Understanding. This track requires models to recover structured chart data and generate fa…
ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval
Zixu Li, Yupeng Hu, Zhiwei Chen +3
The Composed Image Retrieval (CIR) task provides a flexible retrieval paradigm via a reference image and modification text, but it heavily relies on expensive and error-prone tripl…
UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval
Haokun Wen, Xuemeng Song, Haoyu Zhang +3
Composed image retrieval, multi-turn composed image retrieval, and composed video retrieval all share a common paradigm: composing the reference visual with modification text to re…
Open Multimodal Retrieval-Augmented Factual Image Generation
Yang Tian, Fan Liu, Jingyuan Zhang +3
Large Multimodal Models (LMMs) have achieved remarkable progress in generating photorealistic and prompt-aligned images, but they often produce outputs that contradict verifiable k…
OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval
Zhiwei Chen, Yupeng Hu, Zixu Li +3
Composed Image Retrieval (CIR) represents a novel retrieval paradigm that is capable of expressing users' intricate retrieval requirements flexibly. It enables the user to give a m…
CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG
Yang Tian, Fan Liu, Jingyuan Zhang +3
Multimodal Retrieval-Augmented Generation (MMRAG) has been introduced to enhance Multimodal Large Language Models by incorporating externally retrieved multimodal knowledge, but it…
KEPLA: A Knowledge-Enhanced Deep Learning Framework for Accurate Protein-Ligand Binding Affinity Prediction
Han Liu, Keyan Ding, Peilin Chen +4
Accurate prediction of protein-ligand binding affinity is critical for drug discovery. While recent deep learning approaches have demonstrated promising results, they often rely so…
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
Wenqi Liu, Xuemeng Song, Jiaxi Li +4
Direct Preference Optimization (DPO) has emerged as an effective approach for mitigating hallucination in Multimodal Large Language Models (MLLMs). Although existing methods have a…
FineCIR: Explicit Parsing of Fine-Grained Modification Semantics for Composed Image Retrieval
Zixu Li, Zhiheng Fu, Yupeng Hu +3
Composed Image Retrieval (CIR) facilitates image retrieval through a multimodal query consisting of a reference image and modification text. The reference image defines the retriev…
Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval
Haoqiang Lin, Haokun Wen, Xuemeng Song +3
Composed Image Retrieval (CIR) allows users to search target images with a multimodal query, comprising a reference image and a modification text that describes the user's modifica…
A Comprehensive Survey on Composed Image Retrieval
Xuemeng Song, Haoqiang Lin, Haokun Wen +3
Composed Image Retrieval (CIR) is an emerging yet challenging task that allows users to search for target images using a multimodal query, comprising a reference image and a modifi…
TIME: Temporal-Sensitive Multi-Dimensional Instruction Tuning and Robust Benchmarking for Video-LLMs
Yunxiao Wang, Meng Liu, Wenqi Liu +7
Video large language models have achieved remarkable performance in tasks such as video question answering, however, their temporal understanding remains suboptimal. To address thi…
Vision-guided and Mask-enhanced Adaptive Denoising for Prompt-based Image Editing
Kejie Wang, Xuemeng Song, Meng Liu +2
Text-to-image diffusion models have demonstrated remarkable progress in synthesizing high-quality images from text prompts, which boosts researches on prompt-based image editing th…
GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching
Xiao Han, Zijian Zhang, Xiangyu Zhao +6
As urban residents demand higher travel quality, vehicle dispatch has become a critical component of online ride-hailing services. However, current vehicle dispatch systems struggl…
Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrieval
Haokun Wen, Xuemeng Song, Xiaolin Chen +3
Composed image retrieval (CIR) aims to retrieve the target image based on a multimodal query, i.e., a reference image paired with corresponding modification text. Recent CIR studie…
MMGRec: Multimodal Generative Recommendation with Transformer Model
Han Liu, Yinwei Wei, Xuemeng Song +3
Multimodal recommendation aims to recommend user-preferred candidates based on her/his historically interacted items and associated multimodal information. Previous studies commonl…
Interactive Garment Recommendation with User in the Loop
Federico Becattini, Xiaolin Chen, Andrea Puccia +4
Recommending fashion items often leverages rich user profiles and makes targeted suggestions based on past history and previous purchases. In this paper, we work under the assumpti…
Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue
Kun Ouyang, Liqiang Jing, Xuemeng Song +3
Sarcasm Explanation in Dialogue (SED) is a new yet challenging task, which aims to generate a natural language explanation for the given sarcastic dialogue that involves multiple m…
VK-G2T: Vision and Context Knowledge enhanced Gloss2Text
Liqiang Jing, Xuemeng Song, Xinxing Zu +3
Existing sign language translation methods follow a two-stage pipeline: first converting the sign language video to a gloss sequence (i.e. Sign2Gloss) and then translating the gene…
Visual Self-paced Iterative Learning for Unsupervised Temporal Action Localization
Yupeng Hu, Han Jiang, Hao Liu +3
Recently, temporal action localization (TAL) has garnered significant interest in information retrieval community. However, existing supervised/weakly supervised methods are heavil…
An Empirical Study of Frame Selection for Text-to-Video Retrieval
Mengxia Wu, Min Cao, Yang Bai +4
Text-to-video retrieval (TVR) aims to find the most relevant video in a large video gallery given a query text. The intricate and abundant context of the video challenges the perfo…
Target-Guided Composed Image Retrieval
Haokun Wen, Xian Zhang, Xuemeng Song +2
Composed image retrieval (CIR) is a new and flexible image retrieval paradigm, which can retrieve the target image for a multimodal query, including a reference image and its corre…