49 citations · 144 across the 32 of their papers we have counts for
36 papers
I-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis
Xiaoting Xie, Feng Qiao, Xianwei Zhang +5
Multimodal sentiment analysis (MSA) aims to infer sentiment by integrating linguistic, acoustic, and visual information. However, many fusion methods integrate modality-specific re…
Optimizing VLP-aligned Multimodal Intent Representation with Correct Visual Instantiation for Zero-Shot Composed Image Retrieval
Xuri Ge, Chunhao Wang, Junchen Fu +7
ZS-CIR aims to retrieve a target image from a reference image and a modification text without paired supervision, typically by encoding composed queries as text-dominant representa…
Tracing Query Expansion Effects through Sparse Autoencoder Features
Fangan Dong, Weiran Shi, Zhiwei Xu +5
Query expansion (QE) is a critical technique in information retrieval that enriches underspecified queries with additional textual context. However, its effect is often unreliable…
A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resources
Xuri Ge, Tianshuo Zhang, Ruihan Li +6
Facial state analysis plays a crucial role in understanding human expressions, psychological modeling, and human computer interaction. Traditional unimodal vision-based methods are…
Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
Kaiwen Zheng, Junchen Fu, Wenhao Deng +3
Recent advances in multimodal large language models (MLLMs) have significantly improved the performance of multimodal emotion recognition (MER) and enabled interpretable descriptio…
Stream-aware Side Adaptation for Large Pre-trained Multimodal Embedding Models in Sequential Recommendation
Junchen Fu, Kaiwen Zheng, Ioannis Arapakis +4
Recently, large pretrained multimodal embedding models such as Qwen3-VL Embedding have shown strong promise for sequential recommendation, as they provide reusable semantic item re…
DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation
Miduo Cui, Haochen Wang, Shangqin Mao +6
Auto-bidding is a core component of real-time advertising systems, where decisions must optimize long-term performance under budget and cost constraints, while online exploration i…
The 2nd EReL@MIR Workshop on Efficient Representation Learning for Multimodal Information Retrieval
Junchen Fu, Xuri Ge, Xin Xin +6
Multimodal representation learning has attracted increasing attention in AI, driven by the strong performance of large, pretrained multimodal foundation models such as Qwen, LLaVA,…
MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval
Xuri Ge, Chunhao Wang, Xindi Wang +3
Composed Image Retrieval (CIR) aims to retrieve target images based on a reference image and modified texts. However, existing methods often struggle to extract the correct semanti…
Hierarchical Dual-Change Collaborative Learning for UAV Scene Change Captioning
Fuhai Chen, Pengpeng Huang, Junwen Wu +4
This paper proposes a novel task for UAV scene understanding - UAV Scene Change Captioning (UAV-SCC) - which aims to generate natural language descriptions of semantic changes in d…
Benchmarking Multimodal Large Language Models for Missing Modality Completion in Product Catalogues
Junchen Fu, Wenhao Deng, Kaiwen Zheng +5
Missing-modality information on e-commerce platforms, such as absent product images or textual descriptions, often arises from annotation errors or incomplete metadata, impairing b…
Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering
Fangan Dong, Zuming Yan, Xuri Ge +7
Despite the strong reasoning capabilities of recent large language models (LLMs), achieving reliable performance on challenging tasks often requires post-training or computationall…
Differentiable Semantic ID for Generative Recommendation
Junchen Fu, Xuri Ge, Alexandros Karatzoglou +4
Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content. Most existing methods treat SIDs as…
Reinforced Efficient Reasoning via Semantically Diverse Exploration
Ziqi Zhao, Zhaochun Ren, Jiahong Zou +9
Reinforcement learning with verifiable rewards (RLVR) has proven effective in enhancing the reasoning of large language models (LLMs). Monte Carlo Tree Search (MCTS)-based extensio…
Focal-RegionFace: Generating Fine-Grained Multi-attribute Descriptions for Arbitrarily Selected Face Focal Regions
Kaiwen Zheng, Junchen Fu, Songpei Xu +4
In this paper, we introduce an underexplored problem in facial analysis: generating and recognizing multi-attribute natural language descriptions, containing facial action units (A…
The 1st EReL@MIR Workshop on Efficient Representation Learning for Multimodal Information Retrieval
Junchen Fu, Xuri Ge, Xin Xin +5
Multimodal representation learning has garnered significant attention in the AI community, largely due to the success of large pre-trained multimodal foundation models like LLaMA,…
Multimodal Representation Learning Techniques for Comprehensive Facial State Analysis
Kaiwen Zheng, Xuri Ge, Junchen Fu +2
Multimodal foundation models have significantly improved feature representation by integrating information from multiple modalities, making them highly suitable for a broader set o…
CROSSAN: Towards Efficient and Effective Adaptation of Multiple Multimodal Foundation Models for Sequential Recommendation
Junchen Fu, Yongxin Ni, Joemon M. Jose +4
In this paper, we explore a less-studied yet practically important problem: how to efficiently and effectively adapt multiple (2) multimodal foundation models (MFMs) for the seq…
LLMPopcorn: Exploring LLMs as Assistants for Popular Micro-video Generation
Junchen Fu, Xuri Ge, Kaiwen Zheng +5
In an era where micro-videos dominate platforms like TikTok and YouTube, AI-generated content is nearing cinematic quality. The next frontier is using large language models (LLMs)…
Multimodal Sentiment Analysis Based on Causal Reasoning
Fuhai Chen, Pengpeng Huang, Xuri Ge +2
With the rapid development of multimedia, the shift from unimodal textual sentiment analysis to multimodal image-text sentiment analysis has obtained academic and industrial attent…
R^3AG: First Workshop on Refined and Reliable Retrieval Augmented Generation
Zihan Wang, Xuri Ge, Joemon M. Jose +4
Retrieval-augmented generation (RAG) has gained wide attention as the key component to improve generative models with external knowledge augmentation from information retrieval. It…
Efficient and Effective Adaptation of Multimodal Foundation Models in Sequential Recommendation
Junchen Fu, Xuri Ge, Xin Xin +5
Multimodal foundation models (MFMs) have revolutionized sequential recommender systems through advanced representation learning. While Parameter-efficient Fine-tuning (PEFT) is com…
Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning
Xuri Ge, Junchen Fu, Fuhai Chen +3
Facial action units (AUs), as defined in the Facial Action Coding System (FACS), have received significant research interest owing to their diverse range of applications in facial…
Hire: Hybrid-modal Interaction with Multiple Relational Enhancements for Image-Text Matching
Xuri Ge, Fuhai Chen, Songpei Xu +3
Image-text matching (ITM) is a fundamental problem in computer vision. The key issue lies in jointly learning the visual and textual representation to estimate their similarity acc…