Spectrum-based Modality Representation Fusion Graph Convolutional Network for Multimodal Recommendation

Abstract

Incorporating multi-modal features as side information has recently become a trend in recommender systems. To elucidate user-item preferences, recent studies focus on fusing modalities via concatenation, element-wise sum, or attention mechanisms. Despite having notable success, existing approaches do not account for the modality-specific noise encapsulated within each modality. As a result, direct fusion of modalities will lead to the amplification of cross-modality noise. Moreover, the variation of noise that is unique within each modality results in noise alleviation and fusion being more challenging. In this work, we propose a new Spectrum-based Modality Representation (SMORE) fusion graph recommender that aims to capture both uni-modal and fusion preferences while simultaneously suppressing modality noise. Specifically, SMORE projects the multi-modal features into the frequency domain and leverages the spectral space for fusion. To reduce dynamic contamination that is unique to each modality, we introduce a filter to attenuate and suppress the modality noise adaptively while capturing the universal modality patterns effectively. Furthermore, we explore the item latent structures by designing a new multi-modal graph learning module to capture associative semantic correlations and universal fusion patterns among similar items. Finally, we formulate a new modality-aware preference module, which infuses behavioral features and balances the uni- and multi-modal features for precise preference modeling. This empowers SMORE with the ability to infer both user modality-specific and fusion preferences more accurately. Experiments on three real-world datasets show the efficacy of our proposed model. The source code for this work has been made publicly available at https://github.com/kennethorq/SMORE.

arXiv commentAccepted to ACM Web Search and Data Mining (WSDM) 2025

References in corpus (9)

• Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering
• Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation
• Mining Latent Structures for Multimedia Recommendation
• Bootstrap Latent Representations for Multi-modal Recommendation
• A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation
• Multi-Modal Self-Supervised Learning for Recommendation
• Multi-View Graph Convolutional Network for Multimedia Recommendation
• MMRec: Simplifying Multimodal Recommendation
• SelfCF: A Simple Framework for Self-supervised Collaborative Filtering