Filter-enhanced MLP is All You Need for Sequential Recommendation

Abstract

Recently, deep neural networks such as RNN, CNN and Transformer have been applied in the task of sequential recommendation, which aims to capture the dynamic preference characteristics from logged user behavior data for accurate recommendation. However, in online platforms, logged user behavior data is inevitable to contain noise, and deep recommendation models are easy to overfit on these logged data. To tackle this problem, we borrow the idea of filtering algorithms from signal processing that attenuates the noise in the frequency domain. In our empirical experiments, we find that filtering algorithms can substantially improve representative sequential recommendation models, and integrating simple filtering algorithms (eg Band-Stop Filter) with an all-MLP architecture can even outperform competitive Transformer-based models. Motivated by it, we propose \textbf{FMLP-Rec}, an all-MLP model with learnable filters for sequential recommendation task. The all-MLP architecture endows our model with lower time complexity, and the learnable filters can adaptively attenuate the noise information in the frequency domain. Extensive experiments conducted on eight real-world datasets demonstrate the superiority of our proposed method over competitive RNN, CNN, GNN and Transformer-based methods. Our code and data are publicly available at the link: \textcolor{blue}{\url{https://github.com/RUCAIBox/FMLP-Rec}}.

12 pages, Accepted by WWW 2022

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Session-based Recommendation with Graph Neural Networks
• MLP-Mixer: An all-MLP Architecture for Vision
• Facial Expression Analysis under Partial Occlusion: A Survey
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
• LocalViT: Analyzing Locality in Vision Transformers
• Reducing Transformer Depth on Demand with Structured Dropout
• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
• CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation
• DeLighT: Deep and Light-weight Transformer
• An Attention Free Transformer

Cited by in corpus (15)

• Shilling Black-box Recommender Systems by Learning to Generate Fake User Profiles
• Hierarchical Transformer with Spatio-Temporal Context Aggregation for Next Point-of-Interest Recommendation
• Dataset Regeneration for Sequential Recommendation
• When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation
• MELT: Mutual Enhancement of Long-Tailed User and Item for Sequential Recommendation
• Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation
• MUSE: Music Recommender System with Shuffle Play Recommendation Enhancement
• Mitigating Spurious Correlations for Self-supervised Recommendation
• Relative Contrastive Learning for Sequential Recommendation with Similarity-based Positive Pair Selection
• Filtering with Time-frequency Analysis: An Adaptive and Lightweight Model for Sequential Recommender Systems Based on Discrete Wavelet Transform
• TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation
• Pyramid Mixer: Multi-dimensional Multi-period Interest Modeling for Sequential Recommendation
• Generative Archetype-Grounded Item Representations for Sequential Recommendation
• Empowering Denoising Sequential Recommendation with Large Language Model Embeddings
• Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity