Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation

Abstract

Learning dynamic user preference has become an increasingly important component for many online platforms (e.g., video-sharing sites, e-commerce systems) to make sequential recommendations. Previous works have made many efforts to model item-item transitions over user interaction sequences, based on various architectures, e.g., recurrent neural networks and self-attention mechanism. Recently emerged graph neural networks also serve as useful backbone models to capture item dependencies in sequential recommendation scenarios. Despite their effectiveness, existing methods have far focused on item sequence representation with singular type of interactions, and thus are limited to capture dynamic heterogeneous relational structures between users and items (e.g., page view, add-to-favorite, purchase). To tackle this challenge, we design a Multi-Behavior Hypergraph-enhanced Transformer framework (MBHT) to capture both short-term and long-term cross-type behavior dependencies. Specifically, a multi-scale Transformer is equipped with low-rank self-attention to jointly encode behavior-aware sequential patterns from fine-grained and coarse-grained levels. Additionally, we incorporate the global multi-behavior dependency into the hypergraph neural architecture to capture the hierarchical long-range item correlations in a customized manner. Experimental results demonstrate the superiority of our MBHT over various state-of-the-art recommendation solutions across different settings. Further ablation studies validate the effectiveness of our model design and benefits of the new MBHT framework. Our implementation code is released at: https://github.com/yuh-yang/MBHT-KDD22.

arXiv commentPublished as a KDD'22 full paper

References in corpus (11)

• Session-based Recommendation with Graph Neural Networks
• Facial Expression Analysis under Partial Occlusion: A Survey
• Linformer: Self-Attention with Linear Complexity
• Knowledge Graph Contrastive Learning for Recommendation
• Hypergraph Contrastive Collaborative Filtering
• LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation
• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
• Graph Meta Network for Multi-Behavior Recommendation
• Contrastive Meta Learning with Behavior Multiplicity for Recommendation
• Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction
• Click-Through Rate Prediction with Multi-Modal Hypergraphs

Cited by in corpus (9)

• A Comprehensive Survey on Deep Graph Representation Learning
• Debiased Contrastive Learning for Sequential Recommendation
• Disentangled Contrastive Collaborative Filtering
• LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems
• Learning Graph ODE for Continuous-Time Sequential Recommendation
• Graph Masked Autoencoder for Sequential Recommendation
• Multi-Modal Multi-Behavior Sequential Recommendation with Conditional Diffusion-Based Feature Denoising
• HMAR: Hierarchical Masked Attention for Multi-Behaviour Recommendation
• PCL: Prompt-based Continual Learning for User Modeling in Recommender Systems