Self-supervised Graph Learning for Recommendation

Abstract

Representation learning on user-item graph for recommendation has evolved from using single ID or interaction history to exploiting higher-order neighbors. This leads to the success of graph convolution networks (GCNs) for recommendation such as PinSage and LightGCN. Despite effectiveness, we argue that they suffer from two limitations: (1) high-degree nodes exert larger impact on the representation learning, deteriorating the recommendations of low-degree (long-tail) items; and (2) representations are vulnerable to noisy interactions, as the neighborhood aggregation scheme further enlarges the impact of observed edges. In this work, we explore self-supervised learning on user-item graph, so as to improve the accuracy and robustness of GCNs for recommendation. The idea is to supplement the classical supervised task of recommendation with an auxiliary self-supervised task, which reinforces node representation learning via self-discrimination. Specifically, we generate multiple views of a node, maximizing the agreement between different views of the same node compared to that of other nodes. We devise three operators to generate the views -- node dropout, edge dropout, and random walk -- that change the graph structure in different manners. We term this new learning paradigm as \textit{Self-supervised Graph Learning} (SGL), implementing it on the state-of-the-art model LightGCN. Through theoretical analyses, we find that SGL has the ability of automatically mining hard negatives. Empirical studies on three benchmark datasets demonstrate the effectiveness of SGL, which improves the recommendation accuracy, especially on long-tail items, and the robustness against interaction noises. Our implementations are available at \url{https://github.com/wujcan/SGL}.

11 pages, 7 figures, 5 tables. Accepted by SIGIR 2021

References in corpus (5)

• KGAT: Knowledge Graph Attention Network for Recommendation
• Knowledge Graph Convolutional Networks for Recommender Systems
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks

Cited by in corpus (108)

• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Hypergraph Contrastive Collaborative Filtering
• A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
• Intent Contrastive Learning for Sequential Recommendation
• Bootstrap Latent Representations for Multi-modal Recommendation
• Heterogeneous Graph Contrastive Learning for Recommendation
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• Multi-Modal Self-Supervised Learning for Recommendation
• Representation Learning with Large Language Models for Recommendation
• Contrastive Meta Learning with Behavior Multiplicity for Recommendation
• Knowledge Graph Self-Supervised Rationalization for Recommendation
• Debiased Contrastive Learning for Sequential Recommendation
• A Review-aware Graph Contrastive Learning Framework for Recommendation
• Self-Supervised Hypergraph Transformer for Recommender Systems
• Disentangled Contrastive Collaborative Filtering
• Automated Self-Supervised Learning for Recommendation
• Multi-Behavior Recommendation with Cascading Graph Convolution Networks
• Knowledge Enhancement for Contrastive Multi-Behavior Recommendation
• CrossCBR: Cross-view Contrastive Learning for Bundle Recommendation
• COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
• Improving Knowledge-aware Recommendation with Multi-level Interactive Contrastive Learning
• Graph Transformer for Recommendation
• Multi-behavior Self-supervised Learning for Recommendation
• Self-Guided Learning to Denoise for Robust Recommendation
• Multi-Behavior Sequential Recommendation with Temporal Graph Transformer
• Collaboration-Aware Graph Convolutional Network for Recommender Systems
• On the Effectiveness of Sampled Softmax Loss for Item Recommendation
• Investigating Accuracy-Novelty Performance for Graph-based Collaborative Filtering
• MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering
• Meta-optimized Contrastive Learning for Sequential Recommendation
• Graph-less Collaborative Filtering
• Joint Multi-grained Popularity-aware Graph Convolution Collaborative Filtering for Recommendation
• GIF: A General Graph Unlearning Strategy via Influence Function
• Neighborhood-Enhanced Supervised Contrastive Learning for Collaborative Filtering
• SSLRec: A Self-Supervised Learning Framework for Recommendation
• ConsRec: Learning Consensus Behind Interactions for Group Recommendation
• Thinking inside The Box: Learning Hypercube Representations for Group Recommendation
• Poisoning Deep Learning Based Recommender Model in Federated Learning Scenarios
• Multi-level Contrastive Learning Framework for Sequential Recommendation
• Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning
• Formalizing Multimedia Recommendation through Multimodal Deep Learning
• Personalized Graph Signal Processing for Collaborative Filtering
• Graph Contrastive Learning with Generative Adversarial Network
• Knowledge-refined Denoising Network for Robust Recommendation
• Diffusion-based Contrastive Learning for Sequential Recommendation
• SIGformer: Sign-aware Graph Transformer for Recommendation
• Adap-$\tau$: Adaptively Modulating Embedding Magnitude for Recommendation
• Contrastive Multi-Level Graph Neural Networks for Session-based Recommendation
• Unifying Graph Convolution and Contrastive Learning in Collaborative Filtering
• Improving Implicit Feedback-Based Recommendation through Multi-Behavior Alignment
• Augmented Negative Sampling for Collaborative Filtering
• APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation
• Knowledge Enhanced Multi-intent Transformer Network for Recommendation
• GNN-Geo: A Graph Neural Network-based Fine-grained IP geolocation Framework
• A Comprehensive Survey on Self-Supervised Learning for Recommendation
• RevGNN: Negative Sampling Enhanced Contrastive Graph Learning for Academic Reviewer Recommendation
• Scenario-Adaptive and Self-Supervised Model for Multi-Scenario Personalized Recommendation
• LightGNN: Simple Graph Neural Network for Recommendation
• GraFN: Semi-Supervised Node Classification on Graph with Few Labels via Non-Parametric Distribution Assignment
• A Comparative Analysis of Bias Amplification in Graph Neural Network Approaches for Recommender Systems
• Towards Robust Recommendation via Decision Boundary-aware Graph Contrastive Learning
• Adapting Job Recommendations to User Preference Drift with Behavioral-Semantic Fusion Learning
• Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis
• Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering
• Dimension Independent Mixup for Hard Negative Sample in Collaborative Filtering
• Toward a Better Understanding of Loss Functions for Collaborative Filtering
• Recommending on graphs: a comprehensive review from a data perspective
• Unleashing the Power of Large Language Model for Denoising Recommendation
• Collaborative-Enhanced Prediction of Spending on Newly Downloaded Mobile Games under Consumption Uncertainty
• Group Identification via Transitional Hypergraph Convolution with Cross-view Self-supervised Learning
• Top-Personalized-K Recommendation
• Doubly Calibrated Estimator for Recommendation on Data Missing Not At Random
• Unlocking the Hidden Treasures: Enhancing Recommendations with Unlabeled Data
• Ego-Vehicle Action Recognition based on Semi-Supervised Contrastive Learning
• Test Time Embedding Normalization for Popularity Bias Mitigation
• MixRec: Individual and Collective Mixing Empowers Data Augmentation for Recommender Systems
• Fair Augmentation for Graph Collaborative Filtering
• A Survey of Latent Factor Models in Recommender Systems
• Robust Basket Recommendation via Noise-tolerated Graph Contrastive Learning
• LightSAGE: Graph Neural Networks for Large Scale Item Retrieval in Shopee's Advertisement Recommendation
• Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering
• Mitigating Spurious Correlations for Self-supervised Recommendation
• User Invariant Preference Learning for Multi-Behavior Recommendation
• Stock Recommendations for Individual Investors: A Temporal Graph Network Approach with Mean-Variance Efficient Sampling
• A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions
• Unbiased and Robust: External Attention-enhanced Graph Contrastive Learning for Cross-domain Sequential Recommendation
• Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation
• Intent Propagation Contrastive Collaborative Filtering
• A Vlogger-augmented Graph Neural Network Model for Micro-video Recommendation
• Rankformer: A Graph Transformer for Recommendation based on Ranking Objective
• Bootstrapping Contrastive Learning Enhanced Music Cold-Start Matching
• XInsight: Revealing Model Insights for GNNs with Flow-based Explanations
• Contrastive Learning for Conversion Rate Prediction
• TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning
• Hierarchical Vector Quantized Graph Autoencoder with Annealing-Based Code Selection
• Pre-training for Recommendation Unlearning
• FedHCDR: Federated Cross-Domain Recommendation with Hypergraph Signal Decoupling
• Cyberswarm: a novel swarm intelligence algorithm inspired by cyber community dynamics
• SPARK: Adaptive Low-Rank Knowledge Graph Modeling in Hybrid Geometric Spaces for Recommendation
• EfficientRec an unlimited user-item scale recommendation system based on clustering and users interaction embedding profile
• Graph-based Recommendation for Sparse and Heterogeneous User Interactions
• Link Prediction in Bipartite Networks
• Enhancing Recommendation with Denoising Auxiliary Task
• The Hidden Cost of Defaults in Recommender System Evaluation
• Generative Data Augmentation in Graph Contrastive Learning for Recommendation
• RUEL: Retrieval-Augmented User Representation with Edge Browser Logs for Sequential Recommendation
• Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity