Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering

Abstract

Contrastive Learning (CL) has shown promising performance in collaborative filtering. The key idea is to generate augmentation-invariant embeddings by maximizing the Mutual Information between different augmented views of the same instance. However, we empirically observe that existing CL models suffer from the \textsl{dimensional collapse} issue, where user/item embeddings only span a low-dimension subspace of the entire feature space. This suppresses other dimensional information and weakens the distinguishability of embeddings. Here we propose a non-contrastive learning objective, named nCL, which explicitly mitigates dimensional collapse of representations in collaborative filtering. Our nCL aims to achieve geometric properties of \textsl{Alignment} and \textsl{Compactness} on the embedding space. In particular, the alignment tries to push together representations of positive-related user-item pairs, while compactness tends to find the optimal coding length of user/item embeddings, subject to a given distortion. More importantly, our nCL does not require data augmentation nor negative sampling during training, making it scalable to large datasets. Experimental results demonstrate the superiority of our nCL.

References in corpus (27)

• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Neural Graph Collaborative Filtering
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Self-supervised Graph Learning for Recommendation
• Graph Contrastive Learning with Augmentations
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
• Disentangled Graph Collaborative Filtering
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation
• Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks
• Parameterized Explainer for Graph Neural Network
• Towards Representation Alignment and Uniformity in Collaborative Filtering
• Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure
• Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakage
• Understanding Dimensional Collapse in Contrastive Self-supervised Learning
• Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation
• Denoising Self-attentive Sequential Recommendation
• A Fast Proximal Point Method for Computing Exact Wasserstein Distance
• ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction
• A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal
• UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation
• MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentation
• Universal Rate-Distortion-Perception Representations for Lossy Compression
• SimpleX: A Simple and Strong Baseline for Collaborative Filtering
• Enhancing Transformers without Self-supervised Learning: A Loss Landscape Perspective in Sequential Recommendation

Cited by in corpus (2)

• A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems
• Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation