Rankformer: A Graph Transformer for Recommendation based on Ranking Objective

Abstract

Recommender Systems (RS) aim to generate personalized ranked lists for each user and are evaluated using ranking metrics. Although personalized ranking is a fundamental aspect of RS, this critical property is often overlooked in the design of model architectures. To address this issue, we propose Rankformer, a ranking-inspired recommendation model. The architecture of Rankformer is inspired by the gradient of the ranking objective, embodying a unique (graph) transformer architecture -- it leverages global information from all users and items to produce more informative representations and employs specific attention weights to guide the evolution of embeddings towards improved ranking performance. We further develop an acceleration algorithm for Rankformer, reducing its complexity to a linear level with respect to the number of positive instances. Extensive experimental results demonstrate that Rankformer outperforms state-of-the-art methods. The code is available at https://github.com/StupidThree/Rankformer.

Accepted by WWW2025

References in corpus (16)

• Neural Graph Collaborative Filtering
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Self-supervised Graph Learning for Recommendation
• A Survey of Graph Neural Networks for Social Recommender Systems
• AutoDebias: Learning to Debias for Recommendation
• Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning
• NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification
• Collaboration-Aware Graph Convolutional Network for Recommender Systems
• On the Effectiveness of Sampled Softmax Loss for Item Recommendation
• Rethinking Graph Transformers with Spectral Attention
• Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation
• SIGformer: Sign-aware Graph Transformer for Recommendation
• Hierarchical Graph Transformer with Adaptive Node Sampling
• Adap-$\tau$: Adaptively Modulating Embedding Magnitude for Recommendation
• Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
• Masked Graph Transformer for Large-Scale Recommendation