Controllable Multi-Objective Re-ranking with Policy Hypernetworks

Abstract

Multi-stage ranking pipelines have become widely used strategies in modern recommender systems, where the final stage aims to return a ranked list of items that balances a number of requirements such as user preference, diversity, novelty etc. Linear scalarization is arguably the most widely used technique to merge multiple requirements into one optimization objective, by summing up the requirements with certain preference weights. Existing final-stage ranking methods often adopt a static model where the preference weights are determined during offline training and kept unchanged during online serving. Whenever a modification of the preference weights is needed, the model has to be re-trained, which is time and resources inefficient. Meanwhile, the most appropriate weights may vary greatly for different groups of targeting users or at different time periods (e.g., during holiday promotions). In this paper, we propose a framework called controllable multi-objective re-ranking (CMR) which incorporates a hypernetwork to generate parameters for a re-ranking model according to different preference weights. In this way, CMR is enabled to adapt the preference weights according to the environment changes in an online manner, without retraining the models. Moreover, we classify practical business-oriented tasks into four main categories and seamlessly incorporate them in a new proposed re-ranking model based on an Actor-Evaluator framework, which serves as a reliable real-world testbed for CMR. Offline experiments based on the dataset collected from Taobao App showed that CMR improved several popular re-ranking models by using them as underlying models. Online A/B tests also demonstrated the effectiveness and trustworthiness of CMR.

References in corpus (8)

• Conditional Generative Adversarial Nets
• An Overview of Multi-Task Learning in Deep Neural Networks
• Adaptive, Personalized Diversity for Visual Discovery
• Seq2Slate: Re-ranking and Slate Optimization with RNNs
• Learning the Pareto Front with Hypernetworks
• Controllable Pareto Multi-Task Learning
• Sequential Evaluation and Generation Framework for Combinatorial Recommender System
• Neural Re-ranking in Multi-stage Recommender Systems: A Review

Cited by in corpus (4)

• Reinforcement Re-ranking with 2D Grid-based Recommendation Panels
• Impression-Aware Recommender Systems
• DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
• Improving Conversational Recommendation with Contextual Adaptation of External Recommenders and LLM-based Reranking