Learning a Product Relevance Model from Click-Through Data in E-Commerce

Abstract

The search engine plays a fundamental role in online e-commerce systems, to help users find the products they want from the massive product collections. Relevance is an essential requirement for e-commerce search, since showing products that do not match search query intent will degrade user experience. With the existence of vocabulary gap between user language of queries and seller language of products, measuring semantic relevance is necessary and neural networks are engaged to address this task. However, semantic relevance is different from click-through rate prediction in that no direct training signal is available. Most previous attempts learn relevance models from user click-through data that are cheap and abundant. Unfortunately, click behavior is noisy and misleading, which is affected by not only relevance but also factors including price, image and attractive titles. Therefore, it is challenging but valuable to learn relevance models from click-through data. In this paper, we propose a new relevance learning framework that concentrates on how to train a relevance model from the weak supervision of click-through data. Different from previous efforts that treat samples as either relevant or irrelevant, we construct more fine-grained samples for training. We propose a novel way to consider samples of different relevance confidence, and come up with a new training objective to learn a robust relevance model with desirable score distribution. The proposed model is evaluated on offline annotated data and online A/B testing, and it achieves both promising performance and high computational efficiency. The model has already been deployed online, serving the search traffic of Taobao for over a year.

Accepted to TheWebConf/WWW 2021

References in corpus (11)

• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• Passage Re-ranking with BERT
• Multi-Stage Document Ranking with BERT
• DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval
• Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN
• Neural Models for Information Retrieval
• Semantic Modelling with Long-Short-Term Memory for Information Retrieval
• Semantic Product Search

Cited by in corpus (7)

• Enhancing Relevance of Embedding-based Retrieval at Walmart
• MAKE: Vision-Language Pre-training based Product Retrieval in Taobao Search
• Relevance Filtering for Embedding-based Retrieval
• Delving into E-Commerce Product Retrieval with Vision-Language Pre-training
• Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commerce
• Meta Learning to Rank for Sparsely Supervised Queries
• CWRCzech: 100M Query-Document Czech Click Dataset and Its Application to Web Relevance Ranking