Revisiting Neural Retrieval on Accelerators

Abstract

Retrieval finds a small number of relevant candidates from a large corpus for information retrieval and recommendation applications. A key component of retrieval is to model (user, item) similarity, which is commonly represented as the dot product of two learned embeddings. This formulation permits efficient inference, commonly known as Maximum Inner Product Search (MIPS). Despite its popularity, dot products cannot capture complex user-item interactions, which are multifaceted and likely high rank. We hence examine non-dot-product retrieval settings on accelerators, and propose \textit{mixture of logits} (MoL), which models (user, item) similarity as an adaptive composition of elementary similarity functions. This new formulation is expressive, capable of modeling high rank (user, item) interactions, and further generalizes to the long tail. When combined with a hierarchical retrieval strategy, \textit{h-indexer}, we are able to scale up MoL to 100M corpus on a single GPU with latency comparable to MIPS baselines. On public datasets, our approach leads to uplifts of up to 77.3\% in hit rate (HR). Experiments on a large recommendation surface at Meta showed strong metric gains and reduced popularity bias, validating the proposed approach's performance and improved generalization.

arXiv commentTo appear in the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2023)

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Mixed Precision Training
• Learning Tree-based Deep Model for Recommender Systems
• Embedding-based Retrieval in Facebook Search
• BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
• Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)
• Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks
• A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models
• FP8 Formats for Deep Learning
• Quantization based Fast Inner Product Search
• Joint Optimization of Tree-based Index and Deep Model for Recommender Systems
• Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time
• FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
• SDM: Sequential Deep Matching Model for Online Large-scale Recommender System
• TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s

Cited by in corpus (5)

• Temporal Interest Network for User Response Prediction
• Retrieval with Learned Similarities
• eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashion
• Optimizing Recall or Relevance? A Multi-Task Multi-Head Approach for Item-to-Item Retrieval in Recommendation
• Finding Interest Needle in Popularity Haystack: Improving Retrieval by Modeling Item Exposure