Efficient Document Re-Ranking for Transformers by Precomputing Term Representations

Abstract

Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking. However, their computational expenses deem them cost-prohibitive in practice. Our proposed approach, called PreTTR (Precomputing Transformer Term Representations), considerably reduces the query-time latency of deep transformer networks (up to a 42x speedup on web document ranking) making these networks more practical to use in a real-time ranking scenario. Specifically, we precompute part of the document term representations at indexing time (without a query), and merge them with the query representation at query time to compute the final ranking score. Due to the large size of the token representations, we also propose an effective approach to reduce the storage requirement by training a compression layer to match attention scores. Our compression technique reduces the storage required up to 95% and it can be applied without a substantial degradation in ranking performance.

Accepted at SIGIR 2020 (long)

References in corpus (3)

• End-to-End Open-Domain Question Answering with BERTserini
• Deeper Text Understanding for IR with Contextual Neural Language Modeling
• Content-Based Weak Supervision for Ad-Hoc Re-Ranking

Cited by in corpus (12)

• Combining EfficientNet and Vision Transformers for Video Deepfake Detection
• Information Retrieval: Recent Advances and Beyond
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Simplified Data Wrangling with ir_datasets
• Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers
• Query Embedding Pruning for Dense Retrieval
• Adaptive Re-Ranking with a Corpus Graph
• Dissecting FLOPs along input dimensions for GreenAI cost estimations
• Scattered or Connected? An Optimized Parameter-efficient Tuning Approach for Information Retrieval
• Investigating the Effects of Sparse Attention on Cross-Encoders
• A Modern Perspective on Query Likelihood with Deep Generative Retrieval Models
• Explicit and Implicit Semantic Ranking Framework