End-to-End Neural Ad-hoc Ranking with Kernel Pooling

Abstract

This paper proposes K-NRM, a kernel based neural model for document ranking. Given a query and a set of documents, K-NRM uses a translation matrix that models word-level similarities via word embeddings, a new kernel-pooling technique that uses kernels to extract multi-level soft match features, and a learning-to-rank layer that combines those features into the final ranking score. The whole model is trained end-to-end. The ranking layer learns desired feature patterns from the pairwise ranking loss. The kernels transfer the feature patterns into soft-match targets at each similarity level and enforce them on the translation matrix. The word embeddings are tuned accordingly so that they can produce the desired soft matches. Experiments on a commercial search engine's query log demonstrate the improvements of K-NRM over prior feature-based and neural-based states-of-the-art, and explain the source of K-NRM's advantage: Its kernel-guided embedding encodes a similarity metric tailored for matching query words to document words, and provides effective multi-level soft matches.

References in corpus (2)

• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• A Deep Relevance Matching Model for Ad-hoc Retrieval

Cited by in corpus (37)

• Deeper Text Understanding for IR with Contextual Neural Language Modeling
• Multi-Stage Document Ranking with BERT
• Understanding the Behaviors of BERT in Ranking
• Simple Applications of BERT for Ad Hoc Document Retrieval
• Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval
• Word-Entity Duet Representations for Document Ranking
• MatchZoo: A Learning, Practicing, and Developing System for Neural Text Matching
• A Study of Neural Matching Models for Cross-lingual IR
• An Updated Duet Model for Passage Re-ranking
• Cross-lingual Information Retrieval with BERT
• Selective Weak Supervision for Neural Information Retrieval
• Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
• Passage Ranking with Weak Supervision
• Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach
• A Deep Look into Neural Ranking Models for Information Retrieval
• Learning Contextualized Document Representations for Healthcare Answer Retrieval
• TU Wien @ TREC Deep Learning '19 -- Simple Contextualization for Re-ranking
• Let's measure run time! Extending the IR replicability infrastructure to include performance aspects
• GRAPHENE: A Precise Biomedical Literature Retrieval Engine with Graph Augmented Deep Learning and External Knowledge Empowerment
• Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval
• Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations
• One word at a time: adversarial attacks on retrieval models
• Longformer for MS MARCO Document Re-ranking Task
• Learning Colour Representations of Search Queries
• SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
• A Comparison of Supervised Learning to Match Methods for Product Search
• LATTE: Latent Type Modeling for Biomedical Entity Linking
• Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
• Developing Multi-Task Recommendations with Long-Term Rewards via Policy Distilled Reinforcement Learning
• Neural Ranking Models with Multiple Document Fields
• Patient Cohort Retrieval using Transformer Language Models
• Beyond Lexical: A Semantic Retrieval Framework for Textual SearchEngine
• DeText: A Deep Text Ranking Framework with BERT
• Investigating Retrieval Method Selection with Axiomatic Features
• Place Deduplication with Embeddings
• Enriching Article Recommendation with Phrase Awareness
• Target-Guided Open-Domain Conversation