Neural Models for Information Retrieval

Abstract

Neural ranking models for information retrieval (IR) use shallow or deep neural networks to rank search results in response to a query. Traditional learning to rank models employ machine learning techniques over hand-crafted IR features. By contrast, neural models learn representations of language from raw text that can bridge the gap between query and document vocabulary. Unlike classical IR models, these new machine learning based approaches are data-hungry, requiring large scale training data before they can be deployed. This tutorial introduces basic concepts and intuitions behind neural IR models, and places them in the context of traditional retrieval models. We begin by introducing fundamental concepts of IR and different neural and non-neural approaches to learning vector representations of text. We then review shallow neural IR methods that employ pre-trained neural term embeddings without learning the IR task end-to-end. We introduce deep neural networks next, discussing popular deep architectures. Finally, we review the current DNN models for information retrieval. We conclude with a discussion on potential future directions for neural IR.

References in corpus (50)

• Efficient Estimation of Word Representations in Vector Space
• Deep Learning in Neural Networks: An Overview
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Natural Language Processing (almost) from Scratch
• Caffe: Convolutional Architecture for Fast Feature Embedding
• From Frequency to Meaning: Vector Space Models of Semantics
• Teaching Machines to Read and Comprehend
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• A Neural Conversational Model
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Convolutional Neural Networks for Sentence Classification
• Understanding deep learning requires rethinking generalization
• Recurrent Models of Visual Attention
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• Sequence Level Training with Recurrent Neural Networks
• Matching Networks for One Shot Learning
• Exploring the Limits of Language Modeling
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• On the Number of Linear Regions of Deep Neural Networks
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• word2vec Parameter Learning Explained
• FractalNet: Ultra-Deep Neural Networks without Residuals
• How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation
• Large-scale Simple Question Answering with Memory Networks
• The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
• DyNet: The Dynamic Neural Network Toolkit
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Text Matching as Image Recognition
• Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN
• A Dual Embedding Space Model for Document Ranking
• deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
• Modelling, Visualising and Summarising Documents with a Single Convolutional Neural Network
• Using Word Embeddings for Automatic Query Expansion
• A Study of MatchPyramid Models on Ad-hoc Retrieval
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Benchmarking State-of-the-Art Deep Learning Software Tools
• Word Emdeddings through Hellinger PCA
• Semantic Modelling with Long-Short-Term Memory for Information Retrieval
• Multimodal Convolutional Neural Networks for Matching Image and Sentence
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
• Query Expansion with Locally-Trained Word Embeddings
• Learning to Compose Words into Sentences with Reinforcement Learning
• A Fast Unified Model for Parsing and Sentence Understanding
• Adaptability of Neural Networks on Varying Granularity IR Tasks
• Toward a Deep Neural Approach for Knowledge-Based IR
• Semantic Regularities in Document Representations
• A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion
• Pyndri: a Python Interface to the Indri Search Engine
• Luandri: a Clean Lua Interface to the Indri Search Engine

Cited by in corpus (34)

• Neural Networks for Entity Matching: A Survey
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Modeling Diverse Relevance Patterns in Ad-hoc Retrieval
• TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• Dynamic Integration of Background Knowledge in Neural NLU Systems
• Seq2Slate: Re-ranking and Slate Optimization with RNNs
• Adversarial Retriever-Ranker for dense text retrieval
• Learning a Product Relevance Model from Click-Through Data in E-Commerce
• Exploring the Effectiveness of Convolutional Neural Networks for Answer Selection in End-to-End Question Answering
• Microblog Retrieval for Post-Disaster Relief: Applying and Comparing Neural IR Models
• A Deep Look into Neural Ranking Models for Information Retrieval
• Benchmark for Complex Answer Retrieval
• A Fast Deep Learning Model for Textual Relevance in Biomedical Information Retrieval
• Neural IR Meets Graph Embedding: A Ranking Model for Product Search
• End-to-End Neural Ranking for eCommerce Product Search: an application of task models and textual embeddings
• An Exploration of Approaches to Integrating Neural Reranking Models in Multi-Stage Ranking Architectures
• A Comparison of Supervised Learning to Match Methods for Product Search
• Utilizing Embeddings for Ad-hoc Retrieval by Document-to-document Similarity
• EXS: Explainable Search Using Local Model Agnostic Interpretability
• Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
• A Social Search Model for Large Scale Social Networks
• Separate and Attend in Personal Email Search
• Beyond Lexical: A Semantic Retrieval Framework for Textual SearchEngine
• Zero-shot Neural Transfer for Cross-lingual Entity Linking
• Pentagon at MEDIQA 2019: Multi-task Learning for Filtering and Re-ranking Answers using Language Inference and Question Entailment
• Representation Learning for Words and Entities
• Cascade Neural Ensemble for Identifying Scientifically Sound Articles
• Embracing Structure in Data for Billion-Scale Semantic Product Search
• Personalizing Search Results Using Hierarchical RNN with Query-aware Attention
• Scalable Methods for Calculating Term Co-Occurrence Frequencies
• Record Linkage to Match Customer Names: A Probabilistic Approach
• Learning Fast Matching Models from Weak Annotations
• Unbiased Sentence Encoder For Large-Scale Multi-lingual Search Engines