Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems

Abstract

Intelligent personal assistant systems with either text-based or voice-based conversational interfaces are becoming increasingly popular around the world. Retrieval-based conversation models have the advantages of returning fluent and informative responses. Most existing studies in this area are on open domain "chit-chat" conversations or task / transaction oriented conversations. More research is needed for information-seeking conversations. There is also a lack of modeling external knowledge beyond the dialog utterances among current conversational models. In this paper, we propose a learning framework on the top of deep neural matching networks that leverages external knowledge for response ranking in information-seeking conversation systems. We incorporate external knowledge into deep neural models with pseudo-relevance feedback and QA correspondence knowledge distillation. Extensive experiments with three information-seeking conversation data sets including both open benchmarks and commercial data show that, our methods outperform various baseline methods including several deep text matching models and the state-of-the-art method on response selection in multi-turn conversations. We also perform analysis over different response types, model variations and ranking examples. Our models and research findings provide new insights on how to utilize external knowledge with deep neural models for response selection and have implications for the design of the next generation of information-seeking conversation systems.

arXiv commentAccepted by the 41th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2018), Ann Arbor, Michigan, U.S.A. July 8-12, 2018 (Full Oral Paper)

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• A Neural Conversational Model
• Convolutional Neural Network Architectures for Matching Natural Language Sentences
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• End-to-End Neural Ad-hoc Ranking with Kernel Pooling
• Deep Reinforcement Learning for Dialogue Generation
• Neural Responding Machine for Short-Text Conversation
• An Information Retrieval Approach to Short Text Conversation
• A Persona-Based Neural Conversation Model
• Attentive Memory Networks: Efficient Machine Reading for Conversational Search
• Neural Matching Models for Question Retrieval and Next Question Prediction in Conversation

Cited by in corpus (38)

• Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
• BERT with History Answer Embedding for Conversational Question Answering
• Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching
• Evaluating Conversational Recommender Systems via User Simulation
• Open-Retrieval Conversational Question Answering
• User Intent Prediction in Information-seeking Conversations
• What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
• Attentive History Selection for Conversational Question Answering
• Meta-evaluation of Conversational Search Evaluation Metrics
• Challenges in Building Intelligent Open-domain Dialog Systems
• Analysing Mixed Initiatives and Search Strategies during Conversational Search
• Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching
• Lifelong and Interactive Learning of Factual Knowledge in Dialogues
• Answer Interaction in Non-factoid Question Answering Systems
• A Hybrid Retrieval-Generation Neural Conversation Model
• Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey
• Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset
• Deep Retrieval-Based Dialogue Systems: A Short Review
• A Linguistic Study on Relevance Modeling in Information Retrieval
• Simulating and Modeling the Risk of Conversational Search
• A Deep Look into Neural Ranking Models for Information Retrieval
• Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce
• A Re-classification of Information Seeking Tasks and Their Computational Solutions
• A Survey of Document Grounded Dialogue Systems (DGDS)
• The Technological Gap Between Virtual Assistants and Recommendation Systems
• COOKIE: A Dataset for Conversational Recommendation over Knowledge Graphs in E-commerce
• Retrieval-Enhanced Adversarial Training for Neural Response Generation
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots
• Open-domain question classification and completion in conversational information search
• IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systems
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• On the Calibration and Uncertainty of Neural Learning to Rank Models
• Weakly-Supervised Open-Retrieval Conversational Question Answering
• Controlling the Risk of Conversational Search via Reinforcement Learning
• Domain Representation for Knowledge Graph Embedding
• Weakly-Supervised Neural Response Selection from an Ensemble of Task-Specialised Dialogue Agents
• Topic Propagation in Conversational Search
• Searching Personal Collections