Improving Retrieval Modeling Using Cross Convolution Networks And Multi Frequency Word Embedding

Abstract

To build a satisfying chatbot that has the ability of managing a goal-oriented multi-turn dialogue, accurate modeling of human conversation is crucial. In this paper we concentrate on the task of response selection for multi-turn human-computer conversation with a given context. Previous approaches show weakness in capturing information of rare keywords that appear in either or both context and correct response, and struggle with long input sequences. We propose Cross Convolution Network (CCN) and Multi Frequency word embedding to address both problems. We train several models using the Ubuntu Dialogue dataset which is the largest freely available multi-turn based dialogue corpus. We further build an ensemble model by averaging predictions of multiple models. We achieve a new state-of-the-art on this dataset with considerable improvements compared to previous best results.

References in corpus (12)

• Sequence to Sequence Learning with Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Popular Ensemble Methods: An Empirical Study
• Deep Learning for Answer Sentence Selection
• A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
• An Information Retrieval Approach to Short Text Conversation
• Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
• Improved Deep Learning Baselines for Ubuntu Corpus Dialogs
• A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
• Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking
• Syntax-based Deep Matching of Short Texts
• Sentence Pair Scoring: Towards Unified Framework for Text Comprehension