Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus

Abstract

Ubuntu dialogue corpus is the largest public available dialogue corpus to make it feasible to build end-to-end deep neural network models directly from the conversation data. One challenge of Ubuntu dialogue corpus is the large number of out-of-vocabulary words. In this paper we proposed a method which combines the general pre-trained word embedding vectors with those generated on the task-specific training set to address this issue. We integrated character embedding into Chen et al's Enhanced LSTM method (ESIM) and used it to evaluate the effectiveness of our proposed method. For the task of next utterance selection, the proposed method has demonstrated a significant performance improvement against original ESIM and the new model has achieved state-of-the-art results on both Ubuntu dialogue corpus and Douban conversation corpus. In addition, we investigated the performance impact of end-of-utterance and end-of-turn token tags.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Bidirectional Attention Flow for Machine Comprehension
• Enhanced LSTM for Natural Language Inference
• Convolutional Neural Networks for Sentence Classification
• Character-Aware Neural Language Models
• ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
• Enriching Word Vectors with Subword Information
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Multi-Task Cross-Lingual Sequence Tagging from Scratch
• An Information Retrieval Approach to Short Text Conversation
• Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• Improved Deep Learning Baselines for Ubuntu Corpus Dialogs
• Learning Natural Language Inference with LSTM
• A Decomposable Attention Model for Natural Language Inference
• Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
• An Ensemble Method to Produce High-Quality Word Embeddings (2016)
• Words or Characters? Fine-grained Gating for Reading Comprehension
• Sentence Pair Scoring: Towards Unified Framework for Text Comprehension

Cited by in corpus (7)

• Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
• Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
• Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems
• Building Sequential Inference Models for End-to-End Response Selection
• Knowledge-incorporating ESIM models for Response Selection in Retrieval-based Dialog Systems
• Enriching Conversation Context in Retrieval-based Chatbots
• Multi-turn Response Selection using Dialogue Dependency Relations