LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues

Abstract

We introduce an LSTM-based method for dynamically integrating several word-prediction experts to obtain a conditional language model which can be good simultaneously at several subtasks. We illustrate this general approach with an application to dialogue where we integrate a neural chat model, good at conversational aspects, with a neural question-answering model, good at retrieving precise information from a knowledge-base, and show how the integration combines the strengths of the independent components. We hope that this focused contribution will attract attention on the benefits of using such mixtures of experts in NLP.

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• A Neural Conversational Model
• Neural Responding Machine for Short-Text Conversation
• Question Answering with Subgraph Embeddings
• Neural Generative Question Answering

Cited by in corpus (7)

• Lifelong and Interactive Learning of Factual Knowledge in Dialogues
• Lifelong Knowledge Learning in Rule-based Dialogue Systems
• Deep Active Learning for Dialogue Generation
• Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge
• Knowledge-incorporating ESIM models for Response Selection in Retrieval-based Dialog Systems
• Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms
• Knowledge-based end-to-end memory networks