Beam Search with Bidirectional Strategies for Neural Response Generation

Abstract

Sequence-to-sequence neural networks have been widely used in language-based applications as they have flexible capabilities to learn various language models. However, when seeking for the optimal language response through trained neural networks, current existing approaches such as beam-search decoder strategies are still not able reaching to promising performances. Instead of developing various decoder strategies based on a "regular sentence order" neural network (a trained model by outputting sentences from left-to-right order), we leveraged "reverse" order as additional language model (a trained model by outputting sentences from right-to-left order) which can provide different perspectives for the path finding problems. In this paper, we propose bidirectional strategies in searching paths by combining two networks (left-to-right and right-to-left language models) making a bidirectional beam search possible. Besides, our solution allows us using any similarity measure in our sentence selection criterion. Our approaches demonstrate better performance compared to the unidirectional beam search strategy.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• A Critical Review of Recurrent Neural Networks for Sequence Learning
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking
• Code-switched inspired losses for generic spoken dialog representations

Cited by in corpus (5)

• Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation
• Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks
• A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification
• Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models
• A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions