Beam Search with Bidirectional Strategies for Neural Response Generation

Abstract

Sequence-to-sequence neural networks have been widely used in language-based applications as they have flexible capabilities to learn various language models. However, when seeking for the optimal language response through trained neural networks, current existing approaches such as beam-search decoder strategies are still not able reaching to promising performances. Instead of developing various decoder strategies based on a "regular sentence order" neural network (a trained model by outputting sentences from left-to-right order), we leveraged "reverse" order as additional language model (a trained model by outputting sentences from right-to-left order) which can provide different perspectives for the path finding problems. In this paper, we propose bidirectional strategies in searching paths by combining two networks (left-to-right and right-to-left language models) making a bidirectional beam search possible. Besides, our solution allows us using any similarity measure in our sentence selection criterion. Our approaches demonstrate better performance compared to the unidirectional beam search strategy.

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• Code-switched inspired losses for generic spoken dialog representations

Cited by in corpus (4)

• Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation
• Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks
• A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification
• Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models