Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
arXiv:2104.08455
Abstract
Dialogue systems powered by large pre-trained language models (LM) exhibit an innate ability to deliver fluent and natural-looking responses. Despite their impressive generation performance, these models can often generate factually incorrect statements impeding their widespread adoption. In this paper, we focus on the task of improving the faithfulness -- and thus reduce hallucination -- of Neural Dialogue Systems to known facts supplied by a Knowledge Graph (KG). We propose Neural Path Hunter which follows a generate-then-refine strategy whereby a generated response is amended using the k-hop subgraph of a KG. Neural Path Hunter leverages a separate token-level fact critic to identify plausible sources of hallucination followed by a refinement stage consisting of a chain of two neural LM's that retrieves correct entities by crafting a query signal that is propagated over the k-hop subgraph. Our proposed model can easily be applied to any dialogue generated responses without retraining the model. We empirically validate our proposed approach on the OpenDialKG dataset against a suite of metrics and report a relative improvement of faithfulness over dialogue responses by 20.35% based on FeQA (Durmus et al., 2020).
arXiv commentEMNLP 2021 18 pages
References in corpus (5)
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- Pathologies of Neural Models Make Interpretations Difficult
- Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
- Flexible End-to-End Dialogue System for Knowledge Grounded Conversation
- Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation