Towards Data Distillation for End-to-end Spoken Conversational Question Answering

Abstract

In spoken question answering, QA systems are designed to answer questions from contiguous text spans within the related speech transcripts. However, the most natural way that human seek or test their knowledge is via human conversations. Therefore, we propose a new Spoken Conversational Question Answering task (SCQA), aiming at enabling QA systems to model complex dialogues flow given the speech utterances and text corpora. In this task, our main objective is to build a QA system to deal with conversational questions both in spoken and text forms, and to explore the plausibility of providing more cues in spoken documents with systems in information gathering. To this end, instead of adopting automatically generated speech transcripts with highly noisy data, we propose a novel unified data distillation approach, DDNet, which directly fuse audio-text features to reduce the misalignment between automatic speech recognition hypotheses and the reference transcriptions. In addition, to evaluate the capacity of QA systems in a dialogue-style interaction, we assemble a Spoken Conversational Question Answering (Spoken-CoQA) dataset with more than 120k question-answer pairs. Experiments demonstrate that our proposed method achieves superior performance in spoken conversational question answering.

References in corpus (8)

• Distilling the Knowledge in a Neural Network
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
• SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering
• Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
• Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension
• Review Conversational Reading Comprehension

Cited by in corpus (8)

• QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
• EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System
• Video-Guided Curriculum Learning for Spoken Video Grounding
• V2I Connectivity-Based Dynamic Queue-Jump Lane for Emergency Vehicles: A Deep Reinforcement Learning Approach
• Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension
• Knowledge Distillation for Improved Accuracy in Spoken Question Answering
• Self-supervised Dialogue Learning for Spoken Conversational Question Answering
• Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering