1.9k citations · 3k across the 69 of their papers we have counts for
17 papers · 2 filters
Sequence-level self-learning with multiple hypotheses
Kenichi Kumatani, Dimitrios Dimitriadis, Yashesh Gaur +4
In this work, we develop new self-learning techniques with an attention-based sequence-to-sequence (seq2seq) model for automatic speech recognition (ASR). For untranscribed speech…
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
Yichong Xu, Chenguang Zhu, Shuohang Wang +7
Most of today's AI systems focus on using self-attention mechanisms and transformer architectures on large amounts of diverse data to achieve impressive performance gains. In this…
Leveraging Knowledge in Multilingual Commonsense Reasoning
Yuwei Fang, Shuohang Wang, Yichong Xu +4
Commonsense reasoning (CSR) requires the model to be equipped with general world knowledge. While CSR is a language-agnostic process, most comprehensive knowledge sources are in fe…
End-to-End Segmentation-based News Summarization
Yang Liu, Chenguang Zhu, Michael Zeng
In this paper, we bring a new way of digesting news content by introducing the task of segmenting a news article into multiple sections and generating the corresponding summary to…
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen +15
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks. As speech signal contai…
Dict-BERT: Enhancing Language Model Pre-training with Dictionary
Wenhao Yu, Chenguang Zhu, Yuwei Fang +5
Pre-trained language models (PLMs) aim to learn universal language representations by conducting self-supervised training tasks on large-scale corpora. Since PLMs capture word sema…
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
Donghan Yu, Chenguang Zhu, Yuwei Fang +6
Current Open-Domain Question Answering (ODQA) model paradigm often contains a retrieving module and a reading module. Given an input question, the reading module predicts the answe…
DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization
Ming Zhong, Yang Liu, Yichong Xu +2
Dialogue is an essential part of human communication and cooperation. Existing research mainly focuses on short dialogue scenarios in a one-on-one fashion. However, multi-person in…
Does Knowledge Help General NLU? An Empirical Study
Ruochen Xu, Yuwei Fang, Chenguang Zhu +1
It is often observed in knowledge-centric tasks (e.g., common sense question and answering, relation classification) that the integration of external knowledge such as entity repre…
Want To Reduce Labeling Cost? GPT-3 Can Help
Shuohang Wang, Yang Liu, Yichong Xu +2
Data annotation is a time-consuming and labor-intensive process for many NLP tasks. Although there exist various methods to produce pseudo data labels, they are often task-specific…
A Joint and Domain-Adaptive Approach to Spoken Language Understanding
Linhao Zhang, Yu Shi, Linjun Shou +3
Spoken Language Understanding (SLU) is composed of two subtasks: intent detection (ID) and slot filling (SF). There are two lines of research on SLU. One jointly tackles these two…
Retrieval Enhanced Model for Commonsense Generation
Han Wang, Yang Liu, Chenguang Zhu +4
Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts. Its requirement of reasoning over commonsen…
MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization
Chenguang Zhu, Yang Liu, Jie Mei +1
MediaSum, a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. To create this dataset, we collect interview transcripts from NPR and C…
Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model
Junwei Liao, Yu Shi, Ming Gong +5
Modern Automatic Speech Recognition (ASR) systems can achieve high performance in terms of recognition accuracy. However, a perfectly accurate transcript still can be challenging t…
Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders
Junwei Liao, Yu Shi, Ming Gong +3
Recently, universal neural machine translation (NMT) with shared encoder-decoder gained good performance on zero-shot translation. Unlike universal NMT, jointly trained language-sp…
Speech-language Pre-training for End-to-end Spoken Language Understanding
Yao Qian, Ximo Bian, Yu Shi +4
End-to-end (E2E) spoken language understanding (SLU) can infer semantics directly from speech signal without cascading an automatic speech recognizer (ASR) with a natural language…
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian +5
In this paper, we propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data, in which supervised phonetic CTC le…