271 citations · 441 across the 14 of their papers we have counts for
14 papers
A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction
Zefa Hu, Ziyi Ni, Jing Shi +2
This paper focuses on term-status pair extraction from medical dialogues (MD-TSPE), which is essential in diagnosis dialogue systems and the automatic scribe of electronic medical…
X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages
Feilong Chen, Minglun Han, Haozhi Zhao +4
Large language models (LLMs) have demonstrated remarkable language abilities. GPT-4, based on advanced LLMs, exhibits extraordinary multimodal capabilities beyond previous visual l…
VILAS: Exploring the Effects of Vision and Language Context in Automatic Speech Recognition
Ziyi Ni, Minglun Han, Feilong Chen +4
Enhancing automatic speech recognition (ASR) performance by leveraging additional multimodal information has shown promising results in previous studies. However, most of these wor…
Matching-based Term Semantics Pre-training for Spoken Patient Query Understanding
Zefa Hu, Xiuyi Chen, Haoran Wu +5
Medical Slot Filling (MSF) task aims to convert medical queries into structured information, playing an essential role in diagnosis dialogue systems. However, the lack of sufficien…
Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation
Minglun Han, Feilong Chen, Jing Shi +2
Large-scale pre-trained language models (PLMs) have shown great potential in natural language processing tasks. Leveraging the capabilities of PLMs to enhance automatic speech reco…
Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learning
Feilong Chen, Xiuyi Chen, Shuang Xu +1
Visual Dialog is a challenging vision-language task since the visual dialog agent needs to answer a series of questions after reasoning over both the image content and dialog histo…
VLP: A Survey on Vision-Language Pre-training
Feilong Chen, Duzhen Zhang, Minglun Han +4
In the past few years, the emergence of pre-training models has brought uni-modal fields such as computer vision (CV) and natural language processing (NLP) to a new era. Substantia…
Consecutive Decoding for Speech-to-text Translation
Qianqian Dong, Mingxuan Wang, Hao Zhou +3
Speech-to-text translation (ST), which directly translates the source language speech to the target language text, has attracted intensive attention recently. However, the combinat…
"Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text Translation
Qianqian Dong, Rong Ye, Mingxuan Wang +4
An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel cor…
A Comparison of Label-Synchronous and Frame-Synchronous End-to-End Models for Speech Recognition
Linhao Dong, Cheng Yi, Jianzong Wang +4
End-to-end models are gaining wider attention in the field of automatic speech recognition (ASR). One of their advantages is the simplicity of building that directly recognizes the…
Single-channel Speech Dereverberation via Generative Adversarial Training
Chenxing Li, Tieqiang Wang, Shuang Xu +1
In this paper, we propose a single-channel speech dereverberation system (DeReGAT) based on convolutional, bidirectional long short-term memory and deep feed-forward neural network…
Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages
Shiyu Zhou, Shuang Xu, Bo Xu
Sequence-to-sequence attention-based models integrate an acoustic, pronunciation and language model into a single neural network, which make them very suitable for multilingual aut…
A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese
Shiyu Zhou, Linhao Dong, Shuang Xu +1
The choice of modeling units is critical to automatic speech recognition (ASR) tasks. Conventional ASR systems typically choose context-dependent states (CD-states) or context-depe…
Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
Shiyu Zhou, Linhao Dong, Shuang Xu +1
Sequence-to-sequence attention-based models have recently shown very promising results on automatic speech recognition (ASR) tasks, which integrate an acoustic, pronunciation and l…