271 citations · 472 across the 14 of their papers we have counts for
14 papers
OmniMER: Auxiliary-Enhanced LLM Adaptation for Indonesian Multimodal Emotion Recognition
Xueming Yan, Boyan Xu, Yaochu Jin +7
Indonesian, spoken by over 200 million people, remains underserved in multimodal emotion recognition research despite its dominant presence on Southeast Asian social media platform…
SSCFormer: Push the Limit of Chunk-wise Conformer for Streaming ASR Using Sequentially Sampled Chunks and Chunked Causal Convolution
Fangyuan Wang, Bo Xu
Currently, the chunk-wise schemes are often used to make Automatic Speech Recognition (ASR) models to support streaming deployment. However, existing approaches are unable to captu…
Sequence-level Speaker Change Detection with Difference-based Continuous Integrate-and-fire
Zhiyun Fan, Linhao Dong, Meng Cai +2
Speaker change detection is an important task in multi-party interactions such as meetings and conversations. In this paper, we address the speaker change detection task from the p…
Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection
Minglun Han, Linhao Dong, Zhenlin Liang +4
Nowadays, most methods in end-to-end contextual speech recognition bias the recognition process towards contextual knowledge. Since all-neural contextual biasing methods rely on ph…
Shifted Chunk Encoder for Transformer Based Streaming End-to-End ASR
Fangyuan Wang, Bo Xu
Currently, there are mainly three kinds of Transformer encoder based streaming End to End (E2E) Automatic Speech Recognition (ASR) approaches, namely time-restricted methods, chunk…
VLP: A Survey on Vision-Language Pre-training
Feilong Chen, Duzhen Zhang, Minglun Han +4
In the past few years, the emergence of pre-training models has brought uni-modal fields such as computer vision (CV) and natural language processing (NLP) to a new era. Substantia…
LiMuSE: Lightweight Multi-modal Speaker Extraction
Qinghua Liu, Yating Huang, Yunzhe Hao +2
Multi-modal cues, including spatial information, facial expression and voiceprint, are introduced to the speech separation and speaker extraction tasks to serve as complementary in…
MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition
Linghui Meng, Jin Xu, Xu Tan +3
In this paper, we propose MixSpeech, a simple yet effective data augmentation method based on mixup for automatic speech recognition (ASR). MixSpeech trains an ASR model by taking…
Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for Low-resource Speech Recognition
Cheng Yi, Shiyu Zhou, Bo Xu
End-to-end models have achieved impressive results on the task of automatic speech recognition (ASR). For low-resource ASR tasks, however, labeled data can hardly satisfy the deman…
Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languages
Cheng Yi, Jianzhong Wang, Ning Cheng +2
There are several domains that own corresponding widely used feature extractors, such as ResNet, BERT, and GPT-x. These models are usually pre-trained on large amounts of unlabeled…
Consecutive Decoding for Speech-to-text Translation
Qianqian Dong, Mingxuan Wang, Hao Zhou +3
Speech-to-text translation (ST), which directly translates the source language speech to the target language text, has attracted intensive attention recently. However, the combinat…
"Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text Translation
Qianqian Dong, Rong Ye, Mingxuan Wang +4
An end-to-end speech-to-text translation (ST) takes audio in a source language and outputs the text in a target language. Existing methods are limited by the amount of parallel cor…
A Comparison of Label-Synchronous and Frame-Synchronous End-to-End Models for Speech Recognition
Linhao Dong, Cheng Yi, Jianzong Wang +4
End-to-end models are gaining wider attention in the field of automatic speech recognition (ASR). One of their advantages is the simplicity of building that directly recognizes the…
Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages
Shiyu Zhou, Shuang Xu, Bo Xu
Sequence-to-sequence attention-based models integrate an acoustic, pronunciation and language model into a single neural network, which make them very suitable for multilingual aut…