9 citations · 25 across the 23 of their papers we have counts for
8 papers · 1 filter
Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
Qian Chen, Wen Wang, Qinglin Zhang +7
Recently, unified speech-text models, such as SpeechGPT, VioLA, and AudioPaLM, have achieved remarkable performance on various speech tasks. These models discretize speech signals…
Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling
Hai Yu, Chong Deng, Qinglin Zhang +3
Topic segmentation is critical for obtaining structured documents and improving downstream tasks such as information retrieval. Due to its ability of automatically exploring clues…
Improving BERT with Hybrid Pooling Network and Drop Mask
Qian Chen, Wen Wang, Qinglin Zhang +3
Transformer-based pre-trained language models, such as BERT, achieve great success in various natural language understanding tasks. Prior research found that BERT captures a rich h…
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings
Qian Chen, Wen Wang, Qinglin Zhang +6
Prior studies diagnose the anisotropy problem in sentence representations from pre-trained language models, e.g., BERT, without fine-tuning. Our analysis reveals that the sentence…
Meeting Action Item Detection with Regularized Context Modeling
Jiaqing Liu, Chong Deng, Qinglin Zhang +2
Meetings are increasingly important for collaborations. Action items in meeting transcripts are crucial for managing post-meeting to-do tasks, which usually are summarized laboriou…
MUG: A General Meeting Understanding and Generation Benchmark
Qinglin Zhang, Chong Deng, Jiaqing Liu +7
Listening to long video/audio recordings from video conferencing and online courses for acquiring information is extremely inefficient. Even after ASR systems transcribe recordings…
Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG)
Qinglin Zhang, Chong Deng, Jiaqing Liu +7
ICASSP2023 General Meeting Understanding and Generation Challenge (MUG) focuses on prompting a wide range of spoken language processing (SLP) research on meeting transcripts, as SL…
Weighted Sampling for Masked Language Modeling
Linhan Zhang, Qian Chen, Wen Wang +5
Masked Language Modeling (MLM) is widely used to pretrain language models. The standard random masking strategy in MLM causes the pre-trained language models (PLMs) to be biased to…