8 citations · 25 across the 24 of their papers we have counts for
25 papers
Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction
Lujia Bao, Qian Chen, Luyao Cheng +15
Real-time voice assistants must reason over evolving requests, execute actions, and follow conversational rules. Qwen-Audio-3.1-Realtime brings these requirements together through…
Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework
Tianrui Pan, Qinglin Zhang, Chong Deng +6
Compared with half-duplex dialogue systems where the system waits for user turn completion before it responds, natural full-duplex dialogue systems require agents to act proactivel…
FGGM: Fisher-Guided Gradient Masking for Continual Learning
Chao-Hong Tan, Qian Chen, Wen Wang +6
Catastrophic forgetting impairs the continuous learning of large language models. We propose Fisher-Guided Gradient Masking (FGGM), a framework that mitigates this by strategically…
Fun-Audio-Chat Technical Report
Tongyi Fun Team, Qian Chen, Luyao Cheng +10
Recent advancements in joint speech-text models show great potential for seamless voice interactions. However, existing models face critical challenges: temporal resolution mismatc…
Say More with Less: Variable-Frame-Rate Speech Tokenization via Adaptive Clustering and Implicit Duration Coding
Rui-Chen Zheng, Wenrui Liu, Hui-Peng Du +6
Existing speech tokenizers typically assign a fixed number of tokens per second, regardless of the varying information density or temporal fluctuations in the speech signal. This u…
DrVoice: Parallel Speech-Text Voice Conversation Model via Dual-Resolution Speech Representations
Chao-Hong Tan, Qian Chen, Wen Wang +14
Recent studies on end-to-end (E2E) speech generation with large language models (LLMs) have attracted significant community attention, with multiple works extending text-based LLMs…
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
Qinglin Zhang, Luyao Cheng, Chong Deng +8
Full-duplex spoken dialogue systems significantly surpass traditional turn-based dialogue systems, as they allow simultaneous bidirectional communication, closely mirroring human-h…
Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization
Luyao Cheng, Hui Wang, Siqi Zheng +5
Speaker diarization, the process of segmenting an audio stream or transcribed speech content into homogenous partitions based on speaker identity, plays a crucial role in the inter…
Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts
Jiaqing Liu, Chong Deng, Qinglin Zhang +4
Automatic Speech Recognition (ASR) transcripts exhibit recognition errors and various spoken language phenomena such as disfluencies, ungrammatical sentences, and incomplete senten…
Multimodal Fusion and Coherence Modeling for Video Topic Segmentation
Hai Yu, Chong Deng, Qinglin Zhang +3
The video topic segmentation (VTS) task segments videos into intelligible, non-overlapping topics, facilitating efficient comprehension of video content and quick access to specifi…
Skip-Layer Attention: Bridging Abstract and Detailed Dependencies in Transformers
Qian Chen, Wen Wang, Qinglin Zhang +7
The Transformer architecture has significantly advanced deep learning, particularly in natural language processing, by effectively managing long-range dependencies. However, as the…
Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
Qian Chen, Wen Wang, Qinglin Zhang +7
Recently, unified speech-text models, such as SpeechGPT, VioLA, and AudioPaLM, have achieved remarkable performance on various speech tasks. These models discretize speech signals…
Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling
Hai Yu, Chong Deng, Qinglin Zhang +3
Topic segmentation is critical for obtaining structured documents and improving downstream tasks such as information retrieval. Due to its ability of automatically exploring clues…
Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagation
Luyao Cheng, Siqi Zheng, Qinglin Zhang +4
Speaker diarization has gained considerable attention within speech processing research community. Mainstream speaker diarization rely primarily on speakers' voice characteristics…
Improving BERT with Hybrid Pooling Network and Drop Mask
Qian Chen, Wen Wang, Qinglin Zhang +3
Transformer-based pre-trained language models, such as BERT, achieve great success in various natural language understanding tasks. Prior research found that BERT captures a rich h…
Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Control
Yunzhe Li, Qian Chen, Weixiang Yan +3
Existing works on outline-conditioned text generation typically aim to generate text using provided outlines as rough sketches, such as keywords and phrases. However, these approac…
Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings
Qian Chen, Wen Wang, Qinglin Zhang +6
Prior studies diagnose the anisotropy problem in sentence representations from pre-trained language models, e.g., BERT, without fine-tuning. Our analysis reveals that the sentence…
Meeting Action Item Detection with Regularized Context Modeling
Jiaqing Liu, Chong Deng, Qinglin Zhang +2
Meetings are increasingly important for collaborations. Action items in meeting transcripts are crucial for managing post-meeting to-do tasks, which usually are summarized laboriou…
MUG: A General Meeting Understanding and Generation Benchmark
Qinglin Zhang, Chong Deng, Jiaqing Liu +7
Listening to long video/audio recordings from video conferencing and online courses for acquiring information is extremely inefficient. Even after ASR systems transcribe recordings…
Overview of the ICASSP 2023 General Meeting Understanding and Generation Challenge (MUG)
Qinglin Zhang, Chong Deng, Jiaqing Liu +7
ICASSP2023 General Meeting Understanding and Generation Challenge (MUG) focuses on prompting a wide range of spoken language processing (SLP) research on meeting transcripts, as SL…
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect
Jinglin Liu, Zhenhui Ye, Qian Chen +4
Recently, binaural audio synthesis (BAS) has emerged as a promising research field for its applications in augmented and virtual realities. Binaural audio helps users orient themse…
PoNet: Pooling Network for Efficient Token Mixing in Long Sequences
Chao-Hong Tan, Qian Chen, Wen Wang +3
Transformer-based models have achieved great success in various NLP, vision, and speech tasks. However, the core of Transformer, the self-attention mechanism, has a quadratic time…
Sequence Model with Self-Adaptive Sliding Window for Efficient Spoken Document Segmentation
Qinglin Zhang, Qian Chen, Yali Li +2
Transcripts generated by automatic speech recognition (ASR) systems for spoken documents lack structural annotations such as paragraphs, significantly reducing their readability. A…
Pre-training for Spoken Language Understanding with Joint Textual and Phonetic Representation Learning
Qian Chen, Wen Wang, Qinglin Zhang
In the traditional cascading architecture for spoken language understanding (SLU), it has been observed that automatic speech recognition errors could be detrimental to the perform…