34 citations · 67 across the 41 of their papers we have counts for
18 papers · 1 filter
GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model
Haoyang Li, Xuyi Zhuang, Azmat Adnan +6
Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech. We propose GenT…
Next-Frame Feature Prediction for Multimodal Deepfake Detection and Temporal Localization
Ashutosh Anshul, Shreyas Gopal, Deepu Rajan +1
Recent multimodal deepfake detection methods designed for generalization conjecture that single-stage supervised training struggles to generalize across unseen manipulations and da…
Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception
Ziyang Ma, Ruiyang Xu, Zhenghao Xing +9
Fine-grained perception of multimodal information is critical for advancing human-AI interaction. With recent progress in audio-visual technologies, Omni Language Models (OLMs), ca…
Hierarchical Self-Supervised Representation Learning for Depression Detection from Speech
Yuxin Li, Eng Siong Chng, Cuntai Guan
Speech-based depression detection (SDD) has emerged as a non-invasive and scalable alternative to conventional clinical assessments. However, existing methods still struggle to cap…
Chronological Thinking in Full-Duplex Spoken Dialogue Language Models
Donghang Wu, Haoyang Zhang, Chen Chen +8
Recent advances in spoken dialogue language models (SDLMs) reflect growing interest in shifting from turn-based to full-duplex systems, where the models continuously perceive user…
Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods
Bingshen Mu, Pengcheng Guo, Zhaokai Sun +8
This paper summarizes the Interspeech2025 Multilingual Conversational Speech Language Model (MLC-SLM) challenge, which aims to advance the exploration of building effective multili…
Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Function
Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng
Rare word recognition can be improved by adapting ASR models to synthetic data that includes these words. Further improvements can be achieved through contextual biasing, which tra…
Zero-shot Context Biasing with Trie-based Decoding using Synthetic Multi-Pronunciation
Changsong Liu, Yizhou Peng, Eng Siong Chng
Contextual automatic speech recognition (ASR) systems allow for recognizing out-of-vocabulary (OOV) words, such as named entities or rare words. However, it remains challenging due…
Robust Localization of Partially Fake Speech: Metrics and Out-of-Domain Evaluation
Hieu-Thi Luong, Inbal Rimon, Haim Permuter +2
Partial audio deepfake localization poses unique challenges and remain underexplored compared to full-utterance spoofing detection. While recent methods report strong in-domain per…
FD-Bench: A Full-Duplex Benchmarking Pipeline Designed for Full Duplex Spoken Dialogue Systems
Yizhou Peng, Yi-Wen Chao, Dianwen Ng +4
Full-duplex spoken dialogue systems (FDSDS) enable more natural human-machine interactions by allowing real-time user interruptions and backchanneling, compared to traditional SDS…
Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR
Yizhou Peng, Hexin Liu, Eng Siong Chng
This paper introduces the integration of language-specific bi-directional context into a speech large language model (SLLM) to improve multilingual continuous conversational automa…
NTU Speechlab LLM-Based Multilingual ASR System for Interspeech MLC-SLM Challenge 2025
Yizhou Peng, Bin Wang, Yi-Wen Chao +5
This report details the NTU Speechlab system developed for the Interspeech 2025 Multilingual Conversational Speech and Language Model (MLC-SLM) Challenge (Task I), where we achieve…
Impact of Frame Rates on Speech Tokenizer: A Case Study on Mandarin and English
Haoyang Zhang, Hexin Liu, Xiangyu Zhang +7
The speech tokenizer plays a crucial role in recent speech tasks, generally serving as a bridge between speech signals and language models. While low-frame-rate codecs are widely e…
EASY: Emotion-aware Speaker Anonymization via Factorized Distillation
Jixun Yao, Hexin Liu, Eng Siong Chng +1
Emotion plays a significant role in speech interaction, conveyed through tone, pitch, and rhythm, enabling the expression of feelings and intentions beyond words to create a more p…
Multi-band Frequency Reconstruction for Neural Psychoacoustic Coding
Dianwen Ng, Kun Zhou, Yi-Wen Chao +3
Achieving high-fidelity audio compression while preserving perceptual quality across diverse content remains a key challenge in Neural Audio Coding (NAC). We introduce MUFFIN, a fu…
GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modeling
Jixun Yao, Hexin Liu, Chen Chen +3
Semantic information refers to the meaning conveyed through words, phrases, and contextual relationships within a given linguistic structure. Humans can leverage semantic informati…
Continual Learning with Embedding Layer Surgery and Task-wise Beam Search using Whisper
Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng
Current Multilingual ASR models only support a fraction of the world's languages. Continual Learning (CL) aims to tackle this problem by adding new languages to pre-trained models…
Audio Large Language Models Can Be Descriptive Speech Quality Evaluators
Chen Chen, Yuchen Hu, Siyin Wang +5
An ideal multimodal agent should be aware of the quality of its input modalities. Recent advances have enabled large language models (LLMs) to incorporate auditory systems for hand…