7 papers
CALM: Joint Contextual Acoustic-Linguistic Modeling for Personalization of Multi-Speaker ASR
Muhammad Shakeel, Yosuke Fukumoto, Chikara Maeda +2
We present CALM, a joint Contextual Acoustic-Linguistic Modeling framework for multi-speaker automatic speech recognition (ASR). In personalized AI scenarios, the joint availabilit…
Unifying Diarization, Separation, and ASR with Multi-Speaker Encoder
Muhammad Shakeel, Yui Sudo, Yifan Peng +2
This paper presents a unified multi-speaker encoder (UME), a novel architecture that jointly learns representations for speaker diarization (SD), speech separation (SS), and multi-…
Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics
Naohiro Tawara, Samuele Cornell, Alexander Polok +3
Conversational automatic speech recognition remains challenging due to overlapping speech, far-field noise, and varying speaker counts. While recent LLM-based systems perform well…
Improving Multilingual Speech Models on ML-SUPERB 2.0: Fine-tuning with Data Augmentation and LID-Aware CTC
Qingzheng Wang, Jiancheng Sun, Yifan Peng +1
Multilingual speech processing with self-supervised or supervised pre-trained Speech Foundation Models (SFM) has achieved strong performance on tasks like Language Identification (…
DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition
Yui Sudo, Yosuke Fukumoto, Muhammad Shakeel +3
Contextual biasing (CB) improves automatic speech recognition for rare and unseen phrases. Recent studies have introduced dynamic vocabulary, which represents context phrases as ex…
OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning
Yifan Peng, Shakeel Muhammad, Yui Sudo +4
The Open Whisper-style Speech Models (OWSM) project has developed a series of fully open speech foundation models using academic-scale resources, but their training data remains in…