15 papers
SpeakerCard-1M: An Evidence-Grounded Corpus for In-the-Wild Speaker Verification
Junyi Peng, Oldřich Plchot, Xiao Song +9
Modern speaker verification (SV) systems rely on speaker embeddings that are effective but difficult to interpret or query in natural language. Most existing speech-text corpora ta…
Grounding Spoken LLMs in Multi-Speaker Audio via Diarization Conditioning
Alexander Polok, Samuele Cornell, Sathvik Udupa +3
We propose diarization-conditioned spoken language models (SLMs), a strategy for extending SLMs to far-field multi-talker audio. Rather than adapting the decoder via Serialized Out…
DeCRED: Decoder-Centric Regularization for Encoder-Decoder Based Speech Recognition
Alexander Polok, Santosh Kesiraju, Karel Beneš +3
This paper presents a simple yet effective regularization for the internal language model induced by the decoder in encoder-decoder ASR models, thereby improving robustness and gen…
Hybrid Pruning: In-Situ Compression of Self-Supervised Speech Models for Speaker Verification and Anti-Spoofing
Junyi Peng, Lin Zhang, Jiangyu Han +5
Although large-scale self-supervised learning (SSL) models like WavLM have achieved state-of-the-art performance in speech processing, their significant size impedes deployment on…
BUT System for the MLC-SLM Challenge
Alexander Polok, Jiangyu Han, Dominik Klement +3
We present a two-speaker automatic speech recognition (ASR) system that combines DiCoW -- a diarization-conditioned variant of Whisper -- with DiariZen, a diarization pipeline buil…
Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
Pradyoth Hegde, Santosh Kesiraju, Jan Švec +5
This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use…
Approaching Dialogue State Tracking via Aligning Speech Encoders and LLMs
Šimon Sedláček, Bolaji Yusuf, Ján Švec +4
In this work, we approach spoken Dialogue State Tracking (DST) by bridging the representation spaces of speech encoders and LLMs via a small connector module, with a focus on fully…
Aligning Pre-trained Models for Spoken Language Translation
Šimon Sedláček, Santosh Kesiraju, Alexander Polok +1
This paper investigates a novel approach to end-to-end speech translation (ST) based on aligning frozen pre-trained automatic speech recognition (ASR) and machine translation (MT)…
Improving Automatic Speech Recognition with Decoder-Centric Regularisation in Encoder-Decoder Models
Alexander Polok, Santosh Kesiraju, Karel Beneš +2
This paper proposes a simple yet effective way of regularising the encoder-decoder-based automatic speech recognition (ASR) models that enhance the robustness of the model and impr…
State-of-the-art Embeddings with Video-free Segmentation of the Source VoxCeleb Data
Sara Barahona, Ladislav Mošner, Themos Stafylakis +4
In this paper, we refine and validate our method for training speaker embedding extractors using weak annotations. More specifically, we use only the audio stream of the source Vox…
CA-MHFA: A Context-Aware Multi-Head Factorized Attentive Pooling for SSL-Based Speaker Verification
Junyi Peng, Ladislav Mošner, Lin Zhang +4
Self-supervised learning (SSL) models for speaker verification (SV) have gained significant attention in recent years. However, existing SSL-based SV systems often struggle to capt…
Beyond the Labels: Unveiling Text-Dependency in Paralinguistic Speech Recognition Datasets
Jan Pešán, Santosh Kesiraju, Lukáš Burget +1
Paralinguistic traits like cognitive load and emotion are increasingly recognized as pivotal areas in speech recognition research, often examined through specialized datasets like…
Improving Speaker Verification with Self-Pretrained Transformer Models
Junyi Peng, Oldřich Plchot, Themos Stafylakis +3
Recently, fine-tuning large pre-trained Transformer models using downstream datasets has received a rising interest. Despite their success, it is still challenging to disentangle t…
Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters
Junyi Peng, Themos Stafylakis, Rongzhi Gu +4
Recently, the pre-trained Transformer models have received a rising interest in the field of speech processing thanks to their great success in various downstream tasks. However, m…
Training Speaker Embedding Extractors Using Multi-Speaker Audio with Unknown Speaker Boundaries
Themos Stafylakis, Ladislav Mošner, Oldřich Plchot +4
In this paper, we demonstrate a method for training speaker embedding extractors using weak annotation. More specifically, we are using the full VoxCeleb recordings and the name of…