19 citations · 28 across the 26 of their papers we have counts for
27 papers
SphereVBx: Spherical Variational Bayes Clustering for Simplified EEND-VC Diarization
Petr Pálka, Jiangyu Han, Prachi Singh +3
We propose SphereVBx, a Bayesian clustering framework for hyperspherical embeddings based on Toroidal Probabilistic Spherical Discriminant Analysis (T-PSDA). The method follows the…
Grounding Spoken LLMs in Multi-Speaker Audio via Diarization Conditioning
Alexander Polok, Samuele Cornell, Sathvik Udupa +3
We propose diarization-conditioned spoken language models (SLMs), a strategy for extending SLMs to far-field multi-talker audio. Rather than adapting the decoder via Serialized Out…
BUT System Description for CHiME-9 MCoRec Challenge
Dominik Klement, Alexander Polok, Nguyen Hai Phong +2
Multi-talker automatic speech recognition (ASR) in conversational recordings remains an open problem, particularly in scenarios with large portion of overlapping speech where ident…
Who Spoke What When? Evaluating Spoken Language Models for Conversational ASR with Semantic and Overlap-Aware Metrics
Naohiro Tawara, Samuele Cornell, Alexander Polok +3
Conversational automatic speech recognition remains challenging due to overlapping speech, far-field noise, and varying speaker counts. While recent LLM-based systems perform well…
Modeling Overlapped Speech with Shuffles
Matthew Wiesner, Samuele Cornell, Alexander Polok +3
We propose to model parallel streams of data, such as overlapped speech, using shuffles. Specifically, this paper shows how the shuffle product and partial order finite-state autom…
On the Role of Spatial Features in Foundation-Model-Based Speaker Diarization
Marc Deegen, Tobias Gburrek, Tobias Cord-Landwehr +4
Recent advances in speaker diarization exploit large pretrained foundation models, such as WavLM, to achieve state-of-the-art performance on multiple datasets. Systems like DiariZe…
DeCRED: Decoder-Centric Regularization for Encoder-Decoder Based Speech Recognition
Alexander Polok, Santosh Kesiraju, Karel Beneš +3
This paper presents a simple yet effective regularization for the internal language model induced by the decoder in encoder-decoder ASR models, thereby improving robustness and gen…
BUT System for the MLC-SLM Challenge
Alexander Polok, Jiangyu Han, Dominik Klement +3
We present a two-speaker automatic speech recognition (ASR) system that combines DiCoW -- a diarization-conditioned variant of Whisper -- with DiariZen, a diarization pipeline buil…
Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models
Jiangyu Han, Petr Pálka, Marc Delcroix +4
This work presents a framework for compressing self-supervised models for speaker diarization through structured pruning guided by knowledge distillation. We investigate pruning ob…
Analysis of ABC Frontend Audio Systems for the NIST-SRE24
Sara Barahona, Anna Silnova, Ladislav Mošner +14
We present a comprehensive analysis of the embedding extractors (frontends) developed by the ABC team for the audio track of NIST SRE 2024. We follow the two scenarios imposed by N…
Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization
Petr Pálka, Federico Landini, Dominik Klement +4
In spite of the popularity of end-to-end diarization systems nowadays, modular systems comprised of voice activity detection (VAD), speaker embedding extraction plus clustering, an…
Improving Automatic Speech Recognition with Decoder-Centric Regularisation in Encoder-Decoder Models
Alexander Polok, Santosh Kesiraju, Karel Beneš +2
This paper proposes a simple yet effective way of regularising the encoder-decoder-based automatic speech recognition (ASR) models that enhance the robustness of the model and impr…
State-of-the-art Embeddings with Video-free Segmentation of the Source VoxCeleb Data
Sara Barahona, Ladislav Mošner, Themos Stafylakis +4
In this paper, we refine and validate our method for training speaker embedding extractors using weak annotations. More specifically, we use only the audio stream of the source Vox…
CA-MHFA: A Context-Aware Multi-Head Factorized Attentive Pooling for SSL-Based Speaker Verification
Junyi Peng, Ladislav Mošner, Lin Zhang +4
Self-supervised learning (SSL) models for speaker verification (SV) have gained significant attention in recent years. However, existing SSL-based SV systems often struggle to capt…
BUT Systems and Analyses for the ASVspoof 5 Challenge
Johan Rohdin, Lin Zhang, Oldřich Plchot +8
This paper describes the BUT submitted systems for the ASVspoof 5 challenge, along with analyses. For the conventional deepfake detection task, we use ResNet18 and self-supervised…
Beyond the Labels: Unveiling Text-Dependency in Paralinguistic Speech Recognition Datasets
Jan Pešán, Santosh Kesiraju, Lukáš Burget +1
Paralinguistic traits like cognitive load and emotion are increasingly recognized as pivotal areas in speech recognition research, often examined through specialized datasets like…
Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information?
Lin Zhang, Themos Stafylakis, Federico Landini +3
In this paper, we apply the variational information bottleneck approach to end-to-end neural diarization with encoder-decoder attractors (EEND-EDA). This allows us to investigate w…
DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors
Federico Landini, Mireia Diez, Themos Stafylakis +1
Until recently, the field of speaker diarization was dominated by cascaded systems. Due to their limitations, mainly regarding overlapped speech and cumbersome pipelines, end-to-en…
Discriminative Training of VBx Diarization
Dominik Klement, Mireia Diez, Federico Landini +4
Bayesian HMM clustering of x-vector sequences (VBx) has become a widely adopted diarization baseline model in publications and challenges. It uses an HMM to model speaker turns, a…
Improving Speaker Verification with Self-Pretrained Transformer Models
Junyi Peng, Oldřich Plchot, Themos Stafylakis +3
Recently, fine-tuning large pre-trained Transformer models using downstream datasets has received a rising interest. Despite their success, it is still challenging to disentangle t…
Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapters
Junyi Peng, Themos Stafylakis, Rongzhi Gu +4
Recently, the pre-trained Transformer models have received a rising interest in the field of speech processing thanks to their great success in various downstream tasks. However, m…
Probabilistic Spherical Discriminant Analysis: An Alternative to PLDA for length-normalized embeddings
Niko Brümmer, Albert Swart, Ladislav Mošner +4
In speaker recognition, where speech segments are mapped to embeddings on the unit hypersphere, two scoring backends are commonly used, namely cosine scoring or PLDA. Both have adv…
Training Speaker Embedding Extractors Using Multi-Speaker Audio with Unknown Speaker Boundaries
Themos Stafylakis, Ladislav Mošner, Oldřich Plchot +4
In this paper, we demonstrate a method for training speaker embedding extractors using weak annotation. More specifically, we are using the full VoxCeleb recordings and the name of…
A Bayesian Multilingual Document Model for Zero-shot Topic Identification and Discovery
Santosh Kesiraju, Sangeet Sagar, Ondřej Glembek +3
In this paper, we present a Bayesian multilingual document model for learning language-independent document embeddings. The model is an extension of BaySMM [Kesiraju et al 2020] to…