30 citations · 36 across the 16 of their papers we have counts for
17 papers
Streaming Speech-to-Text Translation with a SpeechLLM
Titouan Parcollet, Shucong Zhang, Xianrui Zheng +1
Normally, a system that translates speech into text consists of separate modules for speech recognition and text-to-text translation. Combining those tasks into a SpeechLLM promise…
DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
Haaris Mehmood, Jie Xu, Karthikeyan Saravanan +2
Federated learning (FL) enables the collaborative training of large-scale language models (LLMs) across edge devices while keeping user data on-device. However, FL still exposes se…
Robust Unsupervised Adaptation of a Speech Recogniser Using Entropy Minimisation and Speaker Codes
Rogier C. van Dalen, Shucong Zhang, Titouan Parcollet +1
Speech recognisers usually perform optimally only in a specific environment and need to be adapted to work well in another. For adaptation to a new speaker, there is often too litt…
Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition
Yuan Tseng, Titouan Parcollet, Rogier van Dalen +2
Recent work suggests that large language models (LLMs) can improve performance of speech tasks compared to existing systems. To support their claims, results on LibriSpeech and Com…
Loquacious Set: 25,000 Hours of Transcribed and Diverse English Speech Recognition Data for Research and Commercial Use
Titouan Parcollet, Yuan Tseng, Shucong Zhang +1
Automatic speech recognition (ASR) research is driven by the availability of common datasets between industrial researchers and academics, encouraging comparisons and evaluations.…
Benchmarking Rotary Position Embeddings for Automatic Speech Recognition
Shucong Zhang, Titouan Parcollet, Rogier van Dalen +1
Self-attention relies on positional embeddings to encode input order. Relative Position (RelPos) embeddings are widely used in Automatic Speech Recognition (ASR). However, RelPos h…
Linear Time Complexity Conformers with SummaryMixing for Streaming Speech Recognition
Titouan Parcollet, Rogier van Dalen, Shucong Zhang +1
Automatic speech recognition (ASR) with an encoder equipped with self-attention, whether streaming or non-streaming, takes quadratic time in the length of the speech utterance. Thi…
Linear-Complexity Self-Supervised Learning for Speech Processing
Shucong Zhang, Titouan Parcollet, Rogier van Dalen +1
Self-supervised learning (SSL) models usually require weeks of pre-training with dozens of high-end GPUs. These models typically have a multi-headed self-attention (MHSA) context e…
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation
Jie Xu, Karthikeyan Saravanan, Rogier van Dalen +3
Federated learning (FL) allows clients to collaboratively train a global model without sharing their local data with a server. However, clients' contributions to the server can sti…
pfl-research: simulation framework for accelerating research in Private Federated Learning
Filip Granqvist, Congzheng Song, Áine Cahill +7
Federated learning (FL) is an emerging machine learning (ML) training paradigm where clients own their data and collaborate to train a global model, without revealing any data to t…
Globally Normalising the Transducer for Streaming Speech Recognition
Rogier van Dalen
The Transducer (e.g. RNN-Transducer or Conformer-Transducer) generates an output label sequence as it traverses the input sequence. It is straightforward to use in streaming mode,…
SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding
Titouan Parcollet, Rogier van Dalen, Shucong Zhang +1
Modern speech processing systems rely on self-attention. Unfortunately, token mixing with self-attention takes quadratic time in the length of the speech utterance, slowing down in…
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices
Mingbin Xu, Congzheng Song, Ye Tian +10
Federated Learning (FL) is a technique to train models using data distributed across devices. Differential Privacy (DP) provides a formal privacy guarantee for sensitive data. Our…
Training a Tokenizer for Free with Private Federated Learning
Eugene Bagdasaryan, Congzheng Song, Rogier van Dalen +2
Federated learning with differential privacy, i.e. private federated learning (PFL), makes it possible to train models on private data distributed across users' devices without har…
Enforcing fairness in private federated learning via the modified method of differential multipliers
Borja Rodríguez-Gálvez, Filip Granqvist, Rogier van Dalen +1
Federated learning with differential privacy, or private federated learning, provides a strategy to train machine learning models while respecting users' privacy. However, differen…
Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications
Matthias Paulik, Matt Seigel, Henry Mason +19
We describe the design of our federated task processing system. Originally, the system was created to support two specific federated tasks: evaluation and tuning of on-device ML sy…
Improving on-device speaker verification using federated learning with privacy
Filip Granqvist, Matt Seigel, Rogier van Dalen +3
Information on speaker characteristics can be useful as side information in improving speaker recognition accuracy. However, such information is often private. This paper investiga…