61 citations · 199 across the 16 of their papers we have counts for
18 papers
Aligning Multimodal Representations through an Information Bottleneck
Antonio Almudévar, José Miguel Hernández-Lobato, Sameer Khurana +2
Contrastive losses have been extensively used as a tool for multimodal representation learning. However, it has been empirically observed that their use is not effective to learn a…
LFAR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models
Santiago Cuervo, Adel Moumen, Yanis Labrak +5
Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as audio and images while effectively levera…
Direct Text to Speech Translation System using Acoustic Units
Victoria Mingote, Pablo Gimeno, Luis Vicente +3
This paper proposes a direct text to speech translation system using discrete acoustic units. This framework employs text in different source languages as input to generate speech…
Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers
Yuan Gong, Sameer Khurana, Leonid Karlinsky +1
In this paper, we focus on Whisper, a recent automatic speech recognition model trained with a massive 680k hour labeled speech corpus recorded in diverse conditions. We first show…
Improved Cross-Lingual Transfer Learning For Automatic Speech Translation
Sameer Khurana, Nauman Dawalatabad, Antoine Laurent +4
Research in multilingual speech-to-text translation is topical. Having a single model that supports multiple translation tasks is desirable. The goal of this work it to improve cro…
Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages
Andrew Rouditchenko, Sameer Khurana, Samuel Thomas +6
Recent models such as XLS-R and Whisper have made multilingual speech technologies more accessible by pre-training on audio from around 100 spoken languages each. However, there ar…
On Unsupervised Uncertainty-Driven Speech Pseudo-Label Filtering and Model Calibration
Nauman Dawalatabad, Sameer Khurana, Antoine Laurent +1
Pseudo-label (PL) filtering forms a crucial part of Self-Training (ST) methods for unsupervised domain adaptation. Dropout-based Uncertainty-driven Self-Training (DUST) proceeds by…
SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation
Sameer Khurana, Antoine Laurent, James Glass
We propose the SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation learning framework. Unlike previous works on speech representation lea…
CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification
Yuan Gong, Sameer Khurana, Andrew Rouditchenko +1
Audio classification is an active research area with a wide range of applications. Over the past decade, convolutional neural networks (CNNs) have been the de-facto standard buildi…
Magic dust for cross-lingual adaptation of monolingual wav2vec-2.0
Sameer Khurana, Antoine Laurent, James Glass
We propose a simple and effective cross-lingual transfer learning method to adapt monolingual wav2vec-2.0 models for Automatic Speech Recognition (ASR) in resource-scarce languages…
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
Cheng-I Jeff Lai, Yang Zhang, Alexander H. Liu +7
Self-supervised speech representation learning (speech SSL) has demonstrated the benefit of scale in learning rich representations for Automatic Speech Recognition (ASR) with limit…
Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training
Sameer Khurana, Niko Moritz, Takaaki Hori +1
The performance of automatic speech recognition (ASR) systems typically degrades significantly when the training and test data domains are mismatched. In this paper, we show that s…
CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning
Sameer Khurana, Antoine Laurent, James Glass
More than half of the 7,000 languages in the world are in imminent danger of going extinct. Traditional methods of documenting language proceed by collecting audio data followed by…
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning
Sameer Khurana, Antoine Laurent, Wei-Ning Hsu +4
Probabilistic Latent Variable Models (LVMs) provide an alternative to self-supervised learning approaches for linguistic representation learning from speech. LVMs admit an intuitiv…
Robust Training of Vector Quantized Bottleneck Models
Adrian Łańcucki, Jan Chorowski, Guillaume Sanchez +6
In this paper we demonstrate methods for reliable and efficient training of discrete representation using Vector-Quantized Variational Auto-Encoder models (VQ-VAEs). Discrete laten…
DARTS: Dialectal Arabic Transcription System
Sameer Khurana, Ahmed Ali, James Glass
We present the speech to text transcription system, called DARTS, for low resource Egyptian Arabic dialect. We analyze the following; transfer learning from high resource broadcast…
Multi-view Dimensionality Reduction for Dialect Identification of Arabic Broadcast Speech
Sameer Khurana, Ahmed Ali, Steve Renals
In this work, we present a new Vector Space Model (VSM) of speech utterances for the task of spoken dialect identification. Generally, DID systems are built using two sets of featu…
Automatic Dialect Detection in Arabic Broadcast Speech
Ahmed Ali, Najim Dehak, Patrick Cardinal +5
We investigate different approaches for dialect identification in Arabic broadcast speech, using phonetic, lexical features obtained from a speech recognition system, and acoustic…