4 papers
Zero-resource Speech Translation and Recognition with LLMs
Karel Mundnich, Xing Niu, Prashant Mathur +10
Despite recent advancements in speech processing, zero-resource speech translation (ST) and automatic speech recognition (ASR) remain challenging problems. In this work, we propose…
SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models
Raghuveer Peri, Sai Muralidhar Jayanthi, Srikanth Ronanki +11
Integrated Speech and Large Language Models (SLMs) that can follow speech instructions and generate relevant text responses have gained popularity lately. However, the safety and r…
Don't Stop Self-Supervision: Accent Adaptation of Speech Representations via Residual Adapters
Anshu Bhatia, Sanchit Sinha, Saket Dingliwal +3
Speech representations learned in a self-supervised fashion from massive unlabeled speech corpora have been adapted successfully toward several downstream tasks. However, such repr…
Masked Audio Text Encoders are Effective Multi-Modal Rescorers
Jinglun Cai, Monica Sunkara, Xilai Li +3
Masked Language Models (MLMs) have proven to be effective for second-pass rescoring in Automatic Speech Recognition (ASR) systems. In this work, we propose Masked Audio Text Encode…