5 papers · 1 filter
SPEAR-Gen: Generation-Aware Pre-training for Unified Speech Representations
Xiaoyu Yang, Arthur Hinsvark, Antonios Alexos +3
Speech understanding and generation place different demands on speech representations, and existing models are typically optimised towards one capability or the other. To reduce th…
Text-Prompted CLAP: Learning Text-Conditioned Audio Representations via Contrastive Learning
Mohan Li, Rama Doddipatla, Philip C. Woodland
Contrastive Language-Audio Pretraining (CLAP) aligns text and audio in a shared embedding space, but encoding each modality independently limits its ability to model cross-modal se…
SALMONN-2: Advancing General-Purpose Hearing Abilities with Self-Supervised Representations
Xiaoyu Yang, Xuenan Xu, Wenyi Yu +10
Recent audio large language models (ALLMs) are typically built upon audio encoders trained with large amounts of supervised data. Since self-supervised learning (SSL) audio encoder…
Wav2Prompt: End-to-End Speech Prompt Generation and Tuning For LLM in Zero and Few-shot Learning
Keqi Deng, Guangzhi Sun, Philip C. Woodland
Wav2Prompt is proposed which allows straightforward integration between spoken input and a text-based large language model (LLM). Wav2Prompt uses a simple training process with onl…
FastInject: Injecting Unpaired Text Data into CTC-based ASR training
Keqi Deng, Philip C. Woodland
Recently, connectionist temporal classification (CTC)-based end-to-end (E2E) automatic speech recognition (ASR) models have achieved impressive results, especially with the develop…