4 papers
Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving
Ruchao Fan, Yiming Wang, Rui Zhao +10
Speech-LLM integration has shown promising results by leveraging extensive textual pretraining, yet its specific benefits for automatic speech recognition (ASR) remain unclear. We…
LLM can Read Spectrogram: Encoder-free Speech-Language Modeling
Ruchao Fan, Yiming Wang, Yuxuan Hu +6
Recent speech-aware large language models (Speech-LLMs) rely on pre-trained speech encoders to convert audio into semantic/acoustic rich representations consumable by LLM. In this…
Normalization through Fine-tuning: Understanding Wav2vec 2.0 Embeddings for Phonetic Analysis
Yiming Wang, Yi Yang, Jiahong Yuan
Phonetic normalization plays a crucial role in speech recognition and analysis, ensuring the comparability of features derived from raw audio data. However, in the current paradigm…
Automated Tone Transcription and Clustering with Tone2Vec
Yi Yang, Yiming Wang, ZhiQiang Tang +1
Lexical tones play a crucial role in Sino-Tibetan languages. However, current phonetic fieldwork relies on manual effort, resulting in substantial time and financial costs. This is…