7 citations · 7 across the 8 of their papers we have counts for
4 papers · 1 filter
LLM can Read Spectrogram: Encoder-free Speech-Language Modeling
Ruchao Fan, Yiming Wang, Yuxuan Hu +6
Recent speech-aware large language models (Speech-LLMs) rely on pre-trained speech encoders to convert audio into semantic/acoustic rich representations consumable by LLM. In this…
RLBR: Reinforcement Learning with Biasing Rewards for Contextual Speech Large Language Models
Bo Ren, Ruchao Fan, Yelong Shen +2
Speech large language models (LLMs) have driven significant progress in end-to-end speech understanding and recognition, yet they continue to struggle with accurately recognizing r…
Towards Efficient Speech-Text Jointly Decoding within One Speech Language Model
Haibin Wu, Yuxuan Hu, Ruchao Fan +8
Speech language models (Speech LMs) enable end-to-end speech-text modeling within a single model, offering a promising direction for spoken dialogue systems. The choice of speech-t…
AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM
Ruchao Fan, Bo Ren, Yuxuan Hu +3
Integrating speech into LLM (speech-LLM) has gaining increased attention recently. The mainstream solution is to connect a well-trained speech encoder and LLM with a neural adapter…