Preserving Speech-to-Text LLM Capabilities in Speech-to-Speech Generation

Yuxuan Hu, Heng Lu, Ruchao Fan +8

Strong speech-to-text (S2T) LLMs already provide robust speech perception and text reasoning, but adding speech-to-speech (S2S) output is challenging: fine-tuning the backbone can…