AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM

Ruchao Fan, Bo Ren, Yuxuan Hu +3

Integrating speech into LLM (speech-LLM) has gaining increased attention recently. The mainstream solution is to connect a well-trained speech encoder and LLM with a neural adapter…