Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuning

Congrui Du, Yang Zhang, Kaizhi Qian +1

Instruction tuning for speech language models (SLMs) is substantially more challenging than for text-based large language models (LLMs), as it requires learning a new modality and…