SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation

Zhehuai Chen, He Huang, Andrei Andrusenko +6

We present a novel Speech Augmented Language Model (SALM) with {\em multitask} and {\em in-context} learning capabilities. SALM comprises a frozen text LLM, a audio encoder, a moda…