Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations

Xue Jiang, Xiulian Peng, Yuan Zhang +1

Current large speech language models are mainly based on semantic tokens from discretization of self-supervised learned representations and acoustic tokens from a neural codec, fol…