Rethinking Length-Based Training: Batch Composition and Loss Normalization in Speech Token Language Models

Hongjin Song, Runwu Shi, Weiqiao Shan +4

Short-to-long training is a simple curriculum for speech models, but its gains can be difficult to interpret. In speech token language models, length-based training can change the…