Improving Diversity in Language Models: When Temperature Fails, Change the Loss

Alexandre Verine, Florian Le Bronnec, Kunhao Zheng +3

Increasing diversity in language models is a challenging yet essential objective. A common approach is to raise the decoding temperature. In this work, we investigate this approach…