LLM Probability Concentration: How Alignment Shrinks the Generative Horizon

Chenghao Yang, Sida Li, Ari Holtzman

Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investiga…