Rethinking Perplexity: Revealing the Impact of Input Length on Perplexity Evaluation in LLMs

Letian Cheng, Junyan Wang, Yan Gao +3

Perplexity is a widely adopted metric for assessing the predictive quality of large language models (LLMs) and often serves as a reference metric for downstream evaluations. Howeve…