Predicting LLM Output Length via Entropy-Guided Representations

Huanyi Xie, Yubin Chen, Liangyu Wang +2

The long-tailed distribution of sequence lengths in LLM serving and reinforcement learning (RL) sampling causes significant computational waste due to excessive padding in batched…