Length Generalization Needs Proper Regularization

Pavlo Vasylenko, Matthias Lindemann, André F. T. Martins +1

Length generalization is the ability of sequential models to perform well on context lengths unseen during training. In this work, we show that the challenge of achieving length ge…