Bootstrap Your Own Context Length

Liang Wang, Nan Yang, Xingxing Zhang +2

We introduce a bootstrapping approach to train long-context language models by exploiting their short-context capabilities only. Our method utilizes a simple agent workflow to synt…