4 citations · 9 across the 6 of their papers we have counts for
1 paper · 1 filter
Tongyao Zhu, Qian Liu, Haonan Wang +4
Recent advancements in LLM pretraining have featured ever-expanding context windows to process longer sequences. However, our pilot study reveals that models pretrained with shorte…