E^2-LLM: Efficient and Extreme Length Extension of Large Language Models

Jiaheng Liu, Zhiqi Bai, Yuanxing Zhang +11

Typically, training LLMs with long context sizes is computationally expensive, requiring extensive training hours and GPU resources. Existing long-context extension methods usually…