Two are better than one: Context window extension with multi-grained self-injection

Wei Han, Pan Zhou, Soujanya Poria +1

The limited context window of contemporary large language models (LLMs) remains a huge barrier to their broader application across various domains. While continual pre-training on…