LongQLoRA: Efficient and Effective Method to Extend Context Length of Large Language Models

Jianxin Yang

We present LongQLoRA, an efficient and effective method to extend context length of large language models with less training resources. LongQLoRA combines the advantages of Positio…