2 citations · 7 across the 10 of their papers we have counts for
4 papers · 1 filter
A Controlled Study on Long Context Extension and Generalization in LLMs
Yi Lu, Jing Nathan Yan, Songlin Yang +6
Broad textual understanding and in-context learning require language models that utilize full document contexts. Due to the implementation challenges associated with directly train…
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models
Wei He, Shichun Liu, Jun Zhao +6
Large language models (LLMs) have shown promising abilities of in-context learning (ICL), adapting swiftly to new tasks with only few-shot demonstrations. However, current few-shot…
LongHeads: Multi-Head Attention is Secretly a Long Context Processor
Yi Lu, Xin Zhou, Wei He +5
Large language models (LLMs) have achieved impressive performance in numerous domains but often struggle to process lengthy inputs effectively and efficiently due to limited length…
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
Jun Zhao, Can Zu, Hao Xu +6
Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks. However, LLMs with long context windows have…