2 citations · 7 across the 10 of their papers we have counts for
7 papers · 1 filter
Rethinking Personalization in Large Language Models at the Token Level
Chenheng Zhang, Yijun Lu, Lizhe Fang +7
With large language models (LLMs) now performing strongly across diverse tasks, there is growing demand for them to personalize outputs for individual users. Personalization is typ…
Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
Yi Lu, Wanxu Zhao, Xin Zhou +9
Large Language Models (LLMs) often struggle to process and generate coherent context when the number of input tokens exceeds the pre-trained length. Recent advancements in long-con…
A Controlled Study on Long Context Extension and Generalization in LLMs
Yi Lu, Jing Nathan Yan, Songlin Yang +6
Broad textual understanding and in-context learning require language models that utilize full document contexts. Due to the implementation challenges associated with directly train…
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models
Wei He, Shichun Liu, Jun Zhao +6
Large language models (LLMs) have shown promising abilities of in-context learning (ICL), adapting swiftly to new tasks with only few-shot demonstrations. However, current few-shot…
LongHeads: Multi-Head Attention is Secretly a Long Context Processor
Yi Lu, Xin Zhou, Wei He +5
Large language models (LLMs) have achieved impressive performance in numerous domains but often struggle to process lengthy inputs effectively and efficiently due to limited length…
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
Jun Zhao, Can Zu, Hao Xu +6
Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks. However, LLMs with long context windows have…
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection
Chenglong Wang, Yi Lu, Yongyu Mu +3
Knowledge distillation addresses the problem of transferring knowledge from a teacher model to a student model. In this process, we typically have multiple types of knowledge extra…