5 papers
Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation
Zichong Li, Chen Liang, Liliang Ren +3
Large language models (LLMs) increasingly operate in settings that require reliable long-context understanding, such as retrieval-augmented generation and multi-document reasoning.…
COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs
Liming Liu, Zhenghao Xu, Zixuan Zhang +5
Large Language Models (LLMs) have demonstrated remarkable success across various domains, yet their optimization remains a significant challenge due to the complex and high-dimensi…
NorMuon: Making Muon more efficient and scalable
Zichong Li, Liming Liu, Chen Liang +2
The choice of optimizer significantly impacts the training efficiency and computational costs of large language models (LLMs). Recently, the Muon optimizer has demonstrated promisi…
LLMs Can Generate a Better Answer by Aggregating Their Own Responses
Zichong Li, Xinyu Feng, Yuheng Cai +6
Large Language Models (LLMs) have shown remarkable capabilities across tasks, yet they often require additional prompting techniques when facing complex problems. While approaches…
MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning
Zichong Li, Filip Bjelonic, Victor Klemm +1
Robot soccer, in its full complexity, poses an unsolved research challenge. Current solutions heavily rely on engineered heuristic strategies, which lack robustness and adaptabilit…