5 papers · 1 filter
A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
Junyu Luo, Bohan Wu, Xiao Luo +8
Post-training of Large Language Models (LLMs) is crucial for unlocking their task generalization potential and domain-specific capabilities. However, the current LLM post-training…
ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
Kangjie Zheng, Junwei Yang, Siyue Liang +5
Masked Language Models (MLMs) have achieved remarkable success in many self-supervised representation learning tasks. MLMs are trained by randomly masking portions of the input seq…
MARCO: Meta-Reflection with Cross-Referencing for Code Reasoning
Yusheng Zhao, Xiao Luo, Weizhi Zhang +4
The ability to reason is one of the most fundamental capabilities of large language models (LLMs), enabling a wide range of downstream tasks through sophisticated problem-solving.…
Large Language Model Agent: A Survey on Methodology, Applications and Challenges
Junyu Luo, Weizhi Zhang, Ye Yuan +23
The era of intelligent agents is upon us, driven by revolutionary advancements in large language models. Large Language Model (LLM) agents, with goal-driven behaviors and dynamic a…
Semi-supervised Fine-tuning for Large Language Models
Junyu Luo, Xiao Luo, Xiusi Chen +3
Supervised fine-tuning (SFT) is crucial in adapting large language model (LLMs) to a specific domain or task. However, only a limited amount of labeled data is available in practic…