8 papers
Thinking Seeds: Leveraging Historical Diversity for Position-Aware RL in LLMs
Lei Yang, Wei Bi, Chenxi Sun +2
On-policy reinforcement learning (RL) for language model post-training suffers from a fundamental tension: as training progresses, policy entropy collapses and sampling diversity d…
Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQL
Hanqing Wang, Yongdong Chi, Jian Yang +4
While Large Language Models (LLMs) have achieved remarkable success in Text-to-SQL tasks, their deployment in real-world environments is hindered by latent reliability issues. Iden…
A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RL
Lei Yang, Siyu Ding, Deyi Xiong
Reinforcement learning (RL) post-training improves large language models (LLMs) on individual domains such as mathematical reasoning, code generation, question answering, and creat…
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
Lei Yang, Leiyu Pan, Bojian Xiong +14
Large language models (LLMs) have achieved remarkable success across a wide range of natural language processing tasks, yet their performance remains heavily biased toward high-res…
Representation-Guided Parameter-Efficient LLM Unlearning
Zeguan Xiao, Lang Mo, Yun Chen +4
Large Language Models (LLMs) often memorize sensitive or harmful information, necessitating effective machine unlearning techniques. While existing parameter-efficient unlearning m…
DEP: A Decentralized Large Language Model Evaluation Protocol
Jianxiang Peng, Junhao Li, Hongxiang Wang +15
With the rapid development of Large Language Models (LLMs), a large number of benchmarks have been proposed. However, most benchmarks lack unified evaluation standard and require t…