5 papers
The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View
Xinhao Yao, Lu Yu, Xiaolin Hu +4
The ongoing debate on whether reinforcement learning with verifiable rewards (RLVR) expands or shrinks the reasoning capabilities of large language models (LLMs) remains unresolved…
Put the Space of LoRA Initialization to the Extreme to Preserve Pre-trained Knowledge
Pengwei Tang, Xiaolin Hu, Yong Liu +4
Low-Rank Adaptation (LoRA) is the leading parameter-efficient fine-tuning method for Large Language Models (LLMs), but it still suffers from catastrophic forgetting. Recent work ha…
Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
Zixuan Gong, Xiaolin Hu, Huayi Tang +1
Large language models (LLMs) have demonstrated remarkable in-context learning (ICL) abilities. However, existing theoretical analysis of ICL primarily exhibits two limitations: (a)…
Chemical knowledge-informed framework for privacy-aware retrosynthesis learning
Guikun Chen, Xu Zhang, Xiaolin Hu +3
Chemical reaction data is a pivotal asset, driving advances in competitive fields such as pharmaceuticals, materials science, and industrial chemistry. Its proprietary nature rende…
ADePT: Adaptive Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning
Pengwei Tang, Xiaolin Hu, Yong Liu
Prompt Tuning (PT) enables the adaptation of Pre-trained Large Language Models (PLMs) to downstream tasks by optimizing a small amount of soft virtual tokens, which are prepended t…