2 papers
cs.LG2026
WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
Jiacheng Li, Jianchao Tan, Zhidong Yang +11
Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architect…
cs.LG2026
AFA-LoRA: Enabling Non-Linear Adaptations in LoRA with Activation Function Annealing
Jiacheng Li, Jianchao Tan, Zhidong Yang +4
Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning (PEFT) method. However, its linear adaptation process limits its expressive power. This means there i…