40 citations · 54 across the 13 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Taming Transformer Without Using Learning Rate Warmup
Xianbiao Qi, Yelin He, Jiaquan Ye +5
Scaling Transformer to a large scale without using some technical tricks such as learning rate warump and using an obviously lower learning rate is an extremely challenging task, a…
cs.LG2023★ 8 cited
Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices
Bojia Zi, Xianbiao Qi, Lingzhi Wang +3
In this paper, we present Delta-LoRA, which is a novel parameter-efficient approach to fine-tune large language models (LLMs). In contrast to LoRA and other low-rank adaptation met…