20 citations · 67 across the 10 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2023★ 8 cited
Delta-LoRA: Fine-Tuning High-Rank Parameters with the Delta of Low-Rank Matrices
Bojia Zi, Xianbiao Qi, Lingzhi Wang +3
In this paper, we present Delta-LoRA, which is a novel parameter-efficient approach to fine-tune large language models (LLMs). In contrast to LoRA and other low-rank adaptation met…
cs.LG2023
Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant
Xianbiao Qi, Jianan Wang, Lei Zhang
This article provides a comprehensive understanding of optimization in deep learning, with a primary focus on the challenges of gradient vanishing and gradient exploding, which nor…