4 papers
LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis
Qingyue Zhang, Chang Chu, Tianren Peng +4
LoRA has become a widely adopted method for PEFT, and its initialization methods have attracted increasing attention. However, existing methods have notable limitations: many metho…
Unified Optimization of Source Weights and Transfer Quantities in Multi-Source Transfer Learning: An Asymptotic Framework
Qingyue Zhang, Chang Chu, Haohao Fu +5
In multi-source transfer learning, a key challenge lies in how to appropriately differentiate and utilize heterogeneous source tasks. However, existing multi-source methods typical…
A High-Dimensional Statistical Method for Optimizing Transfer Quantities in Multi-Source Transfer Learning
Qingyue Zhang, Haohao Fu, Guanbo Huang +7
Multi-source transfer learning provides an effective solution to data scarcity in real-world supervised learning scenarios by leveraging multiple source tasks. In this field, exist…
On Theoretical Interpretations of Concept-Based In-Context Learning
Huaze Tang, Tianren Peng, Shao-lun Huang
In-Context Learning (ICL) has emerged as an important new paradigm in natural language processing and large language model (LLM) applications. However, the theoretical understandin…