4 papers · 1 filter
Residual Feature Integration is Sufficient to Prevent Negative Transfer
Yichen Xu, Ryumei Nakada, Linjun Zhang +1
Transfer learning has become a central paradigm in modern machine learning, yet it suffers from the long-standing problem of negative transfer, where leveraging source representati…
PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers
Yibo Zhong, Haoxiang Jiang, Lincan Li +5
Fine-tuning large pre-trained foundation models often yields excellent downstream performance but is prohibitively expensive when updating all parameters. Parameter-efficient fine-…
A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
Ryumei Nakada, Wenlong Ji, Tianxi Cai +2
Prompt engineering has emerged as a powerful technique for guiding large language models (LLMs) toward desired responses, significantly enhancing their performance across diverse t…
SFT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
Xinyu Yang, Jixuan Leng, Geyang Guo +5
Current PEFT methods for LLMs can achieve either high quality, efficient training, or scalable serving, but not all three simultaneously. To address this limitation, we investigate…