2 papers
cs.CL2026
GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
Kaiyuan Tian, Yu Tang, Gongqingjian Jiang +5
Full-parameter fine-tuning of large language models is constrained by substantial GPU memory requirements. Low-rank adaptation methods mitigate this challenge by updating only a su…
cs.LG2025
A Survey on Memory-Efficient Transformer-Based Model Training in AI for Science
Kaiyuan Tian, Linbo Qiao, Baihui Liu +3
Scientific research faces high costs and inefficiencies with traditional methods, but the rise of deep learning and large language models (LLMs) offers innovative solutions. This s…