2 papers
cs.LG2025
A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
Yiming Chen, Yuan Zhang, Yin Liu +2
The memory challenges associated with training Large Language Models (LLMs) have become a critical concern, particularly when using the Adam optimizer. To address this issue, numer…
cs.LG2024
Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures
Yiming Chen, Yuan Zhang, Liyuan Cao +2
Parameter-efficient fine-tuning (PEFT) significantly reduces memory costs when adapting large language models (LLMs) for downstream applications. However, traditional first-order (…