Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
Yuxi Liu, Renjia Deng, Yutong He +3
The substantial memory demands of pre-training and fine-tuning large language models (LLMs) require memory-efficient optimization algorithms. One promising approach is layer-wise o…
cs.LG2025
CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure
Boao Kong, Junzhu Liang, Yuxi Liu +2
Low-rank architectures have become increasingly important for efficient large language model (LLM) pre-training, providing substantial reductions in both parameter complexity and m…