3 papers
cs.LG2026
AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
Ziye Chen, Hongbin Lin, Chenyu Zhang +3
Zeroth-order (ZO) optimization enables large-language-model fine-tuning without storing backpropagation activations, while LoRA supplies compact trainable adapters. Combining them…
cs.CL2025
In-Context Meta LoRA Generation
Yihua Shao, Minxi Yan, Yang Liu +12
Low-rank Adaptation (LoRA) has demonstrated remarkable capabilities for task specific fine-tuning. However, in scenarios that involve multiple tasks, training a separate LoRA model…
cs.LG2025
GWQ: Gradient-Aware Weight Quantization for Large Language Models
Yihua Shao, Yan Gu, Siyu Chen +12
Large language models (LLMs) show impressive performance in solving complex language tasks. However, its large number of parameters presents significant challenges for the deployme…