7 papers · 1 filter
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
Fei Wang, Chao Xue, Taoran Liu +3
Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints. We first identify a phenomenon tha…
Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
Fei Wang, Li Shen, Liang Ding +3
Zeroth-Order optimization presents a promising memory-efficient paradigm for fine-tuning Large Language Models by relying solely on forward passes. However, its practical adoption…
Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning
Zekai Lin, Chao Xue, Di Liang +8
Supervised Fine-Tuning (SFT) of large language models often suffers from task interference and catastrophic forgetting. Recent approaches alleviate this issue by isolating task-cri…
Fibration Policy Optimization
Chang Li, Tshihao Tsu, Yaren Zhang +2
Large language models are increasingly trained as heterogeneous systems spanning multiple domains, expert partitions, and agentic pipelines, yet prevalent proximal objectives opera…
ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learning
Wentao Tan, Qiong Cao, Chao Xue +3
The chart-to-code generation task requires MLLMs to convert chart images into executable code. This task faces two main challenges: limited data diversity and the difficulty of mai…
Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution
Wentao Tan, Qiong Cao, Yibing Zhan +2
Human preference alignment can greatly enhance Multimodal Large Language Models (MLLMs), but collecting high-quality preference data is costly. A promising solution is the self-evo…
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
Fei Wang, Li Shen, Liang Ding +3
Fine-tuning is powerful for adapting large language models to downstream tasks, but it often results in huge memory usages. A promising approach to mitigate this is using Zeroth-Or…