10 papers
A Target-Centric Survey of Quantization-Aware Training
Jiamin Song, Mengjie Zhao, Zijing Wang +6
The rapid development of LLMs incurs prohibitive memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques have emerged as a promising sol…
TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory
Kang Liu, Zijing Wang, Yongkang Liu +5
Multimodal Large Reasoning Models (MLRMs) have achieved strong performance on tasks requiring visual understanding and multi-step inference. However, as reasoning trajectories grow…
Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models
Yichen Gao, Yiqun Zhang, Zijing Wang +7
Audio-language models (ALMs) often follow text that conflicts with audio, even when the audio evidence is clear. This raises a basic question: is the audio-supported answer unavail…
ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning
Yongkang Liu, Zijing Wang, Mengjie Zhao +7
This work presents \textsc{ChunkFT}, a memory-efficient fine-tuning framework that reformulates full-parameter fine-tuning around a dynamically activated working set. \textsc{Chunk…
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
Yongkang Liu, Xing Li, Mengjie Zhao +7
As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large language models. Low-rank Adaptation…
DiM\textsuperscript{3}: Bridging Multilingual and Multimodal Models via Direction- and Magnitude-Aware Merging
Zijing Wang, Mingyang Wang, Ercong Nie +6
Towards more general and human-like intelligence, large language models should seamlessly integrate both multilingual and multimodal capabilities; however, extending an existing mu…
PlaM: Training-Free Plateau-Guided Model Merging for Better Visual Grounding in MLLMs
Zijing Wang, Yongkang Liu, Mingyang Wang +8
Multimodal Large Language Models (MLLMs) rely on strong linguistic reasoning inherited from their base language models. However, multimodal instruction fine-tuning paradoxically de…
High-Rank Structured Modulation for Parameter-Efficient Fine-Tuning
Yongkang Liu, Xing Li, Mengjie Zhao +7
As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large language models. Low-rank Adaptation…
Why Do More Experts Fail? A Theoretical Analysis of Model Merging
Zijing Wang, Xingle Xu, Yongkang Liu +6
Model merging dramatically reduces storage and computational resources by combining multiple expert models into a single multi-task model. Although recent model merging methods hav…
Look Within or Look Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning
Yongkang Liu, Xingle Xu, Ercong Nie +5
Parameter-Efficient Fine-Tuning (PEFT) methods achieve performance comparable to Full Fine-Tuning (FFT) while requiring significantly fewer computing resources, making it the go-to…