Improving MLLM Training Efficiency via Stage-Aware Sparsity

Kean Shi, Liang Chen, Haozhe Zhao +1

Multimodal Large Language Models (MLLMs) have demonstrated outstanding performance across a variety of domains. However, training MLLMs is often inefficient, as much of the computa…