VCMM: Variance-Calibrated Momentum for Multimodal Learning

Zhongjing Gu, Chenyang Huang, Yufa Feng +3

Multimodal joint training often suffers from modality imbalance, where a dominant modality suppresses the optimization of others. Existing methods mainly balance modality learning…