Improving Multimodal Learning Balance and Sufficiency through Data Remixing

Xiaoyu Ma, Hao Chen, Yongjian Deng

Different modalities hold considerable gaps in optimization trajectories, including speeds and paths, which lead to modality laziness and modality clash when jointly training multi…