Balanced Multimodal Learning via On-the-fly Gradient Modulation

Xiaokang Peng, Yake Wei, Andong Deng +2

Multimodal learning helps to comprehensively understand the world, by integrating different senses. Accordingly, multiple input modalities are expected to boost model performance,…