5 papers
Multimodal Information Interactions Bottleneck for Multimodal Self-Supervised Learning
Liangjian Wen, Jiangtao Zheng, Jiang Duan +8
Multimodal self-supervised learning is challenged by heterogeneity, unequal informativeness, and complex information interactions across multimodal data sources such as images, tex…
InfMasking: A Unified Variational Framework for Synergistic Multimodal Learning
Liangjian Wen, Qun Dai, Jiang Duan +7
Multimodal models are expected to exploit synergistic information, that is, information available only when multiple modalities are observed together. In practice, however, a fusio…
MSCR: Jointly Balancing Utilization Across Modalities and Discovering Synergistic Information
Xinyu Chen, Liangjian Wen, Jiang Duan +7
In multimodal learning, modality imbalance often lets one modality dominate optimization and keeps the model from fully exploiting cross-modal interactions. Balance-oriented method…
InfMasking: Unleashing Synergistic Information by Contrastive Multimodal Interactions
Liangjian Wen, Qun Dai, Jianzhuang Liu +7
In multimodal representation learning, synergistic interactions between modalities not only provide complementary information but also create unique outcomes through specific inter…
Web-CogReasoner: Towards Multimodal Knowledge-Induced Cognitive Reasoning for Web Agents
Yuhan Guo, Cong Guo, Aiwen Sun +12
Multimodal large-scale models have significantly advanced the development of web agents, enabling perception and interaction with digital environments akin to human cognition. In t…