4 papers
Rebalanced Multimodal Learning with Data-aware Unimodal Sampling
Qingyuan Jiang, Zhouyang Chi, Xiao Ma +3
To address the modality learning degeneration caused by modality imbalance, existing multimodal learning~(MML) approaches primarily attempt to balance the optimization process of e…
Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation
Yang Yang, Wenjuan Xi, Luping Zhou +1
Vision-language retrieval aims to search for similar instances in one modality based on queries from another modality. The primary objective is to learn cross-modal matching repres…
The Solution for Temporal Action Localisation Task of Perception Test Challenge 2024
Yinan Han, Qingyuan Jiang, Hongming Mei +2
This report presents our method for Temporal Action Localisation (TAL), which focuses on identifying and classifying actions within specific time intervals throughout a video seque…
Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworks
Yang Yang, Hongpeng Pan, Qing-Yuan Jiang +2
Multi-modal learning aims to enhance performance by unifying models from various modalities but often faces the "modality imbalance" problem in real data, leading to a bias towards…