5 papers
Reinforcement Learning from Cross-domain Videos with Video Prediction Model
Zhao Yang, Xinrui Zu, Jacob E. Kooi +5
Reinforcement learning from expert videos across visually distinct domains is challenging due to the absence of reward signals and the presence of domain gaps. We introduce XIPER (…
OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment
Tianchao Li, Shujian Yu, Xinrui Zu +4
Contrastive learning is effective for aligning paired views or modalities, but alignment beyond two modalities remains non-trivial and comparatively underexplored. Pairwise CLIP-st…
Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence
Wenzhe Yin, Zehao Xiao, Pan Zhou +4
Vision-language alignment is crucial for various downstream tasks such as cross-modal generation and retrieval. Previous multimodal approaches like CLIP utilize InfoNCE to maximize…
Towards Uniformity and Alignment for Multimodal Representation Learning
Wenzhe Yin, Pan Zhou, Zehao Xiao +4
Multimodal representation learning aims to construct a shared embedding space in which heterogeneous modalities are semantically aligned. Despite strong empirical results, InfoNCE-…
Cross-Modal Retrieval with Cauchy-Schwarz Divergence
Jiahao Zhang, Wenzhe Yin, Shujian Yu
Effective cross-modal retrieval requires robust alignment of heterogeneous data types. Most existing methods focus on bi-modal retrieval tasks and rely on distributional alignment…