78 citations · 313 across the 17 of their papers we have counts for
5 papers · 1 filter
Multi-modal Alignment using Representation Codebook
Jiali Duan, Liqun Chen, Son Tran +4
Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusi…
Vision-Language Pre-Training with Triple Contrastive Learning
Jinyu Yang, Jiali Duan, Son Tran +6
Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attrib…
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models
Dong Wang, Yuewei Yang, Chenyang Tao +5
Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of…
Weakly supervised cross-domain alignment with optimal transport
Siyang Yuan, Ke Bai, Liqun Chen +6
Cross-domain alignment between image objects and text sequences is key to many visual-language tasks, and it poses a fundamental challenge to both computer vision and natural langu…
LMVP: Video Predictor with Leaked Motion Information
Dong Wang, Yitong Li, Wei Cao +3
We propose a Leaked Motion Video Predictor (LMVP) to predict future frames by capturing the spatial and temporal dependencies from given inputs. The motion is modeled by a newly pr…