4 papers
A Closer Look at Multimodal Representation Collapse
Abhra Chaudhuri, Anjan Dutta, Tu Bui +1
We aim to develop a fundamental understanding of modality collapse, a recently observed empirical phenomenon wherein models trained for multimodal fusion tend to rely only on a sub…
CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitives
Nityanand Mathur, Shyam Marjit, Abhra Chaudhuri +1
With the goal of understanding the visual concepts that CLIP associates with text prompts, we show that the latent space of CLIP can be visualized solely in terms of linear transfo…
Sebra: Debiasing Through Self-Guided Bias Ranking
Adarsh Kappiyath, Abhra Chaudhuri, Ajay Jaiswal +4
Ranking samples by fine-grained estimates of spuriosity (the degree to which spurious cues are present) has recently been shown to significantly benefit bias mitigation, over the t…
DeNetDM: Debiasing by Network Depth Modulation
Silpa Vadakkeeveetil Sreelatha, Adarsh Kappiyath, Abhra Chaudhuri +1
Neural networks trained on biased datasets tend to inadvertently learn spurious correlations, hindering generalization. We formally prove that (1) samples that exhibit spurious cor…