6 papers · 1 filter
SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models
Arijit Ray, Jiafei Duan, Ellis Brown +9
Reasoning about motion and space is a fundamental cognitive capability that is required by multiple real-world applications. While many studies highlight that large multimodal lang…
Scaling Up Temporal Domain Generalization via Temporal Experts Averaging
Aoming Liu, Kevin Miller, Venkatesh Saligrama +4
Temporal Domain Generalization (TDG) aims to generalize across temporal distribution shifts, e.g., lexical change over time. Prior work often addresses this by predicting future mo…
Web Artifact Attacks Disrupt Vision Language Models
Maan Qraitem, Piotr Teterwak, Kate Saenko +1
Vision-language models (VLMs) (e.g. CLIP, LLaVA) are trained on large-scale, lightly curated web datasets, leading them to learn unintended correlations between semantic concepts a…
Is Large-Scale Pretraining the Secret to Good Domain Generalization?
Piotr Teterwak, Kuniaki Saito, Theodoros Tsiligkaridis +2
Multi-Source Domain Generalization (DG) is the task of training on multiple source domains and achieving high classification performance on unseen target domains. Recent methods co…
RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks
Nazia Tasnim, Bryan A. Plummer
Incremental learning aims to adapt to new sets of categories over time with minimal computational overhead. Prior work often addresses this task by training efficient task-specific…
Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks
Maan Qraitem, Nazia Tasnim, Piotr Teterwak +2
Typographic attacks, adding misleading text to images, can deceive vision-language models (LVLMs). The susceptibility of recent large LVLMs like GPT4-V to such attacks is understud…