8 papers
ORCA: Hunting Compositional Failures in Text-to-Image Diffusion
Arshia Hemmat, Amirhossein Vahidi, Amitis Shidani +4
Text-to-image diffusion models fail predictably on compositional prompts: attributes bind to the wrong objects, spatial relations invert, and multi-object scenes lose count. Recent…
TRACER: Persistent Regularization for Robust Multimodal Finetuning
Hesam Asadollahzadeh, Feng Liu, Christopher Leckie +1
Mainstream strategies for finetuning pretrained multimodal models often degrade out-of-distribution (OOD) robustness, a phenomenon known as catastrophic forgetting. In this paper,…
Softmax is not Enough (for Adaptive Conformal Classification)
Navid Akhavan Attar, Hesam Asadollahzadeh, Ling Luo +1
The merit of Conformal Prediction (CP), as a distribution-free framework for uncertainty quantification, depends on generating prediction sets that are efficient, reflected in smal…
Shortest-Path Flow Matching with Mixture-Conditioned Bases for OOD Generalization to Unseen Conditions
Andrea Rubbi, Amir Akbarnejad, Mohammad Vali Sanian +10
Robust generalization under distribution shift remains a key challenge for conditional generative modeling: conditional flow-based methods often fit the training conditions well bu…
DirMoE: Dirichlet-routed Mixture of Experts
Amirhossein Vahidi, Hesam Asadollahzadeh, Navid Akhavan Attar +4
Mixture-of-Experts (MoE) models have demonstrated exceptional performance in large-scale language models. Existing routers typically rely on non-differentiable Top-+Softmax, lim…
GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs
Aryan Yazdan Parast, Parsa Hosseini, Hesam Asadollahzadeh +4
Object hallucination in Multimodal Large Language Models (MLLMs) is a persistent failure mode that causes the model to perceive objects absent in the image. This weakness of MLLMs…
Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation
Mahdi Ghaznavi, Hesam Asadollahzadeh, Fahimeh Hosseini Noohdani +5
Classifiers trained with Empirical Risk Minimization (ERM) tend to rely on attributes that have high spurious correlation with the target. This can degrade the performance on under…
Annotation-Free Group Robustness via Loss-Based Resampling
Mahdi Ghaznavi, Hesam Asadollahzadeh, HamidReza Yaghoubi Araghi +3
It is well-known that training neural networks for image classification with empirical risk minimization (ERM) makes them vulnerable to relying on spurious attributes instead of ca…