39 citations · 137 across the 13 of their papers we have counts for
15 papers
Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models
Juhong Min, Lazar Valkov, Vitali Petsiuk +2
Vision-language models benefit from high-resolution images, but the increase in visual-token count incurs high compute overhead. Humans resolve this tension via foveation: a coarse…
GoldiCLIP: The Goldilocks Approach for Balancing Explicit Supervision for Language-Image Pretraining
Deen Dayal Mohan, Hossein Souri, Vitali Petsiuk +4
Until recently, the success of large-scale vision-language models (VLMs) has primarily relied on billion-sample datasets, posing a significant barrier to progress. Latest works hav…
ARGENT: Adaptive Hierarchical Image-Text Representations
Chuong Huynh, Hossein Souri, Abhinav Kumar +3
Large-scale Vision-Language Models (VLMs) such as CLIP learn powerful semantic representations but operate in Euclidean space, which fails to capture the inherent hierarchical stru…
Generating Potent Poisons and Backdoors from Scratch with Guided Diffusion
Hossein Souri, Arpit Bansal, Hamid Kazemi +7
Modern neural networks are often trained on massive datasets that are web scraped with minimal human inspection. As a result of this insecure curation pipeline, an adversary can po…
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks
Micah Goldblum, Hossein Souri, Renkun Ni +10
Neural network based computer vision systems are typically built on a backbone, a pretrained or randomly initialized feature extractor. Several years ago, the default option was an…
Thinking Two Moves Ahead: Anticipating Other Users Improves Backdoor Attacks in Federated Learning
Yuxin Wen, Jonas Geiping, Liam Fowl +4
Federated learning is particularly susceptible to model poisoning and backdoor attacks because individual users have direct control over the training data and model updates. At the…
Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors
Ravid Shwartz-Ziv, Micah Goldblum, Hossein Souri +4
Deep learning is increasingly moving towards a transfer learning paradigm whereby large foundation models are fine-tuned on downstream tasks, starting from an initialization learne…
A Deep Dive into Dataset Imbalance and Bias in Face Identification
Valeriia Cherepanova, Steven Reich, Samuel Dooley +3
As the deployment of automated face recognition (FR) systems proliferates, bias in these systems is not just an academic question, but a matter of public concern. Media portrayals…
Interpolated Joint Space Adversarial Training for Robust and Generalizable Defenses
Chun Pong Lau, Jiang Liu, Hossein Souri +3
Adversarial training (AT) is considered to be one of the most reliable defenses against adversarial attacks. However, models trained with AT sacrifice standard accuracy and do not…
Mutual Adversarial Training: Learning together is better than going alone
Jiang Liu, Chun Pong Lau, Hossein Souri +2
Recent studies have shown that robustness to adversarial attacks can be transferred across networks. In other words, we can make a weak model more robust with the help of a strong…
Identification of Attack-Specific Signatures in Adversarial Examples
Hossein Souri, Pirazh Khorramshahi, Chun Pong Lau +2
The adversarial attack literature contains a myriad of algorithms for crafting perturbations which yield pathological behavior in neural networks. In many cases, multiple algorithm…
Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
Hossein Souri, Liam Fowl, Rama Chellappa +2
As the curation of data for machine learning becomes increasingly automated, dataset tampering is a mounting threat. Backdoor attackers tamper with training data to embed a vulnera…
GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue
Pirazh Khorramshahi, Hossein Souri, Rama Chellappa +1
Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples. GANs are often…
Towards Gender-Neutral Face Descriptors for Mitigating Bias in Face Recognition
Prithviraj Dhar, Joshua Gleason, Hossein Souri +2
State-of-the-art deep networks implicitly encode gender information while being trained for face recognition. Gender is often viewed as an important attribute with respect to ident…
ATFaceGAN: Single Face Image Restoration and Recognition from Atmospheric Turbulence
Chun Pong Lau, Hossein Souri, Rama Chellappa
Image degradation due to atmospheric turbulence is common while capturing images at long ranges. To mitigate the degradation due to turbulence which includes deformation and blur,…