21 citations · 173 across the 53 of their papers we have counts for
15 papers · 1 filter
SPOT! Revisiting Video-Language Models for Event Understanding
Gengyuan Zhang, Jinhe Bi, Jindong Gu +2
Understanding videos is an important research topic for multimodal learning. Leveraging large-scale datasets of web-crawled video-text pairs as weak supervision has become a pre-tr…
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
Xin Liu, Yichen Zhu, Jindong Gu +3
The security concerns surrounding Large Language Models (LLMs) have been extensively explored, yet the safety of Multimodal Large Language Models (MLLMs) remains understudied. In t…
Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation
Hang Li, Chengzhi Shen, Philip Torr +2
Diffusion-based models have gained significant popularity for text-to-image generation due to their exceptional image-generation capabilities. A risk with these models is the poten…
Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attacks
Jindong Gu, Fangyun Wei, Philip Torr +1
Deep Neural Networks can be easily fooled by small and imperceptible perturbations. The query-based black-box attack (QBBA) is able to create the perturbations using model output p…
Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging
Xiaojun Jia, Yuefeng Chen, Xiaofeng Mao +5
Fast Adversarial Training (FAT) not only improves the model robustness but also reduces the training cost of standard adversarial training. However, fast adversarial training often…
FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning
Haokun Chen, Yao Zhang, Denis Krompass +2
Recently, foundation models have exhibited remarkable advancements in multi-modal learning. These models, equipped with millions (or billions) of parameters, typically require a su…
Multi-event Video-Text Retrieval
Gengyuan Zhang, Jisen Ren, Jindong Gu +1
Video-Text Retrieval (VTR) is a crucial multi-modal task in an era of massive video-text data on the Internet. A plethora of work characterized by using a two-stream Vision-Languag…
Discretization-Induced Dirichlet Posterior for Robust Uncertainty Quantification on Regression
Xuanlong Yu, Gianni Franchi, Jindong Gu +1
Uncertainty quantification is critical for deploying deep neural networks (DNNs) in real-world applications. An Auxiliary Uncertainty Estimator (AuxUE) is one of the most effective…
FedPop: Federated Population-based Hyperparameter Tuning
Haokun Chen, Denis Krompass, Jindong Gu +1
Federated Learning (FL) is a distributed machine learning (ML) paradigm, in which multiple clients collaboratively train ML models without centralizing their local data. Similar to…
Reliable Evaluation of Adversarial Transferability
Wenqian Yu, Jindong Gu, Zhijiang Li +1
Adversarial examples (AEs) with small adversarial perturbations can mislead deep neural networks (DNNs) into wrong predictions. The AEs created on one DNN can also fool another DNN…
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models
Shuo Chen, Jindong Gu, Zhen Han +3
Various adaptation methods, such as LoRA, prompts, and adapters, have been proposed to enhance the performance of pre-trained vision-language models in specific domains. The robust…
Towards Robust Prompts on Vision-Language Models
Jindong Gu, Ahmad Beirami, Xuezhi Wang +3
With the advent of vision-language models (VLMs) that can perform in-context and prompt-based learning, how can we design prompting approaches that robustly generalize to distribut…
Backdoor Defense via Adaptively Splitting Poisoned Dataset
Kuofeng Gao, Yang Bai, Jindong Gu +2
Backdoor defenses have been studied to alleviate the threat of deep neural networks (DNNs) being backdoor attacked and thus maliciously altered. Since DNNs usually adopt some exter…
Influencer Backdoor Attack on Semantic Segmentation
Haoheng Lan, Jindong Gu, Philip Torr +1
When a small number of poisoned samples are injected into the training dataset of a deep neural network, the network can be induced to exhibit malicious behavior during inferences,…
Explainability and Robustness of Deep Visual Classification Models
Jindong Gu
In the computer vision community, Convolutional Neural Networks (CNNs), first proposed in the 1980's, have become the standard visual classification model. Recently, as alternative…