6 citations · 37 across the 19 of their papers we have counts for
20 papers
Robustness of Vision Foundation Models to Common Perturbations
Hongbin Liu, Zhengyuan Jiang, Cheng Hong +1
A vision foundation model outputs an embedding vector for an image, which can be affected by common editing operations (e.g., JPEG compression, brightness, contrast adjustments). T…
Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection
Zedian Shao, Hongbin Liu, Yuepeng Hu +1
Multi-modal large language models (MLLMs) have emerged as powerful tools for analyzing Internet-scale image data, offering significant benefits but also raising critical safety and…
Automatically Generating Visual Hallucination Test Cases for Multimodal Large Language Models
Zhongye Liu, Hongbin Liu, Yuepeng Hu +2
Visual hallucination (VH) occurs when a multimodal large language model (MLLM) generates responses with incorrect visual details for prompts. Existing methods for generating VH tes…
Enhancing Prompt Injection Attacks to LLMs via Poisoning Alignment
Zedian Shao, Hongbin Liu, Jaden Mu +1
Prompt injection attack, where an attacker injects a prompt into the original one, aiming to make an Large Language Model (LLM) follow the injected prompt to perform an attacker-ch…
Refusing Safe Prompts for Multi-modal Large Language Models
Zedian Shao, Hongbin Liu, Yuepeng Hu +1
Multimodal large language models (MLLMs) have become the cornerstone of today's generative AI ecosystem, sparking intense competition among tech giants and startups. In particular,…
Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learning
Yuqi Jia, Minghong Fang, Hongbin Liu +2
Poisoning attacks compromise the training phase of federated learning (FL) such that the learned global model misclassifies attacker-chosen inputs called target inputs. Existing de…
AudioMarkBench: Benchmarking Robustness of Audio Watermarking
Hongbin Liu, Moyang Guo, Zhengyuan Jiang +2
The increasing realism of synthetic speech, driven by advancements in text-to-speech models, raises ethical concerns regarding impersonation and disinformation. Audio watermarking…
Mudjacking: Patching Backdoor Vulnerabilities in Foundation Models
Hongbin Liu, Michael K. Reiter, Neil Zhenqiang Gong
Foundation model has become the backbone of the AI ecosystem. In particular, a foundation model can be used as a general-purpose feature extractor to build various downstream class…
Visual Hallucinations of Multi-modal Large Language Models
Wen Huang, Hongbin Liu, Minxin Guo +1
Visual hallucination (VH) means that a multi-modal LLM (MLLM) imagines incorrect details about an image in visual question answering. Existing studies find VH instances only in exi…
PointCert: Point Cloud Classification with Deterministic Certified Robustness Guarantees
Jinghuai Zhang, Jinyuan Jia, Hongbin Liu +1
Point cloud classification is an essential component in many security-critical applications such as autonomous driving and augmented reality. However, point cloud classifiers are v…
Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning
Hongbin Liu, Wenjie Qu, Jinyuan Jia +1
Classifiers in supervised learning have various security and privacy issues, e.g., 1) data poisoning attacks, backdoor attacks, and adversarial examples on the security side as wel…
CorruptEncoder: Data Poisoning based Backdoor Attacks to Contrastive Learning
Jinghuai Zhang, Hongbin Liu, Jinyuan Jia +1
Contrastive learning (CL) pre-trains general-purpose encoders using an unlabeled pre-training dataset, which consists of images or image-text pairs. CL is vulnerable to data poison…
Semi-Leak: Membership Inference Attacks Against Semi-supervised Learning
Xinlei He, Hongbin Liu, Neil Zhenqiang Gong +1
Semi-supervised learning (SSL) leverages both labeled and unlabeled data to train machine learning (ML) models. State-of-the-art SSL methods can achieve comparable performance to s…
PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning
Hongbin Liu, Jinyuan Jia, Neil Zhenqiang Gong
Contrastive learning pre-trains an image encoder using a large amount of unlabeled data such that the image encoder can be used as a general-purpose feature extractor for various d…
StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learning
Yupei Liu, Jinyuan Jia, Hongbin Liu +1
Pre-trained encoders are general-purpose feature extractors that can be used for many downstream tasks. Recent progress in self-supervised learning can pre-train highly effective e…
10 Security and Privacy Problems in Large Foundation Models
Jinyuan Jia, Hongbin Liu, Neil Zhenqiang Gong
Foundation models--such as GPT, CLIP, and DINO--have achieved revolutionary progress in the past several years and are commonly believed to be a promising approach for general-purp…
EncoderMI: Membership Inference against Pre-trained Encoders in Contrastive Learning
Hongbin Liu, Jinyuan Jia, Wenjie Qu +1
Given a set of unlabeled images or (image, text) pairs, contrastive learning aims to pre-train an image encoder that can be used as a feature extractor for many downstream tasks. I…
PointGuard: Provably Robust 3D Point Cloud Classification
Hongbin Liu, Jinyuan Jia, Neil Zhenqiang Gong
3D point cloud classification has many safety-critical applications such as autonomous driving and robotic grasping. However, several studies showed that it is vulnerable to advers…
Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
Jinyuan Jia, Binghui Wang, Xiaoyu Cao +2
Top-k predictions are used in many real-world applications such as machine learning as a service, recommender systems, and web searches. -norm adversarial perturbation char…
On the Intrinsic Differential Privacy of Bagging
Hongbin Liu, Jinyuan Jia, Neil Zhenqiang Gong
Differentially private machine learning trains models while protecting privacy of the sensitive training data. The key to obtain differentially private models is to introduce noise…