6 papers
Membership Inference for Contrastive Pre-training Models with Text-only PII Queries
Ruoxi Cheng, Yizhong Ding, Jian Zhao +5
Contrastive pretraining models such as CLIP and CLAP, serve as the ubiquitous perceptual backbones for modern multimodal large models, yet their reliance on web-scale data raises g…
Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
Ruoxi Cheng, Haoxuan Ma, Weixin Wang +7
Alignment is vital for safely deploying large language models (LLMs). Existing techniques are either reward-based (training a reward model on preference pairs and optimizing with r…
PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximization
Ruoxi Cheng, Yizhong Ding, Shuirong Cao +6
Understanding the vulnerabilities of Large Vision Language Models (LVLMs) to jailbreak attacks is essential for their responsible real-world deployment. Most previous work requires…
DMRL: Data- and Model-aware Reward Learning for Data Extraction
Zhiqiang Wang, Ruoxi Cheng
Large language models (LLMs) are inherently vulnerable to unintended privacy breaches. Consequently, systematic red-teaming research is essential for developing robust defense mech…
TUNI: A Textual Unimodal Detector for Identity Inference in CLIP Models
Songze Li, Ruoxi Cheng, Xiaojun Jia
The widespread usage of large-scale multimodal models like CLIP has heightened concerns about the leakage of PII. Existing methods for identity inference in CLIP models require que…
Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining
Ruoxi Cheng, Yizhong Ding, Shuirong Cao +2
Audio can disclose PII, particularly when combined with related text data. Therefore, it is essential to develop tools to detect privacy leakage in Contrastive Language-Audio Pretr…