14 papers
DuplexJail: Spoken Interruption Attacks on Full-Duplex Speech Models
Jaechul Roh, Deepak Chandran, Amir Houmansadr +1
Full-duplex speech models accept user speech while generating responses, making input timing a potential safety concern. We introduce DuplexJail, which delivers fixed, request-inde…
Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs
Jaechul Roh, Amir Houmansadr, Virat Shejwalkar
Fine-tuning on benign data is known to degrade safety alignment in text and vision LLMs, but whether distinct input properties drive this vulnerability differently remains unclear.…
SPILLage: Agentic Oversharing on the Web
Jaechul Roh, Eugene Bagdasarian, Hamed Haddadi +1
LLM-powered agents are beginning to automate user's tasks across the open web, often with access to user resources such as emails and calendars. Unlike standard LLMs answering ques…
World-To-Image: Grounding Text-to-Image Generation with Agent-Driven World Knowledge
Moo Hyun Son, Jintaek Oh, Sun Bin Mun +2
While text-to-image (T2I) models can synthesize high-quality images, their performance degrades significantly when prompted with novel or out-of-distribution (OOD) entities due to…
Throttling Web Agents Using Reasoning Gates
Abhinav Kumar, Jaechul Roh, Ali Naseh +2
AI web agents use Internet resources at far greater speed, scale, and complexity -- changing how users and services interact. Deployed maliciously or erroneously, these agents coul…
Bob's Confetti: Phonetic Memorization Attacks in Music and Video Generation
Jaechul Roh, Zachary Novack, Yuefeng Peng +3
Generative AI systems for music and video commonly use text-based filters to prevent regurgitation of copyrighted material. We expose a significant vulnerability in this approach b…
R1dacted: Investigating Local Censorship in DeepSeek's R1 Language Model
Ali Naseh, Harsh Chaudhari, Jaechul Roh +3
DeepSeek recently released R1, a high-performing large language model (LLM) optimized for reasoning tasks. Despite its efficient training pipeline, R1 achieves competitive performa…
Multilingual and Multi-Accent Jailbreaking of Audio LLMs
Jaechul Roh, Virat Shejwalkar, Amir Houmansadr
Large Audio Language Models (LALMs) have significantly advanced audio understanding but introduce critical security risks, particularly through audio jailbreaks. While prior work h…
OverThink: Slowdown Attacks on Reasoning LLMs
Abhinav Kumar, Jaechul Roh, Ali Naseh +4
A reasoning language model (RLM) generates costly reasoning tokens, often hidden from the users, that help it excel at many tasks. Our Overthink attack targets RLM-based applicatio…
FameBias: Embedding Manipulation Bias Attack in Text-to-Image Models
Jaechul Roh, Andrew Yuan, Jinsong Mao
Text-to-Image (T2I) diffusion models have rapidly advanced, enabling the generation of high-quality images that align closely with textual descriptions. However, this progress has…
Backdooring Bias () into Stable Diffusion Models
Ali Naseh, Jaechul Roh, Eugene Bagdasarian +1
Recent advances in large text-conditional diffusion models have revolutionized image generation by enabling users to create realistic, high-quality images from textual prompts, sig…
OSLO: One-Shot Label-Only Membership Inference Attacks
Yuefeng Peng, Jaechul Roh, Subhransu Maji +1
We introduce One-Shot Label-Only (OSLO) membership inference attacks (MIAs), which accurately infer a given sample's membership in a target model's training set with high precision…
Understanding (Un)Intended Memorization in Text-to-Image Generative Models
Ali Naseh, Jaechul Roh, Amir Houmansadr
Multimodal machine learning, especially text-to-image models like Stable Diffusion and DALL-E 3, has gained significance for transforming text into detailed images. Despite their g…
Memory Triggers: Unveiling Memorization in Text-To-Image Generative Models through Word-Level Duplication
Ali Naseh, Jaechul Roh, Amir Houmansadr
Diffusion-based models, such as the Stable Diffusion model, have revolutionized text-to-image synthesis with their ability to produce high-quality, high-resolution images. These ad…