10 papers
T-VSS: Test-Time Visual Subspace Steering for Adversarial Robustness of Vision-Language Models
Jaehyuk Jang, Minseok Seo, Minseok Seo. Seungju Cho +3
Vision-language models (VLMs) achieve strong zero-shot recognition, but they remain highly vulnerable to adversarial perturbations. Recent test-time adaptations improve robustness…
Who Wins the Conflict? Mechanistic Interpretability of Text Bias in Audio LLMs
Hyebin Cho, Suho Yoo, Jaehyuk Jang +2
While Audio Large Language Models (Audio LLMs) excel at multimodal understanding, they suffer from text dominance, a bias where models blindly favor text over acoustic evidence, ca…
Constraining to Generalize: Subspace Tuning for Few-shot Generalization of Audio-Language Models
Jaehyuk Jang, Kangwook Ko, Wonjun Lee +1
Few-shot adaptation of pretrained Audio--Language Models (ALMs) often improves seen-class performance at the cost of unseen-class generalization, leading to the base-to-new trade-o…
Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models
Hyebin Cho, Jaehyuk Jang, Changick Kim +1
Audio-Language Models (ALMs) have shown remarkable success in zero-shot audio classification by aligning audio waveforms with text. Recent efforts to improve downstream performance…
Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models
Seokil Ham, Jaehyuk Jang, Wonjun Lee +1
Fine-tuning-as-a-Service (FaaS) enables personalization of large language models (LLMs), but it can weaken safety-alignment under harmful fine-tuning attacks. Recent work has shown…
Generalizable Prompt Tuning for Audio-Language Models via Semantic Expansion
Jaehyuk Jang, Wonjun Lee, Kangwook Ko +1
Prompt tuning has achieved remarkable progress in vision-language models (VLMs) and is recently being adopted for audio-language models (ALMs). However, its generalization ability…