6 papers
CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs
Yuxuan Liu, Yuntian Shi, Kun Wang +2
Multimodal large language models (MLLMs) enable interaction over both text and images, but their safety behavior can be driven by unimodal shortcuts instead of true joint intent un…
Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem
Tianming Qiu, Christos Sonis, Hao Shen
Weight Space Learning (WSL), which frames neural network weights as a data modality, is an emerging field with potential for tasks like meta-learning or transfer learning. Particul…
Concept-Guided Backdoor Attack on Vision Language Models
Haoyu Shen, Weimin Lyu, Haotian Xu +1
Vision-Language Models (VLMs) have achieved impressive progress in multimodal text generation, yet their rapid adoption raises increasing concerns about security vulnerabilities. E…
An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation
Uzair Akbar, Niki Kilbertus, Hao Shen +2
The technique of data augmentation (DA) is often used in machine learning for regularization purposes to better generalize under i.i.d. settings. In this work, we present a unifyin…
Resource-Efficient Adaptation of Large Language Models for Text Embeddings via Prompt Engineering and Contrastive Fine-tuning
Benedikt Roth, Stephan Rappensperger, Tianming Qiu +3
Large Language Models (LLMs) have become a cornerstone in Natural Language Processing (NLP), achieving impressive performance in text generation. Their token-level representations…
FreeBlend: Advancing Concept Blending with Staged Feedback-Driven Interpolation Diffusion
Yufan Zhou, Haoyu Shen, Huan Wang
Concept blending is a promising yet underexplored area in generative models. While recent approaches, such as embedding mixing and latent modification based on structural sketches,…