11 papers
Beyond Over-Refusal: Defending Indirect Prompt Injection via Latent Instruction Manifolds
Jiahao Chen, Rui Yin, Xinfeng Li +6
Large Language Models (LLMs) have been integrated into complex ecosystems (e.g., Code Agents), while Indirect Prompt Injection (IPI) attacks have emerged as critical barriers to th…
The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrails
Shuo Shi, Rui Yin, Naen Xu +7
Multimodal guard models have emerged as critical safety components for screening content in vision-language systems. While adversarial research has extensively studied jailbreaking…
PhotoBench: Beyond Visual Matching Towards Personalized Intent-Driven Photo Retrieval
Tianyi Xu, Rong Shan, Junjie Wu +11
Personal photo albums are not merely collections of static images but living, ecological archives defined by temporal continuity, social entanglement, and rich metadata, which make…
Adaptive Milestone Reward for GUI Agents
Congmin Zheng, Xiaoyun Mo, Xinbei Ma +10
Reinforcement Learning (RL) has emerged as a mainstream paradigm for training Mobile GUI Agents, yet it struggles with the temporal credit assignment problem inherent in long-horiz…
DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories
Chenlong Deng, Mengjie Deng, Junjie Wu +10
Existing multimodal retrieval systems excel at semantic matching but implicitly assume that query-image relevance can be measured in isolation. This paradigm overlooks the rich dep…
OSCAR: Optimization-Steered Agentic Planning for Composed Image Retrieval
Teng Wang, Rong Shan, Jianghao Lin +8
Composed image retrieval (CIR) requires complex reasoning over heterogeneous visual and textual constraints. Existing approaches largely fall into two paradigms: unified embedding…
FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks
Naen Xu, Jinghuai Zhang, Ping He +6
Large language models (LLMs) have been widely integrated into critical automated workflows, including contract review and job application processes. However, LLMs are susceptible t…
SpecTran: Spectral-Aware Transformer-based Adapter for LLM-Enhanced Sequential Recommendation
Yu Cui, Feng Liu, Zhaoxiang Wang +4
Traditional sequential recommendation (SR) models learn low-dimensional item ID embeddings from user-item interactions, often overlooking textual information such as item titles or…
ColorBench: Benchmarking Mobile Agents with Graph-Structured Framework for Complex Long-Horizon Tasks
Yuanyi Song, Heyuan Huang, Qiqiang Lin +9
The rapid advancement of multimodal large language models has enabled agents to operate mobile devices by directly interacting with graphical user interfaces, opening new possibili…
MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlook
Peng Xu, Shengwu Xiong, Jiajun Zhang +125
This paper reviews the MARS2 2025 Challenge on Multimodal Reasoning. We aim to bring together different approaches in multimodal machine learning and LLMs via a large benchmark. We…
VeriOS: Query-Driven Proactive Human-Agent-GUI Interaction for Trustworthy OS Agents
Zheng Wu, Heyuan Huang, Xingyu Lou +8
With the rapid progress of multimodal large language models, operating system (OS) agents become increasingly capable of automating tasks through on-device graphical user interface…