14 papers
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
Lipeng He, Yihan Wang, Jiawen Zhang +1
Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses…
Beyond Similarity: Trustworthy Memory Search for Personal AI Agents
Jiawen Zhang, Kejia Chen, Jiachen Ma +7
Personal AI agents increasingly rely on long-term memory to provide persistent personalization across sessions. However, existing memory pipelines are largely driven by semantic si…
REFLECTOR: Internalizing Step-wise Reflection against Indirect Jailbreak
Jiachen Ma, Jiawen Zhang, Xiangtian Li +3
While Large Language Models (LLMs) demonstrate remarkable capabilities, they remain susceptible to sophisticated, multi-step jailbreak attacks that circumvent conventional surface-…
Evaluating Cognitive Age Alignment in Interactive AI Agents
Yifan Shen, Jiawen Zhang, Jian Xu +4
While agentic AI and its core multimodal large language models (MLLMs) have demonstrated remarkable promise in language and visual reasoning across domains ranging from daily life…
Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
Kejia Chen, Jiawen Zhang, Boheng Li +6
Many-shot jailbreaking (MSJ) causes safety-aligned language models to answer harmful queries by preceding them with many harmful question-answer demonstrations. We study why this a…
Confidence-Aware Alignment Makes Reasoning LLMs More Reliable
Kejia Chen, Jiawen Zhang, Yihong Wu +5
Large reasoning models often reach correct answers through flawed intermediate steps, creating a gap between final accuracy and reasoning reliability. Existing alignment strategies…
From Pre-trained Models to Large Language Models: A Comprehensive Survey of AI-Driven Psychological Computing
Huiyao Chen, Ruimeng Liu, Yan Luo +4
The intersection of artificial intelligence and psychological science has experienced remarkable growth, with annual publications expanding from 859 papers in 2000 to 29,979 by 202…
Understanding and Preserving Safety in Fine-Tuned LLMs
Jiawen Zhang, Yangfan Hu, Kejia Chen +7
Fine-tuning is an essential and pervasive functionality for applying large language models (LLMs) to downstream tasks. However, it has the potential to substantially degrade safety…
Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
Jiawen Zhang, Lipeng He, Kejia Chen +4
Fine-tuning safety-aligned large language models (LLMs) can substantially compromise their safety. Previous approaches require many safety samples or calibration sets, which not on…
Token-Level Inference-Time Alignment for Vision-Language Models
Kejia Chen, Jiawen Zhang, Jiacong Hu +4
Vision-Language Models (VLMs) have become essential backbones of modern multimodal intelligence, yet their outputs remain prone to hallucination-plausible text misaligned with visu…
Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Kejia Chen, Jiawen Zhang, Jiacong Hu +4
Quantized large language models (LLMs) have gained increasing attention and significance for enabling deployment in resource-constrained environments. However, emerging studies on…
SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner
Kejia Chen, Jiawen Zhang, Jiacong Hu +4
Large Visual Language Models (LVLMs) increasingly rely on preference alignment to ensure reliability, which steers the model behavior via preference fine-tuning on preference data…
SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models
Jiawen Zhang, Kejia Chen, Zunlei Feng +4
With the growing popularity of LLMs among the general public users, privacy-preserving and adversarial robustness have become two pressing demands for LLM-based services, which hav…
Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defense
Jiawen Zhang, Kejia Chen, Lipeng He +7
Large Language Models (LLMs) have showcased remarkable capabilities across various domains. Accompanying the evolving capabilities and expanding deployment scenarios of LLMs, their…