15 papers
TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment
Changyue Li, Jiaming He, Youliang Yuan +4
Fine-Tuning-as-a-Service (FTaaS) platforms let users train large language models (LLMs) on customized tasks, but this pipeline could erode models' safety alignment. In practice, se…
Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation
Changyue Li, Jiaying Li, Youliang Yuan +3
Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics. This paper investigates a novel threat: Semantic-A…
Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs
Jen-Tse Huang, Dasen Dai, Jen-Yuan Huang +7
Humans develop perception through a bottom-up hierarchy: from basic primitives and Gestalt principles to high-level semantics. In contrast, current Multimodal Large Language Models…
Learning to Ask: When LLM Agents Meet Unclear Instruction
Wenxuan Wang, Juluan Shi, Zixuan Ling +7
Equipped with the capability to call functions, modern large language models (LLMs) can leverage external tools for addressing a range of tasks unattainable through language skills…
Identifying the Achilles' Heel: An Iterative Method for Dynamically Uncovering Factual Errors in Large Language Models
Wenxuan Wang, Yuk-Kit Chan, Zixuan Ling +7
Large Language Models (LLMs) like ChatGPT are foundational in various applications due to their extensive knowledge from pre-training and fine-tuning. Despite this, they are prone…
SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs
Sihang Zhao, Kangrui Yu, Youliang Yuan +2
Large Language Models (LLMs) have been widely explored in educational scenarios. We identify a critical vulnerability in current educational LLMs, pedagogical jailbreaks, where stu…