11 papers
UNBIND: UNlearning By INference-time Directional Steering for Code LLMs
Zhengyang Shan, Jiayun Xin, Yanjun Lin +7
Code large language models acquire programming capabilities from large code corpora, but can also memorize implementations that later require removal. Code unlearning is needed to…
MicroVerse: An Instrument for Measuring Self-Authored Identity Drift in Long-Horizon Multi-Agent Language-Model Simulations
Sky Ng, Brihi Joshi, Ishan Gupta +47
Long-horizon, multi-agent language model (LM) simulations are widely proposed for studying social behavior, yet instruments to measure whether persona-conditioned agents maintain i…
PersonaEval: Persona-Based User Simulation for Evaluating Interactive Applications
Yifan Simon Liu, Qianfeng Wen, Yilan Fan +40
Real user studies are important for understanding how people interact with systems under test or already deployed. In practice, however, they are often costly, time-consuming, and…
OBLIVION: Workflow-Level Operational Skill Unlearning for Deployed Agents
Zhengyang Shan, Xu Qian, Jiayun Xin +3
Large language model agents are becoming operational interfaces to files, memories, registries, and external tools. This deployment shift creates a new skill revocation problem: af…
ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents
Udari Madhushani Sehwag, Zhengyang Shan, Heming Liu +3
Clarification-seeking behavior is widely regarded as a desirable property of LLM agents, enabling them to resolve ambiguity before acting on underspecified tasks. However, the secu…
SAGE: Signal-Amplified Guided Embeddings for LLM-based Vulnerability Detection
Zhengyang Shan, Xu Qian, Jiayun Xin +5
Software vulnerabilities are a primary threat to modern infrastructure. While static analysis and Graph Neural Networks have long served as the foundation for vulnerability detecti…
Don't Let the Claw Grip Your Hand: A Security Analysis and Defense Framework for OpenClaw
Zhengyang Shan, Jiayun Xin, Yue Zhang +1
Code agents powered by large language models can execute shell commands on behalf of users, introducing severe security vulnerabilities. This paper presents a two-phase security an…
Measuring Mechanistic Independence: Can Bias Be Removed Without Erasing Demographics?
Zhengyang Shan, Aaron Mueller
We investigate how independent demographic bias mechanisms are from general demographic recognition in language models. Using a multi-task evaluation setup where demographics are a…
A Multimodal Manufacturing Safety Chatbot: Knowledge Base Design, Benchmark Development, and Evaluation of Multiple RAG Approaches
Ryan Singh, Austin Hamilton, Amanda White +8
Ensuring worker safety remains a critical challenge in modern manufacturing environments. Industry 5.0 reorients the prevailing manufacturing paradigm toward more human-centric ope…
Gender Inclusivity Fairness Index (GIFI): A Multilevel Framework for Evaluating Gender Diversity in Large Language Models
Zhengyang Shan, Emily Ruth Diana, Jiawei Zhou
We present a comprehensive evaluation of gender fairness in large language models (LLMs), focusing on their ability to handle both binary and non-binary genders. While previous stu…
Delving into Decision-based Black-box Attacks on Semantic Segmentation
Zhaoyu Chen, Zhengyang Shan, Jingwen Chang +4
Semantic segmentation is a fundamental visual task that finds extensive deployment in applications with security-sensitive considerations. Nonetheless, recent work illustrates the…