6 papers
Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Mode
Zimo Ji, Zongjie Li, Wenyuan Jiang +2
Claude Code's auto mode is the first deployed permission system for AI coding agents, using a two-stage transcript classifier to gate dangerous tool calls. Anthropic reports a 0.4%…
Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework
Zimo Ji, Daoyuan Wu, Wenyuan Jiang +5
Large Language Model (LLM)-based agent systems are increasingly deployed for complex real-world tasks but remain vulnerable to natural language-based attacks that exploit over-priv…
Exploring the Jupyter Ecosystem: An Empirical Study of Bugs and Vulnerabilities
Wenyuan Jiang, Diany Pressato, Harsh Darji +1
Background. Jupyter notebooks are one of the main tools used by data scientists. Notebooks include features (configuration scripts, markdown, images, etc.) that make them challengi…
INTEGRALBENCH: Benchmarking LLMs with Definite Integral Problems
Bintao Tang, Xin Yang, Yuhao Wang +3
We present INTEGRALBENCH, a focused benchmark designed to evaluate Large Language Model (LLM) performance on definite integral problems. INTEGRALBENCH provides both symbolic and nu…
Towards Provable (In)Secure Model Weight Release Schemes
Xin Yang, Bintao Tang, Yuhao Wang +3
Recent secure weight release schemes claim to enable open-source model distribution while protecting model ownership and preventing misuse. However, these approaches lack rigorous…
Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges
Zimo Ji, Daoyuan Wu, Wenyuan Jiang +3
Capture-the-Flag (CTF) competitions are crucial for cybersecurity education and training. As large language models (LLMs) evolve, there is increasing interest in their ability to a…