15 papers
AISPA: User-Centric System Prompt Auditing for Large Language Model Applications
Xiangning Lin, Shenzhe Zhu, Shu Yang +23
The paper presents AISPA, a user‑centric framework for auditing the system prompts that guide large language model behavior in commercial AI products, and reports findings from ana…
Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI
Xuanqiang Angelo Huang, Charlie Tharas, Samuele Marro +4
Ensuring that AI agents behave safely and beneficially when interacting with other parties has emerged as one of the central challenges of modern AI safety. While mechanism design,…
Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records
Fabian Degen, Oishi Deb, Jindong Gu +4
Planning records define restrictions over geographic areas, but their source documents often provide only indirect spatial evidence rather than machine-readable boundaries. We intr…
Benchmarking at the Edge of Comprehension
Samuele Marro, Jialin Yu, Emanuele La Malfa +8
As frontier Large Language Models (LLMs) increasingly saturate new benchmarks shortly after they are published, benchmarking itself is at a juncture: if frontier models keep improv…
LLM Agents Are the Antidote to Walled Gardens
Samuele Marro, Philip Torr
While the Internet's core infrastructure was designed to be open and universal, today's application layer is dominated by closed, proprietary platforms. Open and interoperable APIs…
LLM Wardens: Mitigating Adversarial Persuasion with Third-Party Conversational Oversight
Lennart Wachowiak, Scott D. Blain, David Williams-King +1
LLMs are increasingly capable of persuasion, which raises the question of how to protect users against manipulation. In a preregistered user study (N=120) across four decision-maki…