4 papers
An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios
Hankyul Baek, Jaewon Noh, Sang Seo +5
AI agents are increasingly being adopted in enterprise and personal settings with access to emails, databases, documents, and other tools where they can read, update, and dissemina…
LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories
Krishnapriya Vishnubhotla, Sowmya Vajjala, Akriti Vij +1
We evaluate the consistency of automated judges in conducting a multi-dimensional safety evaluation in a reference-free setup. Our results indicate that Large Language Models are u…
Improving Methodologies for LLM Evaluations Across Global Languages
Akriti Vij, Benjamin Chua, Darshini Ramiah +43
As frontier AI models are deployed globally, it is essential that their behaviour remains safe and reliable across diverse linguistic and cultural contexts. To examine how current…
Improving Methodologies for Agentic Evaluations Across Domains: Leakage of Sensitive Information, Fraud and Cybersecurity Threats
Ee Wei Seah, Yongsen Zheng, Naga Nikshith +67
The rapid rise of autonomous AI systems and advancements in agent capabilities are introducing new risks due to reduced oversight of real-world interactions. Yet agent testing rema…