5 papers
Terminal-Bench-LILT: Multilingual Agentic Coding Benchmark Grounded in Language, Region, and Culture
Yunsu Kim, Kaden Uhlig, Ashwin Purohit +10
Most evaluations for coding agents are conducted exclusively in English, which does not reflect real-world multilingual deployment. We present Terminal-Bench-LILT, a suite of 300 a…
AgenTRIM: Tool Risk Mitigation for Agentic AI
Roy Betser, Amit Giloni, Shamik Bose +4
AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool permissions introduce security risk…
Counterfactual-based Agent Influence Ranker for Agentic AI Workflows
Amit Giloni, Chiara Picardi, Roy Betser +3
An Agentic AI Workflow (AAW), also known as an LLM-based multi-agent system, is an autonomous system that assembles several LLM-based agents to work collaboratively towards a share…
LumiMAS: A Comprehensive Framework for Real-Time Monitoring and Enhanced Observability in Multi-Agent Systems
Ron Solomon, Yarin Yerushalmi Levi, Lior Vaknin +8
The incorporation of LLMs in multi-agent systems (MASs) has the potential to significantly improve our ability to autonomously solve complex problems. However, such systems introdu…
MAPS: A Multilingual Benchmark for Agent Performance and Security
Omer Hofman, Jonathan Brokman, Oren Rachmil +7
Agentic AI systems, which build on Large Language Models (LLMs) and interact with tools and memory, have rapidly advanced in capability and scope. Yet, since LLMs have been shown t…