5 papers
Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy
Kazem Faghih, Yize Cheng, Shoumik Saha +3
Large language models (LLMs) often achieve strong accuracy on benchmarks, yet it remains unclear how reliably they apply this knowledge when the same question is phrased in differe…
Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry
Shoumik Saha, Kazem Faghih, Soheil Feizi
Autonomous AI agents increasingly extend their capabilities through Agent Skills: modular filesystem packages whose SKILL.md files describe when and how agents should use them. Whi…
Your LLM Agents are Temporally Blind: The Misalignment Between Tool Use Decisions and Human Time Perception
Yize Cheng, Arshia Soltani Moakhar, Chenrui Fan +5
Large language model (LLM) agents are increasingly used to interact with and execute tasks in dynamic environments. However, a critical yet overlooked limitation of these agents is…
Quantifying Document Impact in RAG-LLMs
Armin Gerami, Kazem Faghih, Ramani Duraiswami
Retrieval Augmented Generation (RAG) enhances Large Language Models (LLMs) by connecting them to external knowledge, improving accuracy and reducing outdated information. However,…
Tool Preferences in Agentic LLMs are Unreliable
Kazem Faghih, Wenxiao Wang, Yize Cheng +5
Large language models (LLMs) can now access a wide range of external tools, thanks to the Model Context Protocol (MCP). This greatly expands their abilities as various agents. Howe…