3 citations · 5 across the 9 of their papers we have counts for
9 papers
A Jagged Frontier: Evaluating Robustness of Code Agents to Semantics-Preserving Transformations
Hasan Najib Mahmud, Shreya Gupta, Isha Chaudhary +4
AI code agents are increasingly deployed to resolve real software issues, yet their reliability under superficial code variations remains poorly understood. We evaluate whether cod…
Revealing Interpretable Failure Modes of VLMs
Isha Chaudhary, Vedaant V Jain, Kavya Sachdeva +2
Vision-Language Models (VLMs) are increasingly used in safety-critical applications because of their broad reasoning capabilities and ability to generalize with minimal task-specif…
Lumos: Let there be Language Model System Certification
Isha Chaudhary, Vedaant Jain, Prineet Parhar +4
We introduce the first principled framework, Lumos, for specifying and formally certifying Language Model System (LMS) behaviors. Lumos is an imperative probabilistic programming D…
Quantitative Certification of Agentic Tool Selection
Jehyeok Yeon, Isha Chaudhary, Gagandeep Singh
Large language models (LLMs) are increasingly deployed in agentic systems, where a fundamental task is mapping user intents to relevant external tools. Errors in tool selection can…
How Catastrophic is Your LLM? Certifying Risk in Conversation
Chengxiao Wang, Isha Chaudhary, Qian Hu +3
Large Language Models (LLMs) can produce catastrophic responses in conversational settings that pose serious risks to public safety and security. Existing evaluations often fail to…
Specification Generation for Neural Networks in Systems
Isha Chaudhary, Shuyi Lin, Cheng Tan +1
Specifications - precise mathematical representations of correct domain-specific behaviors - are crucial to guarantee the trustworthiness of computer systems. With the increasing d…
Certifying Counterfactual Bias in LLMs
Isha Chaudhary, Qian Hu, Manoj Kumar +3
Large Language Models (LLMs) can produce biased responses that can cause representational harms. However, conventional studies are insufficient to thoroughly evaluate biases across…
Certifying Knowledge Comprehension in LLMs
Isha Chaudhary, Vedaant V. Jain, Gagandeep Singh
Large Language Models (LLMs) are increasingly deployed in safety-critical systems where they provide answers based on in-context information derived from knowledge bases. As LLMs a…
Bypassing the Safety Training of Open-Source LLMs with Priming Attacks
Jason Vega, Isha Chaudhary, Changming Xu +1
With the recent surge in popularity of LLMs has come an ever-increasing need for LLM safety training. In this paper, we investigate the fragility of SOTA open-source LLMs under sim…