4 papers
Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks
Neeraj Karamchandani, Piyush Nagasubramaniam, Xinhong Xie +2
Security benchmarks for LLM-based agents often report the attack success rate (ASR) as a measure of model robustness and use these scores to compare different models and defense me…
Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences
Boyuan Deng, Shuyi Fan, Hongyang Zhang +1
Scientific workflows often require choosing among known relations before a deterministic calculation can proceed. Whether observations share a culture, treatment or reference stand…
LLM-Based Penetration Testing in the Presence of Honeypots
Xinhong Xie, Piyush Nagasubramaniam, Neeraj Karamchandani +1
Large language model (LLM) agents are increasingly employed for offensive cybersecurity tasks such as automated vulnerability discovery, reconnaissance, and penetration testing. Th…
Difference-in-Differences on a Censored Rating Scale Can Manufacture an Effect: Evidence from a Pre-Registered LLM-Judge Audit
Shuyi Fan, Boyuan Deng, Mengyu Xu +3
LLM-judge audits assess bias by comparing ratings across matched conditions. Difference-in-differences designs take the rating difference between two candidate responses to each it…