4 papers
You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model
Ziyang Luo, Zhongyao Chu, Xinjie He +4
A frozen language model on reasoning tasks has two coupled weaknesses: it under-uses evidence its own residual stream already encodes, and it fails to detect when the input is insu…
Safety, or Just Capability? A Validity Audit of Agent-Safety Benchmarks
Youting Wang, Xiao Han, Dingyan Shang +2
Agent-safety benchmarks measure different behaviors, and their scores get quoted interchangeably as an agent's safety. We treat four of them (R-Judge, InjecAgent, AgentHarm, AgentD…
Self-Commitment Latency: A Reward-Free Probe for Prompted Implicit Hacking
Bonan Shen, Youting Wang, Dingyan Shang +1
Implicit reward hacking is hard to audit when a language model's chain of thought appears benign: a final answer may be anchored by a prompt shortcut while the written reasoning st…
Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study
You Wang, Michael Pradel, Zhongxin Liu
Automated issue solving aims to resolve real-world issues in software repositories. The most popular benchmarks for automated issue solving are SWE-bench and its human-filtered sub…