6 papers
BARC-Bench: Benign Asks, Risky Consequences
Kangkang Li, Kelvin Luu, Lily Ma +3
Task instructions from users for agents are inherently underspecified, especially regarding safety boundaries--creating behavioral risks. To study these risks, we introduce BARC-Be…
Maintaining Benchmarks Against Increasingly Capable Agents: Detection and Remediation of Unearned Passes
Weijun Luo, Kelvin Luu, Xinyi Liu +6
Agentic benchmarks guide model selection and training. Yet an agent can pass a task without demonstrating the intended capability. Such outcomes constitute unearned passes; their p…
Geometric -Spanners with Few Crossings
Kelvin Luu, Csaba D. Tóth
For points in the plane and , we construct a -spanner with edges in which every edge has crossin…
HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?
Tu Trinh, Mohamed Elfeki, Guangze Luo +9
Frontier coding agents solve complex tasks when given complete context but collapse when specifications are incomplete or ambiguous. The bottleneck is not raw capability, but judgm…
Time Waits for No One! Analysis and Challenges of Temporal Misalignment
Kelvin Luu, Daniel Khashabi, Suchin Gururangan +2
When an NLP model is trained on text data from one time period and tested or deployed on data from another, the resulting temporal misalignment can degrade end-task performance. In…
Explaining Relationships Between Scientific Documents
Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski +3
We address the task of explaining relationships between two scientific documents using natural language text. This task requires modeling the complex content of long technical docu…