2 citations · 5 across the 9 of their papers we have counts for
4 papers · 1 filter
Articulation Upper Bounds for Language Models
Alexander Spangher, Daniel E. Ho, Diyi Yang +2
Language captures only a subset of the perceptions, intuitions and intents that form meaning between speakers. What, then, are the limits of language? We take steps towards measuri…
Verifiable, Articulable, and Tacit Components of Preference
Alexander Spangher, Sheldon S. Huang, Sheldon Huang +5
What makes a short story gripping; a news article newsworthy; or a math proof elegant? These constructs resist articulation or verification; their meaning is at least partially tac…
API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces
Jennifer Wang, Joachim Baumann, Daniel E. Ho +1
Benchmark scores are a central currency in model releases: they inform purchasing decisions, shape public trust, and influence policy. Yet, a key assumption underlying benchmark sc…
Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing
Justin W. Lin, Eliot Krzysztof Jones, Donovan Julian Jasper +10
We present the first comprehensive evaluation of AI agents against human cybersecurity professionals in a live enterprise environment. We evaluate ten cybersecurity professionals a…