21 citations · 43 across the 30 of their papers we have counts for
6 papers · 1 filter
A Persona-based Rate Action Index
Hayden Helm, Andrew Dassori
We propose an index for predicting the U.S.\ Federal Open Market Committee (FOMC) decision to hike/hold/cut the current federal funds target rate based on how a collection of perso…
Query-efficient model evaluation using cached responses
Hayden Helm, Ben Johnson, Carey Priebe
Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment. For modern evaluation frameworks, generating and evaluating a respo…
When prompt perturbations break your A/B test: A valid statistical test for generative surveying
Hayden Helm, Carey Priebe
Generative surveying -- where collections of LLM-based personas provide feedback on messages -- has emerged as a cheap and scalable alternative to traditional market research. Howe…
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models
Hayden Helm, Xiaodong Liu, Weiwei Yang
Evaluating and mitigating a generative system's susceptibility to jailbreak attacks is critical to its safe deployment. Given the number of deployable systems, full per-configurati…
Control Charts for Multi-agent Systems
Hayden Helm, Carey Priebe, Brandon Duderstadt
Generative agents have proven to be powerful assistants in a wide variety of contexts. Given this success, users are now deploying agents with minimal restrictions in open ended, m…
Black-box model classification under the discriminative factorization
Hayden Helm, Merrick Ohata, Carey Priebe
Access to modern generative systems is often restricted to querying an API (the ``black-box" setting) and many properties of the system are unknown to the user at inference time. W…