From the 1 of 13 linked papers with an AI index.
13 papers
A Persona-based Rate Action Index
Hayden Helm, Andrew Dassori
The paper introduces a persona‑based index that predicts U.S. Federal Open Market Committee rate decisions by modeling each committee member with a retrieval‑augmented generative p…
Query-efficient model evaluation using cached responses
Hayden Helm, Ben Johnson, Carey Priebe
Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment. For modern evaluation frameworks, generating and evaluating a respo…
Detecting Perspective Shifts in Multi-agent Systems
Eric Bridgeford, Hayden Helm
Generative models augmented with external tools and update mechanisms (or \textit{agents}) have demonstrated capabilities beyond intelligent prompting of base models. As agent use…
When prompt perturbations break your A/B test: A valid statistical test for generative surveying
Hayden Helm, Carey Priebe
Generative surveying -- where collections of LLM-based personas provide feedback on messages -- has emerged as a cheap and scalable alternative to traditional market research. Howe…
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models
Hayden Helm, Xiaodong Liu, Weiwei Yang
Evaluating and mitigating a generative system's susceptibility to jailbreak attacks is critical to its safe deployment. Given the number of deployable systems, full per-configurati…
Control Charts for Multi-agent Systems
Hayden Helm, Carey Priebe, Brandon Duderstadt
Generative agents have proven to be powerful assistants in a wide variety of contexts. Given this success, users are now deploying agents with minimal restrictions in open ended, m…