32 papers
A functional tensor model for dynamic multilayer networks with common invariant subspaces and the RKHS estimation
Runshi Tang, Runbing Zheng, Anru R. Zhang +1
Dynamic multilayer networks are frequently used to describe the structure and temporal evolution of multiple relationships among common entities, with applications in fields such a…
Concentration bounds on response-based vector embeddings of black-box generative models
Aranyak Acharyya, Joshua Agterberg, Youngser Park +1
Generative models, such as large language models or text-to-image diffusion models, can generate relevant responses to user-given queries. Response-based vector embeddings of gener…
Query-efficient model evaluation using cached responses
Hayden Helm, Ben Johnson, Carey Priebe
Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment. For modern evaluation frameworks, generating and evaluating a respo…
When prompt perturbations break your A/B test: A valid statistical test for generative surveying
Hayden Helm, Carey Priebe
Generative surveying -- where collections of LLM-based personas provide feedback on messages -- has emerged as a cheap and scalable alternative to traditional market research. Howe…
Control Charts for Multi-agent Systems
Hayden Helm, Carey Priebe, Brandon Duderstadt
Generative agents have proven to be powerful assistants in a wide variety of contexts. Given this success, users are now deploying agents with minimal restrictions in open ended, m…
Black-box model classification under the discriminative factorization
Hayden Helm, Merrick Ohata, Carey Priebe
Access to modern generative systems is often restricted to querying an API (the ``black-box" setting) and many properties of the system are unknown to the user at inference time. W…