From the 1 of 17 linked papers with an AI index.
4 papers · 1 filter
Query-efficient model evaluation using cached responses
Hayden Helm, Ben Johnson, Carey Priebe
Evaluating a new model on an existing benchmark is often necessary to understand its behavior before deployment. For modern evaluation frameworks, generating and evaluating a respo…
Black-box model classification under the discriminative factorization
Hayden Helm, Merrick Ohata, Carey Priebe
Access to modern generative systems is often restricted to querying an API (the ``black-box" setting) and many properties of the system are unknown to the user at inference time. W…
Statistical inference on black-box generative models in the data kernel perspective space
Hayden Helm, Aranyak Acharyya, Brandon Duderstadt +2
Generative models are capable of producing human-expert level content across a variety of topics and domains. As the impact of generative models grows, it is necessary to develop s…
Consistent estimation of generative model representations in the data kernel perspective space
Aranyak Acharyya, Michael W. Trosset, Carey E. Priebe +1
Generative models, such as large language models and text-to-image diffusion models, produce relevant information when presented a query. Different models may produce different inf…