7 citations · 10 across the 8 of their papers we have counts for
9 papers
CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking
Anjali Kantharuban, Jonas Mueller
Recent benchmarks rely on user simulators to evaluate AI agents in multi-turn interaction. While existing simulation techniques demonstrate surface fidelity to human style and beha…
StudentBench: AI and human tutoring yield equivalent GRE learning gains
Curtis Northcutt, Inaara Hasmani, Kevin Feng +3
Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We in…
Efficient Test-Time Adaptation through Human-AI Interaction
Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao +22
AI agents are trained on population-scale data to encode broad capabilities spanning those of many practitioners. Yet the artifacts they produce rarely meet the personal bar profes…
VIALS: A Benchmark for Visual Interpretation of Artifacts in the Life Sciences
Elaine Lau, Thanuka Udumulla, Lee Izhaki-Tavor +3
In professional life sciences workflows, scientists routinely interpret visual artifacts (gel blots, microscopy images, plasmid maps, flow cytometry plots, molecular structures, ..…
CAREBench: A Child-Safety Risk Benchmark for Language Models
Kaavya Krishna-Kumar, Elaine Lau, Vaughn Robinson +6
How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm? Existing child safety evaluations focus on child sexual abuse…
BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows
Elaine Lau, Markus Dücker, Ronak Chaudhary +24
Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional workflows. To evaluate frontier AI agents in a high-value, labor-intensive profe…
Real-Time Trustworthiness Scoring for LLM Structured Outputs and Data Extraction
Hui Wen Goh, Jonas Mueller
Structured Outputs from current LLMs exhibit sporadic errors, hindering enterprise AI deployment. We present CONSTRUCT, a real-time uncertainty estimator that scores the trustworth…
Automated Data Curation for Robust Language Model Fine-Tuning
Jiuhai Chen, Jonas Mueller
Large Language Models have become the de facto approach to sequence-to-sequence text generation tasks, but for specialized tasks/domains, a pretrained LLM lacks specific capabiliti…
Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness
Jiuhai Chen, Jonas Mueller
We introduce BSDetector, a method for detecting bad and speculative answers from a pretrained Large Language Model by estimating a numeric confidence score for any output it genera…