7 citations · 20 across the 12 of their papers we have counts for
14 papers
SeekerGym: A Benchmark for Reliable Information Seeking
Remy Kim, Minseung Lee, Shuo Li +1
Despite their substantial successes, AI agents continue to face fundamental challenges in terms of trustworthiness. Consider deep research agents, tasked with searching for informa…
BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks
Sagnik Anupam, Davis Brown, Shuo Li +3
LLM web agents now browse and take actions on the open web, yet current agent evaluations are constrained to sandboxed environments or artificial tasks. We introduce BrowserArena,…
Quasar: A Programming Language Specialized for LLM Code Actions
Stephen Mell, Botong Zhang, David Mell +5
Large language models (LLMs) often call external tools to solve tasks. One effective strategy is for LLMs to write code, enabling them to use complex control flow such as condition…
Alignment of large language models with constrained learning
Botong Zhang, Shuo Li, Ignacio Hounie +3
We study the problem of computing an optimal large language model (LLM) policy for the constrained alignment problem, where the goal is to maximize a primary reward objective while…
Evaluating the Diversity and Quality of LLM Generated Content
Alexander Shypula, Shuo Li, Botong Zhang +3
Recent work suggests that preference-tuning techniques -- such as Reinforcement Learning from Human Feedback (RLHF) methods like PPO and GRPO, as well as alternatives like DPO -- r…
Conformal Structured Prediction
Botong Zhang, Shuo Li, Osbert Bastani
Conformal prediction has recently emerged as a promising strategy for quantifying the uncertainty of a predictive model; these algorithms modify the model to output sets of labels…