8 papers
Asynchrony Is Nearly Free for Evolution Strategies on Long-Horizon LLM Agentic Tasks
William Hoy, Jingxuan Fan, Nurcin Celik +1
Synchronous policy updates can limit the efficiency of post-training for long-horizon LLM agents. When trajectories span multiple interaction turns, completion times can vary subst…
READY or Not: Reliable Enterprise Agent Deployment
Veronica Chatrath, Bryan Zhu, Jingxuan Fan +15
An AI agent can perform well on benchmarks and still be unsuitable for deployment. Existing AI-agent benchmarks measure whether an agent can complete realistic professional work, w…
CliniCARE-Bench: Clinical Calibrated Audit of Medical Reasoning in EHR
Veronica Chatrath, Bryan Zhu, George Pu +16
Large language models perform strongly on medical knowledge benchmarks, but reliable clinical deployment requires agents to conduct defensible investigations over heterogeneous, lo…
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +378
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
Next-token Preference: Scaling Unsupervised Reward Modeling
Jingxuan Fan, Yueying Li, Zhenting Qi +4
Learning from feedback advances the capabilities and safety of frontier models, but preference supervision remains costly and difficult to scale. We introduce next-token preference…
Circuit Mechanisms for Spatial Relation Generation in Diffusion Transformers
Binxu Wang, Jingxuan Fan, Xu Pan
Diffusion Transformers (DiTs) have greatly advanced text-to-image generation, but models still struggle to generate the correct spatial relations between objects as specified in th…
Demasking Fine-Tuning for Knowledge Injection in Autoregressive and Diffusion LLMs
Xu Pan, Ely Hahami, Jingxuan Fan +2
Large language models (LLMs) are often used in environments where facts evolve, yet factual knowledge updates via fine-tuning on unstructured plain text often suffer from 1) relian…
User-Assistant Bias in LLMs
Xu Pan, Jingxuan Fan, Zidi Xiong +3
Modern large language models (LLMs) are typically trained and deployed using structured role tags (e.g. system, user, assistant, tool) that explicitly mark the source of each piece…