69 citations · 235 across the 60 of their papers we have counts for
65 papers
Learning to Follow In-Context Watermark Instructions via Self-Distillation
Yepeng Liu, Tianyi Chen, Xuandong Zhao +2
In-context watermarking (ICW) prepends an instruction to a query asking the model to embed a statistically detectable signal in its response. It thus equips LLMs with a watermarkin…
MatrAIx: Simulating the World with 8.3 Billion Persona Agents
Xiaomin Li, Yuexing Hao, Jianheng Hou +90
Human evaluation of AI systems and digital products is costly, slow, and difficult to scale. Offline evaluations are more scalable but often abstract away human diversity and inter…
When Do Intrinsic Rewards Work for Code Reasoning? A Comprehensive Study
Xiaolong Jin, Xuandong Zhao, Wenbo Guo +2
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in large language model reasoning, but relies on ground-truth supervision that is costly or in…
VIMPO: Value-Implicit Policy Optimization for LLMs
Zhewei Kang, Aosong Feng, Sergey Levine +2
Reinforcement learning with verifiable rewards has become a central tool for improving the reasoning ability of large language models, but current methods face a trade-off between…
Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds
Ömer Veysel Çağatan, Xuandong Zhao
Reward hacking, where AI systems exploit misspecified objectives to achieve high reward without satisfying intended goals, remains a central challenge in AI safety. Yet most known…
Audio Pirates: Black-box Audio Watermark Removal via Diffusion Priors
Lingfeng Yao, Xincong Zhong, Chenpei Huang +6
With the rise of AI-generated audio, watermarking has become widely used for detecting misuse and protecting intellectual property. However, adversaries may try to remove these wat…
MemFail: Stress-Testing Failure Modes of LLM Memory Systems
Ishir Garg, Neel Kolhe, Dawn Song +1
Large language model (LLM) agents increasingly rely on external memory systems to remain consistent across long-horizon interactions, but little empirical work has been done to und…
Auditing Agent Harness Safety
Chengzhi Liu, Yichen Guo, Yepeng Liu +8
LLM agents increasingly run inside execution harnesses that dispatch tools, allocate resources, and route messages between specialized components. However, a harness can return a c…
DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
Haoyu Hu, Xuandong Zhao, Xuhai "Orson'' Xu +1
Reinforcement learning with verifiable rewards (RLVR) generates hundreds of thousands of tokens per training step, with rollout generation dominating the computational cost. The ov…
Self-Sovereign Agent
Wenjie Qu, Xuandong Zhao, Jiaheng Zhang +1
We investigate the emerging prospect of self-sovereign agents -- AI systems that can economically sustain and extend their own operation without human involvement. Recent advances…
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
Xiangyi Li, Yimin Liu, Wenbo Chen +75
Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to m…
Making Bias Non-Predictive: Training Robust LLM Reasoning via Reinforcement Learning
Qian Wang, Xuandong Zhao, Zirui Zhang +4
Large language models (LLMs) increasingly serve as reasoners and automated evaluators, yet they remain susceptible to cognitive biases -- often altering their reasoning when faced…
Clipping-Free Policy Optimization for Large Language Models
Ömer Veysel Çağatan, Barış Akgün, Gözde Gül Şahin +1
Reinforcement learning has become central to post-training large language models, yet dominant algorithms rely on clipping mechanisms that introduce optimization issues at scale, i…
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini +82
AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not…
InfoSynth: Information-Guided Benchmark Synthesis for LLMs
Ishir Garg, Neel Kolhe, Xuandong Zhao +1
Large language models (LLMs) have demonstrated significant advancements in reasoning and code generation, but efficiently creating new benchmarks to evaluate these capabilities rem…
Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption
Yepeng Liu, Xuandong Zhao, Dawn Song +2
Despite progress in watermarking algorithms for large language models (LLMs), real-world deployment remains limited. We argue that this gap stems from misaligned incentives among L…
PromptArmor: Simple yet Effective Prompt Injection Defenses
Tianneng Shi, Kaijie Zhu, Zhun Wang +13
Despite their potential, recent research has demonstrated that LLM agents are vulnerable to prompt injection attacks, where malicious prompts are injected into the agent's input, c…
Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models
Kaiqu Liang, Haimin Hu, Xuandong Zhao +3
Bullshit, as conceptualized by philosopher Harry Frankfurt, refers to statements made without regard to their truth value. While previous work has explored large language model (LL…
The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
Alexander Xiong, Xuandong Zhao, Aneesh Pappu +1
Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet they also exhibit memorization of their training data. This phenomenon rais…
AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents
Zhun Wang, Vincent Siu, Zhe Ye +6
The strong planning and reasoning capabilities of Large Language Models (LLMs) have fostered the development of agent-based systems capable of leveraging external tools and interac…
AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents
Jingxu Xie, Dylan Xu, Xuandong Zhao +1
We introduce AgentSynth, a scalable and cost-efficient pipeline for automatically synthesizing high-quality tasks and trajectory datasets for generalist computer-use agents. Levera…
Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services
Guoheng Sun, Ziyao Wang, Xuandong Zhao +5
Modern large language model (LLM) services increasingly rely on complex, often abstract operations, such as multi-step reasoning and multi-agent collaboration, to generate high-qua…
OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models
Ziheng Cheng, Yixiao Huang, Hui Xu +4
Text-to-Image (T2I) models have achieved remarkable success in generating visual content from text inputs. Although multiple safety alignment strategies have been proposed to preve…
Learning to Reason without External Rewards
Xuandong Zhao, Zhewei Kang, Aosong Feng +2
Training large language models (LLMs) for complex reasoning via Reinforcement Learning with Verifiable Rewards (RLVR) is effective but limited by reliance on costly, domain-specifi…