7 papers
MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents
Chenxu Xiong, Dongming Shen, Yuzhi Tang +3
Voice provides a natural and immediate interface for AI agents. Many settings in which voice agents could be useful, including meetings, households, and collaborative work, are inh…
Instruct-FD: Can Your Full-Duplex Speech System Follow Turn-Taking Instructions?
Yuzhi Tang, Wentao Ma, Xiling Zhao +17
Current full-duplex (FD) spoken dialogue systems can produce fluid interactions, yet it remains unclear whether they can adapt their turn-taking behavior when explicitly instructed…
IHBench: Evaluating Post-Interruption Recovery in Voice Agents with Structured Workflows
Ahmad Salimi, Wentao Ma, Yuzhi Tang +3
Voice agents deployed in structured workflows (customer service, healthcare scheduling, account management) must handle frequent user interruptions while maintaining progress throu…
ProactBench: Beyond What The User Asked For
Sepehr Harfi, Ahmad Salimi, Dongming Shen +1
Most LLM benchmarks score how well a model responds to explicit requests. They leave unmeasured a different conversational ability: noticing and acting on needs the user has implie…
RPGBENCH: Evaluating Large Language Models as Role-Playing Game Engines
Pengfei Yu, Dongming Shen, Silin Meng +8
We present RPGBench, the first benchmark designed to evaluate large language models (LLMs) as text-based role-playing game (RPG) engines. RPGBench comprises two core tasks: Game Cr…
Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes
Krishna C. Kalagarla, Dhruva Kartik, Dongming Shen +3
Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic…
Optimal Control of Partially Observable Markov Decision Processes with Finite Linear Temporal Logic Constraints
Krishna C. Kalagarla, Dhruva Kartik, Dongming Shen +3
Autonomous agents often operate in scenarios where the state is partially observed. In addition to maximizing their cumulative reward, agents must execute complex tasks with rich t…