4 papers
Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging
Jinglan Gong, Jiefan Lu, Hewei Guo +5
Evaluating large language models (LLMs) as multi-turn conversational partners requires probing capabilities that single-turn benchmarks miss: persona consistency, evolving intent t…
StableRCA: Robust Graph-Agnostic Mechanism-Level Root Cause Analysis
Xiaoyu Lin, Nicholas Tagliapietra, Kehan Li +2
Root-Cause Analysis (RCA) seeks to identify the variables responsible for abnormal system behavior in complex domains such as manufacturing, cloud computing, and healthcare. Existi…
Generating Risky Samples with Conformity Constraints via Diffusion Models
Han Yu, Hao Zou, Xingxuan Zhang +4
Although neural networks achieve promising performance in many tasks, they may still fail when encountering some examples and bring about risks to applications. To discover risky s…
ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
Han Yu, Kehan Li, Dongbai Li +3
Recently, there has been gradually more attention paid to Out-of-Distribution (OOD) performance prediction, whose goal is to predict the performance of trained models on unlabeled…