7 papers
Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
Jiale Liu, Huajun Xi, Shaokun Zhang +6
Automated failure attribution uses LLMs to identify where and why agentic systems fail. As agents become more capable, their failures become subtler, making automated attribution i…
Model-agnostic Selective Labeling with Provable Statistical Guarantees
Huipeng Huang, Wenbo Liao, Huajun Xi +3
Obtaining high-quality labels for large datasets is expensive, requiring massive annotations from human experts. While AI models offer a cost-effective alternative by predicting la…
Distribution-informed Efficient Conformal Prediction for Full Ranking
Wenbo Liao, Huipeng Huang, Chen Jia +3
Quantifying uncertainty is critical for the safe deployment of ranking models in real-world applications. Recent work offers a rigorous solution using conformal prediction in a ful…
Exploring the Noise Robustness of Online Conformal Prediction
Huajun Xi, Kangdao Liu, Hao Zeng +2
Conformal prediction is an emerging technique for uncertainty quantification that constructs prediction sets guaranteed to contain the true label with a predefined probability. Rec…
Online Conformal Selection with Accept-to-Reject Changes
Kangdao Liu, Huajun Xi, Chi-Man Vong +1
Selecting a subset of promising candidates from a large pool is crucial across various scientific and real-world applications. Conformal selection offers a distribution-free and mo…
Does confidence calibration improve conformal prediction?
Huajun Xi, Jianguo Huang, Kangdao Liu +2
Conformal prediction is an emerging technique for uncertainty quantification that constructs prediction sets guaranteed to contain the true label with a predefined probability. Pre…