11 papers
SCOPE: Sequential Conformal Probing for Reliable OOD Rejection in LLM Services
Zhuoyun Li, Boxuan Wang, Changshun Wu +2
Rejecting inputs outside the defined in-distribution (IND) service scope is critical for large language model (LLM) services, where unsupported requests should be filtered before f…
Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Prompts
Boxuan Wang, Zhuoyun Li, Xiaowei Huang +1
Large language models (LLMs) excel in reasoning and knowledge-intensive tasks but remain vulnerable to prompt-level adversarial attacks that preserve intent while triggering common…
FragileFlow: Spectral Control of Correct-but-Fragile Predictions for Foundation Model Robustness
Zhuoyun Li, Boxuan Wang, Jinwei Hu +2
Robust adaptation of LLMs and VLMs is often evaluated by average accuracy or average consistency under perturbations. However, these averages can hide a structured failure mode: a…
Where Do Prompt Perturbations Break Generation? A Segment-Level View of Robustness in LoRA-Tuned Language Models
Zhuoyun Li, Boxuan Wang, Jinwei Hu +6
Large language models are sensitive to minor prompt perturbations, yet existing robustness methods usually enforce consistency at the whole-sequence level. This holistic view can h…
Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models
Boxuan Wang, Zhuoyun Li, Xinmiao Huang +2
This paper primarily demonstrates a method to quantitatively assess the alignment between multi-step, structured reasoning in large language models and human preferences. We introd…
Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models
Xinmiao Huang, Qisong He, Zhenglin Huang +5
Spatial reasoning ability is crucial for Vision Language Models (VLMs) to support real-world applications in diverse domains including robotics, augmented reality, and autonomous n…