7 papers
Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts
Yongxi Zhou, Wenbo Ye, Yuanzhe Liu +3
Automatic safety judges -- for example Llama Guard, or GPT-4o given a grading prompt -- produce the numbers behind reported jailbreak success rates, defense evaluations, and safety…
Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity
Yongxi Zhou, Junwei Yao, Yuanzhe Liu +4
A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form. We ask whether that reading is faithful: when an item's intent…
Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models
Darpan Aswal, Thomas Palmeira Ferraz, Yongxin Zhou +1
Latent reasoning models (LRMs) replace explicit chain-of-thought with continuous thoughts. Recent work treats observable latent-state patterns, such as BFS-like frontiers and decod…
How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluation
Yongxi Zhou, Lai Yun Choi, Jiaxi Wen +2
A benchmark score is a measurement whose uncertainty has two sources: an aleatoric component from stochastic decoding and an epistemic component from the finite problem set. Using…
What Matters to an LLM? Behavioral and Computational Evidences from Summarization
Yongxin Zhou, Changshun Wu, Philippe Mulhem +2
Large Language Models (LLMs) are now state-of-the-art at summarization, yet the internal notion of importance that drives their information selections remains hidden. We propose to…
TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness
Yongxin Zhou, Philippe Mulhem, Didier Schwab
The evaluation of Retrieval-Augmented Generation (RAG) systems typically examines retrieval quality and generation parameters like temperature in isolation, overlooking their inter…
GETALP@AutoMin 2025: Leveraging RAG to Answer Questions based on Meeting Transcripts
Jeongwoo Kang, Markarit Vartampetian, Felix Herron +3
This paper documents GETALP's submission to the Third Run of the Automatic Minuting Shared Task at SIGDial 2025. We participated in Task B: question-answering based on meeting tran…