1 citations · 1 across the 15 of their papers we have counts for
16 papers
Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generation
Yanhang Li, Zhichao Fan, Zexin Zhuang
Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the audited defense's active pipeline hook is known. We propose an active-path a…
The Moving Target: A Longitudinal Audit of Trust-Benchmark Score Drift Across Open-Source Chat LLM Release Lines
Zhichao Fan, Yanhang Li, Zexin Zhuang +2
Trust-benchmark scores reported on a chat-LLM release line are often carried across several checkpoints of the same line, as if the underlying model had not shifted between release…
Exploratory As-Analyzed No-Detection of Culturally-Marked Predicate-Triggered PII Amplification in a Synthetic-English RAG Probe: A Predicate-Resource-Confounded Audit
Yanhang Li, Zhichao Fan, Zexin Zhuang
We ask whether stereotype-loaded queries about culturally marked people leak more personal information from a retrieval-augmented generation (RAG) system than otherwise-equivalent…
Where Privacy Risk Lives in English-Source Multilingual RAG: A Stage-Decomposed Audit Across Five Query Languages
Yanhang Li, Zhichao Fan, Zexin Zhuang
A common assumption holds that switching to a non-English language makes a multilingual RAG system easier to attack for personal information. We test this on an English-source synt…
Auditing the Audit: Five Failure Modes in Benchmark-Validity Audits
Yanhang Li, Zhichao Fan, Zexin Zhuang
Governance frameworks ask AI providers and auditors for documented evaluation evidence, and perturbation-based construct-validity audits are a common form of that evidence. We argu…
Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting
Yingshuo Wang, Xian Sun, Lingdong Kong +4
Standard benchmarks evaluate time series foundation models (TSFMs) using aggregate metrics, but these can mask severe failures in critical operating regimes. We introduce regime-st…
Probe Choice Changes Canary-Memorization Verdicts: Three Post-Hoc Disagreement Case Studies in a Text-Dominant LoRA-Tuned Autoregressive Testbed
Zhichao Fan, Zexin Zhuang, Yanhang Li
We audit a fixed prefix-window mean-NLL memorization probe (K=20) on a Qwen2.5-VL-7B canary testbed and report three post-hoc cases where it disagrees with full-span secret NLL or…
Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees
Yingshuo Wang, Xian Sun, Yanhang Li +2
Tabular foundation models achieve strong accuracy on choice prediction tasks, but their predictions often violate the economic logic those tasks require: raising a price can increa…
When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents
Yanhang Li, Zhichao Fan, Zexin Zhuang
Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for flagging indirect prompt in…
Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MME
Zhichao Fan, Yanhang Li, Zexin Zhuang
Forced chain-of-thought (CoT) is widely assumed to make vision-language models more reliable on video question answering. We propose a small three-probe evaluation recipe to test t…
Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation
Xian Sun, Wei Gao, Yingshuo Wang +9
Reasoning models are increasingly used in settings where the final answer is not the only object of review: educational tools may show students intermediate steps, decision-support…
Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice
Yingshuo Wang, Xian Sun, Yanhang Li +2
Tabular foundation models achieve strong accuracy on choice prediction tasks, but their predictions often violate the economic logic those tasks require: raising a price sometimes…
Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Audit
Zexin Zhuang, Yanhang Li, Zhichao Fan
This is a planning-method note with an unpaired pilot audit. We adapt the classical paired-binary sample-size calculation (Miettinen, 1968) to quantization benchmarks, giving a con…
SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks
Yanhang Li, Zhichao Fan, Zexin Zhuang
Pairwise model comparisons drawn from foundation-model benchmarks ("A is safer than B") are read as quantitative verdicts but hinge on harness choices benchmark papers under-specif…
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
Yanhang Li, Zhichao Fan, Zexin Zhuang
Evaluations of unlearning on reasoning models sometimes show a bypass pattern. The answer side looks unlearned, but the model's own thinking trace keeps emitting the forgotten cont…
MoE: Optimizing Collaborative Inference for Edge Large Language Models
Lyudong Jin, Yanning Zhang, Yanhan Li +4
Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing tasks. Exploiting the heterogeneous capabilities of edge L…