When Less is More: Understanding When Token Filtering Helps and Fails in AI-generated Text Detection

Xiaoyang Han, Lvxiaowei Xu, Ming Cai

The rapid advancement of large language models (LLMs) has made AI-generated text detection increasingly critical. Existing zero-shot detectors assume that more token-level evidence…