8 papers
Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?
Qingjie Zhang, Xingzhang Ren, Zixuan Chen +6
Pretraining corpus composition shapes LLM capabilities, but it often remains hidden even when model weights are released. Prior work has inferred corpus mixtures or traced specific…
Auditing Chinese Web-scale Corpora via Sampled BPE Token Statistics
Qingjie Zhang, Ziqi Tang, Jie Zhang +7
Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora. However, auditing such corpora faces three challenges: (1) their web-scale size makes ful…
Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
Haoting Qian, Qingjie Zhang, Zhicong Huang +2
Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks inc…
Speculating LLMs' Chinese Training Data Pollution from Their Tokens
Qingjie Zhang, Di Wang, Haoting Qian +7
Tokens are basic elements in the datasets for LLM training. It is well-known that many tokens representing Chinese phrases in the vocabulary of GPT (4o/4o-mini/o1/o3/4.5/4.1/o4-min…
Understanding the Dilemma of Unlearning for Large Language Models
Qingjie Zhang, Haoting Qian, Zhicong Huang +5
Unlearning seeks to remove specific knowledge from large language models (LLMs), but its effectiveness remains contested. On one side, "forgotten" knowledge can often be recovered…
Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models
Qingjie Zhang, Yujia Fu, Yang Wang +5
Current answering paradigms for Large Reasoning Models (LRMs) often fail to account for the fact that some questions may lie beyond the model's operational capability boundary, lea…
An Engorgio Prompt Makes Large Language Model Babble on
Jianshuo Dong, Ziyuan Zhang, Qingjie Zhang +7
Auto-regressive large language models (LLMs) have yielded impressive performance in many real-world tasks. However, the new paradigm of these LLMs also exposes novel threats. In th…
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
Qingjie Zhang, Di Wang, Haoting Qian +7
Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability. However, recent works show that LLMs' intrinsic se…