7 papers · 1 filter
Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?
Qingjie Zhang, Xingzhang Ren, Zixuan Chen +6
Pretraining corpus composition shapes LLM capabilities, but it often remains hidden even when model weights are released. Prior work has inferred corpus mixtures or traced specific…
Auditing Chinese Web-scale Corpora via Sampled BPE Token Statistics
Qingjie Zhang, Ziqi Tang, Jie Zhang +7
Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora. However, auditing such corpora faces three challenges: (1) their web-scale size makes ful…
When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Models
Cheng Wang, Gelei Deng, Xianglin Yang +2
Large Audio-Language Models (LALMs) are enhanced with audio perception capabilities, enabling them to effectively process and understand multimodal inputs that combine audio and te…
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
Qingjie Zhang, Di Wang, Haoting Qian +7
Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability. However, recent works show that LLMs' intrinsic se…
Course-Correction: Safety Alignment Using Synthetic Preferences
Rongwu Xu, Yishuo Cai, Zhenhong Zhou +6
The risk of harmful content generated by large language models (LLMs) becomes a critical concern. This paper presents a systematic study on assessing and improving LLMs' capability…
Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Bias
Rongwu Xu, Zi'an Zhou, Tianwei Zhang +5
The common toxicity and societal bias in contents generated by large language models (LLMs) necessitate strategies to reduce harm. Present solutions often demand white-box access t…