5 papers
Debiasing LLMs by Fine-tuning
Zhenyu Gao, Wenxi Jiang, Yutong Yan
Prior research shows that large language models (LLMs) exhibit systematic extrapolation bias when forming predictions from both experimental and real-world data, and that prompt-ba…
DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining
Yutong Yan, Raphael Tang, Zhenyu Gao +2
Large language models pretrained on internet-scale data risk lookahead bias in forecasting tasks, as they may have already seen the true outcome during training. To address this, w…
Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification
Zenghao Duan, Zhiyi Yin, Zhichao Shi +8
Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content, restricting their safe deployment. While traditional methods (e.g.…
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
Yu Yan, Sheng Sun, Zhe Wang +6
With the development of Large Language Models (LLMs), numerous efforts have revealed their vulnerabilities to jailbreak attacks. Although these studies have driven the progress in…
GloSS over Toxicity: Understanding and Mitigating Toxicity in LLMs via Global Toxic Subspace
Zenghao Duan, Zhiyi Yin, Zhichao Shi +6
This paper investigates the underlying mechanisms of toxicity generation in Large Language Models (LLMs) and proposes an effective detoxification approach. Prior work typically con…