6 papers · 1 filter
Large Language Models Are Still Misled by Simple Bias Ensembles
Zhouhao Sun, Zhiyuan Kan, Xiao Ding +5
With the evolution of large language models (LLMs), their robustness against individual simple biases has been enhanced. However, we observe that the ensemble of multiple simple bi…
Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoning
Yang He, Xiao Ding, Bibo Cai +5
While reasoning-augmented large language models (RLLMs) significantly enhance complex task performance through extended reasoning chains, they inevitably introduce substantial unne…
Information Gain-Guided Causal Intervention for Autonomous Debiasing Large Language Models
Zhouhao Sun, Xiao Ding, Li Du +5
Despite significant progress, recent studies indicate that current large language models (LLMs) may still capture dataset biases and utilize them during inference, leading to the p…
Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
Yang Zhao, Li Du, Xiao Ding +10
Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning. However, the limited ava…
Causal-Guided Active Learning for Debiasing Large Language Models
Li Du, Zhouhao Sun, Xiao Ding +5
Although achieving promising performance, recent analyses show that current generative large language models (LLMs) may still capture dataset biases and utilize them for generation…
Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
Yang Zhao, Li Du, Xiao Ding +5
Through pretraining on a corpus with various sources, Large Language Models (LLMs) have gained impressive performance. However, the impact of each component of the pretraining corp…