3 papers
cs.CL2025
Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models
Shuxun Wang, Qingyu Yin, Chak Tou Leong +2
Repetition curse is a phenomenon where Large Language Models (LLMs) generate repetitive sequences of tokens or cyclic sequences. While the repetition curse has been widely observed…
cs.AI2025
Constrain Alignment with Sparse Autoencoders
Qingyu Yin, Chak Tou Leong, Minjun Zhu +7
The alignment of large language models (LLMs) with human preferences remains a key challenge. While post-training techniques like Reinforcement Learning from Human Feedback (RLHF)…
cs.LG2024
Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning
Qingyu Yin, Xuzheng He, Luoao Deng +5
Fine-tuning and in-context learning (ICL) are two prevalent methods in imbuing large language models with task-specific knowledge. It is commonly believed that fine-tuning can surp…