1 citations · 1 across the 7 of their papers we have counts for
8 papers
Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
Yihuai Hong, Shauli Ravfogel, Chen Zhao +1
Chain-of-thought (CoT) traces often serve as a proxy for how Large Language Models (LLMs) arrive at their answers. However, growing evidence shows that models' CoT often fails to r…
Identifying Crucial Attention Heads for Multilingual Language Models: Retrieval and Retrieval-Transition Heads
Shaswat Patel, Vishvesh Trivedi, Yue Han +2
Retrieval heads, a subset of attention heads in Transformers, were studied in English, showing its crucial role in retrieving information from the context. We expand the study of r…
Precise In-Parameter Concept Erasure in Large Language Models
Yoav Gur-Arieh, Clara Suslik, Yihuai Hong +2
Large language models (LLMs) often acquire knowledge during pretraining that is undesirable in downstream deployments, e.g., sensitive information or copyrighted content. Existing…
The Rise of Parameter Specialization for Knowledge Storage in Large Language Models
Yihuai Hong, Yiran Zhao, Wei Tang +3
Over time, a growing wave of large language models from various series has been introduced to the community. Researchers are striving to maximize the performance of language models…
The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction
Yihuai Hong, Dian Zhou, Meng Cao +2
Large language models (LLMs) excel on a variety of reasoning benchmarks, but previous studies suggest they sometimes struggle to generalize to unseen questions, potentially due to…
Dissecting Fine-Tuning Unlearning in Large Language Models
Yihuai Hong, Yuelin Zou, Lijie Hu +3
Fine-tuning-based unlearning methods prevail for preventing targeted harmful, sensitive, or copyrighted information within large language models while preserving overall capabiliti…
Intrinsic Test of Unlearning Using Parametric Knowledge Traces
Yihuai Hong, Lei Yu, Haiqin Yang +2
The task of "unlearning" certain concepts in large language models (LLMs) has attracted immense attention recently, due to its importance in mitigating undesirable model behaviours…
ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference
Ziqian Zeng, Yihuai Hong, Hongliang Dai +2
Early Exiting is one of the most popular methods to achieve efficient inference. Current early exiting methods adopt the (weighted) sum of the cross entropy loss of all internal cl…