2 citations · 2 across the 1 of their papers we have counts for
4 papers · 1 filter
Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity
Lei Yu, Jingcheng Niu, Zining Zhu +2
In this paper, we introduce DiscoGP, a novel framework for extracting self-contained modular units, or sheaves, within neural language models (LMs). Sheaves extend the concept of f…
Intrinsic Test of Unlearning Using Parametric Knowledge Traces
Yihuai Hong, Lei Yu, Haiqin Yang +2
The task of "unlearning" certain concepts in large language models (LLMs) has attracted immense attention recently, due to its importance in mitigating undesirable model behaviours…
Geometric Signatures of Compositionality Across a Language Model's Lifetime
Jin Hwa Lee, Thomas Jiralerspong, Lei Yu +2
By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimension…
The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction
Yihuai Hong, Dian Zhou, Meng Cao +2
Large language models (LLMs) excel on a variety of reasoning benchmarks, but previous studies suggest they sometimes struggle to generalize to unseen questions, potentially due to…