402 citations · 923 across the 41 of their papers we have counts for
17 papers · 1 filter
The Sparsity Whisperer
Linghao Kong, Inimai Subramanian, Micah Adler +3
Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs. We argue that this overlooks a…
Toy Combinatorial Interpretability Models Reveal Lottery Tickets in Early Feature Space
Alon Bebchuk, Nir Shavit
The lottery ticket hypothesis posits that dense networks contain sparse subnetworks, ``winning tickets,'' that, when rewound to their initial weights and retrained in isolation, ma…
An Interpretable Latency Model for Speculative Decoding in LLM Serving
Linghao Kong, Megan Flynn, Michael Peng +3
Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified by a larger target model in p…
Understanding Empirical Unlearning with Combinatorial Interpretability
Shingo Kodama, Niv Cohen, Micah Adler +1
While many recent methods aim to unlearn or remove knowledge from pretrained models, seemingly erased knowledge often persists and can be recovered in various ways. Because large f…
Scalable Energy-Based Models via Adversarial Training: Unifying Discrimination and Generation
Xuwang Yin, Claire Zhang, Julie Steele +2
Simultaneously achieving robust classification and high-fidelity generative modeling within a single framework presents a significant challenge. Hybrid approaches, such as Joint En…
Learning to Interpret Weight Differences in Language Models
Avichal Goel, Yoon Kim, Nir Shavit +1
Finetuning (pretrained) language models is a standard approach for updating their internal parametric knowledge and specializing them to new tasks and domains. However, the corresp…
Expand Neurons, Not Parameters
Linghao Kong, Inimai Subramanian, Yonadav Shavit +3
This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance. We show that this gain cor…
Budgeted Broadcast: An Activity-Dependent Pruning Rule for Neural Network Efficiency
Yaron Meirovitch, Fuming Yang, Jeff Lichtman +1
Most pruning methods remove parameters ranked by impact on loss (e.g., magnitude or gradient). We propose Budgeted Broadcast (BB), which gives each unit a local traffic budget (the…
Negative Pre-activations Differentiate Syntax
Linghao Kong, Angelina Ning, Micah Adler +1
Modern large language models increasingly use smooth activation functions such as GELU or SiLU, allowing negative pre-activations to carry both signal and gradient. Nevertheless, m…
Towards Combinatorial Interpretability of Neural Computation
Micah Adler, Dan Alistarh, Nir Shavit
We introduce combinatorial interpretability, a methodology for understanding neural computation by analyzing the combinatorial structures in the sign-based categorization of a netw…
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Tony T. Wang, John Hughes, Henry Sleight +7
Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem. In this paper, we investigate the d…
Wasserstein Distances, Neuronal Entanglement, and Sparsity
Shashata Sawmya, Linghao Kong, Ilia Markov +2
Disentangling polysemantic neurons is at the core of many current approaches to interpretability of large language models. Here we attempt to study how disentanglement can be used…
Forbidden Facts: An Investigation of Competing Objectives in Llama-2
Tony T. Wang, Miles Wang, Kaivalya Hariharan +1
LLMs often face competing pressures (for example helpfulness vs. harmlessness). To understand how models resolve such conflicts, we study Llama-2-chat models on the forbidden fact…
Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework
Lu Mi, Tianxing He, Core Francisco Park +3
Latent-space interpolation is commonly used to demonstrate the generalization ability of deep latent variable models. Various algorithms have been proposed to calculate the best tr…
On the Predictability of Pruning Across Scales
Jonathan S. Rosenfeld, Jonathan Frankle, Michael Carbin +1
We show that the error of iteratively magnitude-pruned networks empirically follows a scaling law with interpretable coefficients that depend on the architecture and task. We funct…
A Constructive Prediction of the Generalization Error Across Scales
Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov +1
The dependency of the generalization error of neural networks on model and dataset size is of critical importance both in practice and for understanding the theory of neural networ…
Deep Learning is Robust to Massive Label Noise
David Rolnick, Andreas Veit, Serge Belongie +1
Deep neural networks trained on large supervised datasets have led to impressive results in image classification and other tasks. However, well-annotated datasets can be time-consu…