402 citations · 923 across the 41 of their papers we have counts for
9 papers · 1 filter
Scalable Energy-Based Models via Adversarial Training: Unifying Discrimination and Generation
Xuwang Yin, Claire Zhang, Julie Steele +2
Simultaneously achieving robust classification and high-fidelity generative modeling within a single framework presents a significant challenge. Hybrid approaches, such as Joint En…
Learning to Interpret Weight Differences in Language Models
Avichal Goel, Yoon Kim, Nir Shavit +1
Finetuning (pretrained) language models is a standard approach for updating their internal parametric knowledge and specializing them to new tasks and domains. However, the corresp…
Expand Neurons, Not Parameters
Linghao Kong, Inimai Subramanian, Yonadav Shavit +3
This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance. We show that this gain cor…
Budgeted Broadcast: An Activity-Dependent Pruning Rule for Neural Network Efficiency
Yaron Meirovitch, Fuming Yang, Jeff Lichtman +1
Most pruning methods remove parameters ranked by impact on loss (e.g., magnitude or gradient). We propose Budgeted Broadcast (BB), which gives each unit a local traffic budget (the…
Negative Pre-activations Differentiate Syntax
Linghao Kong, Angelina Ning, Micah Adler +1
Modern large language models increasingly use smooth activation functions such as GELU or SiLU, allowing negative pre-activations to carry both signal and gradient. Nevertheless, m…
The Birth of Knowledge: Emergent Features across Time, Space, and Scale in Large Language Models
Shashata Sawmya, Micah Adler, Nir Shavit
This paper studies the emergence of interpretable categorical features within large language models (LLMs), analyzing their behavior across training checkpoints (time), transformer…
Towards Combinatorial Interpretability of Neural Computation
Micah Adler, Dan Alistarh, Nir Shavit
We introduce combinatorial interpretability, a methodology for understanding neural computation by analyzing the combinatorial structures in the sign-based categorization of a netw…
Cascade Detector Analysis and Application to Biomedical Microscopy
Thomas L. Athey, Shashata Sawmya, Nir Shavit
As both computer vision models and biomedical datasets grow in size, there is an increasing need for efficient inference algorithms. We utilize cascade detectors to efficiently ide…
NeuroADDA: Active Discriminative Domain Adaptation in Connectomic
Shashata Sawmya, Thomas L. Athey, Gwyneth Liu +1
Training segmentation models from scratch has been the standard approach for new electron microscopy connectomics datasets. However, leveraging pretrained models from existing data…