42 citations · 157 across the 17 of their papers we have counts for
4 papers · 1 filter
Rethinking Self-Distillation for Multi-Teacher Capability Merging
Roy Xie, Dan Friedman, Feng Nan +7
Combining capabilities of multiple expert models trained starting from the same base checkpoint has become increasingly common in frontier language-model post-training. Recent tren…
Interpretability Illusions in the Generalization of Simplified Models
Dan Friedman, Andrew Lampinen, Lucas Dixon +2
A common method to study deep learning systems is to use simplified model representations--for example, using singular value decomposition to visualize the model's hidden states in…
Learning Transformer Programs
Dan Friedman, Alexander Wettig, Danqi Chen
Recent research in mechanistic interpretability has attempted to reverse-engineer Transformer models by carefully inspecting network weights and activations. However, these approac…
The Vendi Score: A Diversity Evaluation Metric for Machine Learning
Dan Friedman, Adji Bousso Dieng
Diversity is an important criterion for many areas of machine learning (ML), including generative modeling and dataset curation. However, existing metrics for measuring diversity a…