533 citations · 605 across the 21 of their papers we have counts for
Showing 2023 · cs.LGShow all
3 papers · 2 filters
cs.LG2023★ 1 cited
Generating Interpretable Networks using Hypernetworks
Isaac Liao, Ziming Liu, Max Tegmark
An essential goal in mechanistic interpretability to decode a network, i.e., to convert a neural network's raw weights to an interpretable algorithm. Given the difficulty of the de…
cs.LG2023
Grokking as Compression: A Nonlinear Complexity Perspective
Ziming Liu, Ziqian Zhong, Max Tegmark
We attribute grokking, the phenomenon where generalization is much delayed after memorization, to compression. To do so, we define linear mapping number (LMN) to measure network co…
cs.LG2023
A Neural Scaling Law from Lottery Ticket Ensembling
Ziming Liu, Max Tegmark
Neural scaling laws (NSL) refer to the phenomenon where model performance improves with scale. Sharma & Kaplan analyzed NSL using approximation theory and predict that MSE losses d…