Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Equivalence of Context and Parameter Updates in Modern Transformer Blocks
Adrian Goldwaser, Michael Munn, Javier Gonzalvo +1
Recent research has established that the impact of context in a vanilla transformer can be represented implicitly by forming a token-dependent, rank-1 patch to its MLP weights. Thi…
cs.LG2022
Understanding Sparse Feature Updates in Deep Networks using Iterative Linearisation
Adrian Goldwaser, Hong Ge
Larger and deeper networks generalise well despite their increased capacity to overfit. Understanding why this happens is theoretically and practically important. One recent approa…