3 papers
cs.AI2026
The Safety Operator: Modulating the Expression of Safety Instructions via Spectral Optimization
Benoit Dherin, Michael Munn, Xavier Gonzalvo +14
Context tokens in a transformer-based language model can be absorbed into the model's weights as a multiplicative operator. We study this operator in the setting of safety instruct…
cs.LG2025
Equivalence of Context and Parameter Updates in Modern Transformer Blocks
Adrian Goldwaser, Michael Munn, Javier Gonzalvo +1
Recent research has established that the impact of context in a vanilla transformer can be represented implicitly by forming a token-dependent, rank-1 patch to its MLP weights. Thi…
cs.LG2022
Understanding Sparse Feature Updates in Deep Networks using Iterative Linearisation
Adrian Goldwaser, Hong Ge
Larger and deeper networks generalise well despite their increased capacity to overfit. Understanding why this happens is theoretically and practically important. One recent approa…