Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Sparse Inter-Layer Dependencies of Transformer FFN Neurons
Johannes Knittel, Hanspeter Pfister
Feedforward network (FFN) blocks account for a large fraction of the parameters and computation in Transformer architectures, yet their internal structure remains difficult to inte…
cs.LG2024
GPT-2 Through the Lens of Vector Symbolic Architectures
Johannes Knittel, Tushaar Gangavarapu, Hendrik Strobelt +1
Understanding the general priniciples behind transformer models remains a complex endeavor. Experiments with probing and disentangling features using sparse autoencoders (SAE) sugg…