182 citations · 252 across the 46 of their papers we have counts for
3 papers · 2 filters
Neural Collapse by Design: Learning Class Prototypes on the Hypersphere
Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1
Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degree…
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
Andreas D. Demou, Panagiotis Koromilas, James Oldfield +2
Many features in pretrained Transformers span multiple layers: they emerge through stages of inference, persist in the residual stream, or are built jointly by parallel MLPs. Cross…
PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding
Panagiotis Koromilas, Andreas D. Demou, James Oldfield +2
Sparse autoencoders (SAEs) interpret neural network representations by decomposing activations into sparse combinations of dictionary atoms. However, SAEs assume features combine a…