11 papers
Air Quality Downscaling with Station-Guided Pseudo-Supervision
Guorun Wang, Simone Foti, Andreas D. Demou +5
Super-resolving coarse atmospheric fields to local PM variations is uniquely challenged by a mismatch in spatial support: while pixels represent regional averages, ground-t…
PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding
Panagiotis Koromilas, Andreas D. Demou, James Oldfield +2
Sparse autoencoders (SAEs) interpret neural network representations by decomposing activations into sparse combinations of dictionary atoms. However, SAEs assume features combine a…
Neural Collapse by Design: Learning Class Prototypes on the Hypersphere
Panagiotis Koromilas, Theodoros Giannakopoulos, Mihalis A. Nicolaou +1
Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degree…
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
Andreas D. Demou, Panagiotis Koromilas, James Oldfield +2
Many features in pretrained Transformers span multiple layers: they emerge through stages of inference, persist in the residual stream, or are built jointly by parallel MLPs. Cross…
On the hydrodynamic behaviour of the immersed boundary -- lattice Boltzmann method for wetting problems
Elisa Bellantoni, Fabio Guglietta, Andreas Demou +6
We study the hydrodynamic behaviour of a mesoscale numerical model for wetting dynamics based on the immersed boundary - lattice Boltzmann (IBLB) method. This IBLB model features a…
Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders
James Oldfield, Shawn Im, Sharon Li +3
Multilayer perceptrons (MLPs) are an integral part of large language models, yet their dense representations render them difficult to understand, edit, and steer. Recent methods le…