4 papers
MonoLoss: A Training Objective for Interpretable Monosemantic Representations
Ali Nasiri-Sarvi, Anh Tien Nguyen, Hassan Rivaz +2
Sparse autoencoders (SAEs) decompose polysemantic neural representations, where neurons respond to multiple unrelated concepts, into monosemantic features that capture single, inte…
2DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classification
Jingwei Zhang, Anh Tien Nguyen, Xi Han +4
Efficiently modeling large 2D contexts is essential for various fields including Giga-Pixel Whole Slide Imaging (WSI) and remote sensing. Transformer-based models offer high parall…
LBMamba: Locally Bi-directional Mamba
Jingwei Zhang, Xi Han, Hong Qin +2
Mamba, a State Space Model (SSM) that accelerates training by recasting recurrence as a parallel scan, has recently emerged as a linearly-scaling alternative to self-attention. Bec…
GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations
Xi Han, Jingwei Zhang, Dimitris Samaras +2
The neural operator (NO) framework has emerged as a powerful tool for solving partial differential equations (PDEs). Recent NOs are dominated by the Transformer architecture, which…