4 papers
InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
Yifan Luo, Zhennan Zhou, Bin Dong
Understanding the internal representations of large language models (LLMs) is a central challenge in interpretability research. Existing feature interpretability methods often rely…
From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoders
Yifan Luo, Yang Zhan, Jiedong Jiang +4
Sparse autoencoders (SAEs) have proven effective for extracting monosemantic features from large language models (LLMs), yet these features are typically identified in isolation. H…
PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencils
Chun-Wun Cheng, Bin Dong, Carola-Bibiane Schönlieb +1
Neural operator models for solving partial differential equations (PDEs) often rely on global mixing mechanisms-such as spectral convolutions or attention-which tend to oversmooth…
CLGRPO: Reasoning Ability Enhancement for Small VLMs
Fanyi Wang, Binzhi Dong, Haotian Hu +2
Small Vision Language Models (SVLMs) generally refer to models with parameter sizes less than or equal to 2B. Their low cost and power consumption characteristics confer high comme…