7 papers
From Atoms to Trees: Building a Structured Feature Forest with Hierarchical Sparse Autoencoders
Yifan Luo, Yang Zhan, Jiedong Jiang +4
Sparse autoencoders (SAEs) have proven effective for extracting monosemantic features from large language models (LLMs), yet these features are typically identified in isolation. H…
PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencils
Chun-Wun Cheng, Bin Dong, Carola-Bibiane Schönlieb +1
Neural operator models for solving partial differential equations (PDEs) often rely on global mixing mechanisms-such as spectral convolutions or attention-which tend to oversmooth…
CLGRPO: Reasoning Ability Enhancement for Small VLMs
Fanyi Wang, Binzhi Dong, Haotian Hu +2
Small Vision Language Models (SVLMs) generally refer to models with parameter sizes less than or equal to 2B. Their low cost and power consumption characteristics confer high comme…
InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
Yifan Luo, Zhennan Zhou, Bin Dong
Understanding the internal representations of large language models (LLMs) is a central challenge in interpretability research. Existing feature interpretability methods often rely…
Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting
Yifan Luo, Zhennan Zhou, Meitan Wang +1
In this paper, we investigate the safety mechanisms of instruction fine-tuned large language models (LLMs). We discover that re-weighting MLP neurons can significantly compromise a…
Demonstration Notebook: Finding the Most Suited In-Context Learning Example from Interactions
Yiming Tang, Bin Dong
Large language models (LLMs) benefit greatly from prompt engineering, with in-context learning standing as a pivital technique. While former approaches have provided various ways t…
Prompt Engineering Through the Lens of Optimal Control
Yifan Luo, Yiming Tang, Chengfeng Shen +2
Prompt Engineering (PE) has emerged as a critical technique for guiding Large Language Models (LLMs) in solving intricate tasks. Its importance is highlighted by its potential to s…