6 papers
Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
Rui Lin, Zhenyu Jin, Guancheng Zhou +7
While modern transformer neural networks achieve grandmaster-level performance in chess and other reasoning tasks, their internal computation process remains largely opaque. Focusi…
Evolution of Concepts in Language Model Pre-Training
Xuyang Ge, Wentao Shu, Jiaxing Wu +3
Language models obtain extensive capabilities through pre-training. However, the pre-training process remains a black box. In this work, we track linear interpretable feature evolu…
Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
Zhengfu He, Wentao Shu, Xuyang Ge +9
Sparse Autoencoders (SAEs) have emerged as a powerful unsupervised method for extracting sparse representations from language models, yet scalable training remains a significant ch…
TravelAgent: An AI Assistant for Personalized Travel Planning
Aili Chen, Xuyang Ge, Ziquan Fu +2
As global tourism expands and artificial intelligence technology advances, intelligent travel planning services have emerged as a significant research focus. Within dynamic real-wo…
Automatically Identifying Local and Global Circuits with Linear Computation Graphs
Xuyang Ge, Fukang Zhu, Wentao Shu +3
Circuit analysis of any certain model behavior is a central task in mechanistic interpretability. We introduce our circuit discovery pipeline with Sparse Autoencoders (SAEs) and a…
SurveyAgent: A Conversational System for Personalized and Efficient Research Survey
Xintao Wang, Jiangjie Chen, Nianqi Li +6
In the rapidly advancing research fields such as AI, managing and staying abreast of the latest scientific literature has become a significant challenge for researchers. Although p…