3 citations · 3 across the 12 of their papers we have counts for
12 papers
Enhancing SAE-based Steering via Neighbor Integrated Feature Selection
Yutian Liu, Xu Wang, Difan Zou
Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most existing SAE-based steering method…
Mismatch Matters: On-Policy Distillation Beyond Token Agreement
Zichao Yu, Chengzhi Yu, Shengze Xu +4
On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repet…
DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoders
Xu Wang, Bingqing Jiang, Yu Wan +3
Sparse autoencoders (SAEs) have become a standard tool for mechanistic interpretability in autoregressive large language models (LLMs), enabling researchers to extract sparse, huma…
ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
Jingwei Song, Meng Chen, Jie Xiao +15
Reinforcement learning (RL) is a critical stage in post-training large language models (LLMs), involving repeated interaction between rollout generation, reward evaluation, and cen…
SCULPT: Constraint-Guided Pruned MCTS that Carves Efficient Paths for Mathematical Reasoning
Qitong Fang, Haotian Li, Xu Wang
Automated agent workflows can enhance the problem-solving ability of large language models (LLMs), but common search strategies rely on stochastic exploration and often traverse im…
Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders
Xu Wang, Yan Hu, Benyou Wang +1
Sparse Autoencoders (SAEs) are widely used to steer large language models (LLMs), based on the assumption that their interpretable features naturally enable effective model behavio…
Model Unlearning via Sparse Autoencoder Subspace Guided Projections
Xu Wang, Zihao Li, Benyou Wang +2
Large language models (LLMs) store vast amounts of information, making them powerful yet raising privacy and safety concerns when selective knowledge removal is required. Existing…
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models
Zihao Li, Xu Wang, Yuzhe Yang +3
Large Language Models (LLMs) demonstrate the ability to solve reasoning and mathematical problems using the Chain-of-Thought (CoT) technique. Expanding CoT length, as seen in model…
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Xu Wang, Yan Hu, Wenyu Du +3
Fine-tuning significantly improves the performance of Large Language Models (LLMs), yet its underlying mechanisms remain poorly understood. This paper aims to provide an in-depth i…
Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models
Xinyi Leng, Jason Liang, Jack Mauro +8
Narrative data spans all disciplines and provides a coherent model of the world to the reader or viewer. Recent advancement in machine learning and Large Language Models (LLMs) hav…
Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
Wei Wen, Kuang-Hung Liu, Igor Fedorov +19
Neural Architecture Search (NAS) has demonstrated its efficacy in computer vision and potential for ranking systems. However, prior work focused on academic problems, which are eva…
Expanding the Vocabulary of BERT for Knowledge Base Construction
Dong Yang, Xu Wang, Remzi Celebi
Knowledge base construction entails acquiring structured information to create a knowledge base of factual and relational data, facilitating question answering, information retriev…