6 citations · 10 across the 8 of their papers we have counts for
8 papers
INFUSER: Influence-Guided Self-Evolution Improves Reasoning
Siyu Chen, Miao Lu, Beining Wu +7
Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet existing methods either depend…
Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders
Siyu Chen, Heejune Sheen, Xuyuan Xiong +2
We study the challenge of achieving theoretically grounded feature recovery using Sparse Autoencoders (SAEs) for the interpretation of Large Language Models. Existing SAE training…
Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
Siyu Chen, Heejune Sheen, Tianhao Wang +1
In-context learning (ICL) is a cornerstone of large language model (LLM) functionality, yet its theoretical foundations remain elusive due to the complexity of transformer architec…
Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
Heejune Sheen, Siyu Chen, Tianhao Wang +1
We study gradient flow on the exponential loss for a classification problem with a one-layer softmax attention model, where the key and query weight matrices are trained separately…
Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
Siyu Chen, Heejune Sheen, Tianhao Wang +1
We study the dynamics of gradient flow for training a multi-head softmax attention model for in-context learning of multi-task linear regression. We establish the global convergenc…
Tensor Kernel Recovery for Spatio-Temporal Hawkes Processes
Heejune Sheen, Xiaonan Zhu, Yao Xie
We estimate the general influence functions for spatio-temporal Hawkes processes using a tensor recovery approach by formulating the location dependent influence function that capt…
An elementary approach for minimax estimation of Bernoulli proportion in the restricted parameter space
Heejune Sheen, Yajun Mei
We present an elementary mathematical method to find the minimax estimator of the Bernoulli proportion under the squared error loss when belongs to the restricted parameter…
Exploiting Aggregate Sparsity in Second Order Cone Relaxations for Quadratic Constrained Quadratic Programming Problems
Heejune Sheen, Makoto Yamashita
Among many approaches to increase the computational efficiency of semidefinite programming (SDP) relaxation for quadratic constrained quadratic programming problems (QCQPs), exploi…