4 papers
Can Neural Networks Achieve Optimal Computational-statistical Tradeoff? An Analysis on Single-Index Model
Siyu Chen, Beining Wu, Miao Lu +2
In this work, we tackle the following question: Can neural networks trained with gradient-based methods achieve the optimal computational-statistical tradeoff in learning Gaussian…
INFUSER: Influence-Guided Self-Evolution Improves Reasoning
Siyu Chen, Miao Lu, Beining Wu +7
Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet existing methods either depend…
Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
Xingwu Chen, Miao Lu, Beining Wu +1
Using more test-time computation during language model inference, such as generating more intermediate thoughts or sampling multiple candidate answers, has proven effective in sign…
Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates
Miao Lu, Beining Wu, Xiaodong Yang +1
In this work, we theoretically investigate the generalization properties of neural networks (NN) trained by stochastic gradient descent (SGD) algorithm with large learning rates. U…