57 citations · 68 across the 10 of their papers we have counts for
4 papers · 1 filter
Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability
Haodong Wen, Kaiyue Wen, Jiaye Teng
We study logistic regression on linearly separable data under gradient descent with a large constant stepsize . Such dynamics may exhibit a characteristic Edge of Stability phen…
Towards Understanding Momentum Acceleration in River-Valley Loss Landscape
Miao Lu, Zeyu Bian, Kaiyue Wen +4
The empirical success of pretraining large language models has inspired a deeper investigation into the underlying loss landscapes and the optimization dynamics. Recent empirical a…
Configuration-to-Performance Scaling Law with Neural Ansatz
Huaqing Zhang, Kaiyue Wen, Tengyu Ma
Researchers build scaling laws to forecast the training performance of expensive large-scale runs with larger model size N and data size D. These laws assume that other training hy…
Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning
Arvind Mahankali, Kaiyue Wen, Tengyu Ma
Long chain-of-thought reasoning (Long CoT) is now fundamental to state-of-the-art LLMs, especially in mathematical reasoning. However, LLM generation is highly sequential, and long…