5 papers
Understanding Reasoning from Pretraining to Post-Training
Jingyan Shen, Ang Li, Salman Rahman +4
Reinforcement learning (RL) has become central to improving large language models (LLMs) on complex reasoning tasks, yet RL post-training is largely studied in isolation from the p…
Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysis
Seunghoon Paik, Kangjie Zhou, Matus Telgarsky +1
We introduce \textit{basic inequalities} for first-order iterative optimization algorithms, forming a simple and versatile framework that connects implicit and explicit regularizat…
Astral Space: Convex Analysis at Infinity
Miroslav DudÃk, Robert E. Schapire, Matus Telgarsky
Not all convex functions on have finite minimizers; some can only be minimized by a sequence as it heads to infinity. In this work, we aim to develop a theory for un…
Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regression
Jingfeng Wu, Peter Bartlett, Matus Telgarsky +1
In overparameterized logistic regression, gradient descent (GD) iterates diverge in norm while converging in direction to the maximum -margin solution -- a phenomenon known…
Spectrum Extraction and Clipping for Implicitly Linear Layers
Ali Ebrahimpour Boroojeny, Matus Telgarsky, Hari Sundaram
We show the effectiveness of automatic differentiation in efficiently and correctly computing and controlling the spectrum of implicitly linear operators, a rich family of layer ty…