7 papers
Generalization in Nonlinear Least Squares via Learned Feature Geometry
Ayub Kharel, Ilja Kuzborskij, Patrick Rebeschini +1
We study the generalization of ridge-regularized nonlinear least-squares models via on-average algorithmic stability, deriving error bounds for local minimizers in terms of a data-…
On-Average Stability of Multipass Preconditioned SGD and Effective Dimension
Simon Vary, Tyler Farghly, Ilja Kuzborskij +1
We study trade-offs between the population risk curvature, geometry of the noise, and preconditioning on the generalisation ability of the multipass Preconditioned Stochastic Gradi…
DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMs
Yuanhe Zhang, Ilja Kuzborskij, Jason D. Lee +2
Large Language Models (LLMs) demonstrate strong performance on mathematical problems when prompted with Chain-of-Thought (CoT), yet it remains unclear whether this success stems fr…
Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij +2
Algorithmic stability is a classical framework for analyzing the generalization error of learning algorithms. It predicts that an algorithm has small generalization error if it is…
Low-rank bias, weight decay, and model merging in neural networks
Ilja Kuzborskij, Yasin Abbasi Yadkori
We explore the low-rank structure of the weight matrices in neural networks at the stationary points (limiting solutions of optimization algorithms) with regularization (also…
Pointwise confidence estimation in the non-linear -regularized least squares
Ilja Kuzborskij, Yasin Abbasi Yadkori
We consider a high-probability non-asymptotic confidence estimation in the -regularized non-linear least-squares setting with fixed design. In particular, we study confiden…