60 citations · 171 across the 27 of their papers we have counts for
27 papers
Generalization in Nonlinear Least Squares via Learned Feature Geometry
Ayub Kharel, Ilja Kuzborskij, Patrick Rebeschini +1
We study the generalization of ridge-regularized nonlinear least-squares models via on-average algorithmic stability, deriving error bounds for local minimizers in terms of a data-…
On-Average Stability of Multipass Preconditioned SGD and Effective Dimension
Simon Vary, Tyler Farghly, Ilja Kuzborskij +1
We study trade-offs between the population risk curvature, geometry of the noise, and preconditioning on the generalisation ability of the multipass Preconditioned Stochastic Gradi…
Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij +2
Algorithmic stability is a classical framework for analyzing the generalization error of learning algorithms. It predicts that an algorithm has small generalization error if it is…
DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMs
Yuanhe Zhang, Ilja Kuzborskij, Jason D. Lee +2
Large Language Models (LLMs) demonstrate strong performance on mathematical problems when prompted with Chain-of-Thought (CoT), yet it remains unclear whether this success stems fr…
Pointwise confidence estimation in the non-linear -regularized least squares
Ilja Kuzborskij, Yasin Abbasi Yadkori
We consider a high-probability non-asymptotic confidence estimation in the -regularized non-linear least-squares setting with fixed design. In particular, we study confiden…
Low-rank bias, weight decay, and model merging in neural networks
Ilja Kuzborskij, Yasin Abbasi Yadkori
We explore the low-rank structure of the weight matrices in neural networks at the stationary points (limiting solutions of optimization algorithms) with regularization (also…
To Believe or Not to Believe Your LLM
Yasin Abbasi Yadkori, Ilja Kuzborskij, András György +1
We explore uncertainty quantification in large language models (LLMs), with the goal to identify when uncertainty in responses given a query is large. We simultaneously consider bo…
Mitigating LLM Hallucinations via Conformal Abstention
Yasin Abbasi Yadkori, Ilja Kuzborskij, David Stutz +9
We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e.g., by saying "I don't know") in a general domain, instead of…
Better-than-KL PAC-Bayes Bounds
Ilja Kuzborskij, Kwang-Sung Jun, Yulian Wu +2
Let be a sequence of random elements, where is a fixed scalar function, are independent random variables (data), and i…
Mixture Weight Estimation and Model Prediction in Multi-source Multi-target Domain Adaptation
Yuyang Deng, Ilja Kuzborskij, Mehrdad Mahdavi
We consider the problem of learning a model from multiple heterogeneous sources with the goal of performing well on a new target distribution. The goal of learner is to mix these d…
Tighter PAC-Bayes Bounds Through Coin-Betting
Kyoungseok Jang, Kwang-Sung Jun, Ilja Kuzborskij +1
We consider the problem of estimating the mean of a sequence of random elements where is a fixed scalar function, ar…
Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks
Ilja Kuzborskij, Csaba Szepesvári
We explore the ability of overparameterized shallow ReLU neural networks to learn Lipschitz, nondifferentiable, bounded functions with additive noise when trained by Gradient Desce…
On the Role of Optimization in Double Descent: A Least Squares Study
Ilja Kuzborskij, Csaba Szepesvári, Omar Rivasplata +2
Empirically it has been observed that the performance of deep neural networks steadily improves as we increase model size, contradicting the classical view on overfitting and gener…
Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel
Dominic Richards, Ilja Kuzborskij
We revisit on-average algorithmic stability of GD for training overparameterised shallow neural networks and prove new generalisation and excess risk bounds without the NTK or PL a…
Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stopping
Ilja Kuzborskij, Csaba Szepesvári
We explore the ability of overparameterized shallow neural networks to learn Lipschitz regression functions with and without label noise when trained by Gradient Descent (GD). To a…
A Distribution-Dependent Analysis of Meta-Learning
Mikhail Konobeev, Ilja Kuzborskij, Csaba Szepesvári
A key problem in the theory of meta-learning is to understand how the task distributions influence transfer risk, the expected error of a meta-learner on a new task drawn from the…
PAC-Bayes Analysis Beyond the Usual Bounds
Omar Rivasplata, Ilja Kuzborskij, Csaba Szepesvari +1
We focus on a stochastic learning model where the learner observes a finite set of training examples and the output of the learning process is a data-dependent distribution over a…
Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
Ilja Kuzborskij, Claire Vernade, András György +1
We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated ba…
Locally-Adaptive Nonparametric Online Learning
Ilja Kuzborskij, Nicolò Cesa-Bianchi
One of the main strengths of online algorithms is their ability to adapt to arbitrary data sequences. This is especially important in nonparametric settings, where performance is m…
Efron-Stein PAC-Bayesian Inequalities
Ilja Kuzborskij, Csaba Szepesvári
We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe…
Distribution-Dependent Analysis of Gibbs-ERM Principle
Ilja Kuzborskij, Nicolò Cesa-Bianchi, Csaba Szepesvári
Gibbs-ERM learning is a natural idealized model of learning with stochastic optimization algorithms (such as Stochastic Gradient Langevin Dynamics and ---to some extent--- Stochast…
Efficient Linear Bandits through Matrix Sketching
Ilja Kuzborskij, Leonardo Cella, Nicolò Cesa-Bianchi
We prove that two popular linear contextual bandit algorithms, OFUL and Thompson Sampling, can be made efficient using Frequent Directions, a deterministic online sketching techniq…
Nonparametric Online Regression while Learning the Metric
Ilja Kuzborskij, Nicolò Cesa-Bianchi
We study algorithms for online nonparametric regression that learn the directions along which the regression function is smoother. Our algorithm learns the Mahalanobis metric based…
Data-Dependent Stability of Stochastic Gradient Descent
Ilja Kuzborskij, Christoph H. Lampert
We establish a data-dependent notion of algorithmic stability for Stochastic Gradient Descent (SGD), and employ it to develop novel generalization bounds. This is in contrast to pr…