60 citations · 171 across the 27 of their papers we have counts for
20 papers · 1 filter
On-Average Stability of Multipass Preconditioned SGD and Effective Dimension
Simon Vary, Tyler Farghly, Ilja Kuzborskij +1
We study trade-offs between the population risk curvature, geometry of the noise, and preconditioning on the generalisation ability of the multipass Preconditioned Stochastic Gradi…
Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij +2
Algorithmic stability is a classical framework for analyzing the generalization error of learning algorithms. It predicts that an algorithm has small generalization error if it is…
Pointwise confidence estimation in the non-linear -regularized least squares
Ilja Kuzborskij, Yasin Abbasi Yadkori
We consider a high-probability non-asymptotic confidence estimation in the -regularized non-linear least-squares setting with fixed design. In particular, we study confiden…
Low-rank bias, weight decay, and model merging in neural networks
Ilja Kuzborskij, Yasin Abbasi Yadkori
We explore the low-rank structure of the weight matrices in neural networks at the stationary points (limiting solutions of optimization algorithms) with regularization (also…
To Believe or Not to Believe Your LLM
Yasin Abbasi Yadkori, Ilja Kuzborskij, András György +1
We explore uncertainty quantification in large language models (LLMs), with the goal to identify when uncertainty in responses given a query is large. We simultaneously consider bo…
Mitigating LLM Hallucinations via Conformal Abstention
Yasin Abbasi Yadkori, Ilja Kuzborskij, David Stutz +9
We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e.g., by saying "I don't know") in a general domain, instead of…
Better-than-KL PAC-Bayes Bounds
Ilja Kuzborskij, Kwang-Sung Jun, Yulian Wu +2
Let be a sequence of random elements, where is a fixed scalar function, are independent random variables (data), and i…
Mixture Weight Estimation and Model Prediction in Multi-source Multi-target Domain Adaptation
Yuyang Deng, Ilja Kuzborskij, Mehrdad Mahdavi
We consider the problem of learning a model from multiple heterogeneous sources with the goal of performing well on a new target distribution. The goal of learner is to mix these d…
Tighter PAC-Bayes Bounds Through Coin-Betting
Kyoungseok Jang, Kwang-Sung Jun, Ilja Kuzborskij +1
We consider the problem of estimating the mean of a sequence of random elements where is a fixed scalar function, ar…
Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks
Ilja Kuzborskij, Csaba Szepesvári
We explore the ability of overparameterized shallow ReLU neural networks to learn Lipschitz, nondifferentiable, bounded functions with additive noise when trained by Gradient Desce…
On the Role of Optimization in Double Descent: A Least Squares Study
Ilja Kuzborskij, Csaba Szepesvári, Omar Rivasplata +2
Empirically it has been observed that the performance of deep neural networks steadily improves as we increase model size, contradicting the classical view on overfitting and gener…
Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stopping
Ilja Kuzborskij, Csaba Szepesvári
We explore the ability of overparameterized shallow neural networks to learn Lipschitz regression functions with and without label noise when trained by Gradient Descent (GD). To a…
Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
Ilja Kuzborskij, Claire Vernade, András György +1
We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated ba…
Locally-Adaptive Nonparametric Online Learning
Ilja Kuzborskij, Nicolò Cesa-Bianchi
One of the main strengths of online algorithms is their ability to adapt to arbitrary data sequences. This is especially important in nonparametric settings, where performance is m…
Efron-Stein PAC-Bayesian Inequalities
Ilja Kuzborskij, Csaba Szepesvári
We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe…
Distribution-Dependent Analysis of Gibbs-ERM Principle
Ilja Kuzborskij, Nicolò Cesa-Bianchi, Csaba Szepesvári
Gibbs-ERM learning is a natural idealized model of learning with stochastic optimization algorithms (such as Stochastic Gradient Langevin Dynamics and ---to some extent--- Stochast…
Efficient Linear Bandits through Matrix Sketching
Ilja Kuzborskij, Leonardo Cella, Nicolò Cesa-Bianchi
We prove that two popular linear contextual bandit algorithms, OFUL and Thompson Sampling, can be made efficient using Frequent Directions, a deterministic online sketching techniq…
Nonparametric Online Regression while Learning the Metric
Ilja Kuzborskij, Nicolò Cesa-Bianchi
We study algorithms for online nonparametric regression that learn the directions along which the regression function is smoother. Our algorithm learns the Mahalanobis metric based…
Data-Dependent Stability of Stochastic Gradient Descent
Ilja Kuzborskij, Christoph H. Lampert
We establish a data-dependent notion of algorithmic stability for Stochastic Gradient Descent (SGD), and employ it to develop novel generalization bounds. This is in contrast to pr…
Fast Rates by Transferring from Auxiliary Hypotheses
Ilja Kuzborskij, Francesco Orabona
In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We foc…