49 citations · 412 across the 50 of their papers we have counts for
22 papers · 1 filter
Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty
Felix Biggs, Benjamin Guedj
We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages informati…
On Margins and Generalisation for Voting Classifiers
Felix Biggs, Valentina Zantedeschi, Benjamin Guedj
We study the generalisation properties of majority voting on finite ensembles of classifiers, proving margin-based generalisation bounds via the PAC-Bayes theory. These provide sta…
Online PAC-Bayes Learning
Maxime Haddouche, Benjamin Guedj
Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at once, prior to inference or prediction. This somewhat departs from many contemporary learning…
On PAC-Bayesian reconstruction guarantees for VAEs
Badr-Eddine Chérief-Abdellatif, Yuyang Shi, Arnaud Doucet +1
Despite its wide use and empirical successes, the theoretical understanding and study of the behaviour and performance of the variational autoencoder (VAE) have only emerged in the…
Non-Vacuous Generalisation Bounds for Shallow Neural Networks
Felix Biggs, Benjamin Guedj
We focus on a specific class of shallow neural networks with a single hidden layer, namely those with -normalised data and either a sigmoid-shaped Gaussian error function ("er…
Progress in Self-Certified Neural Networks
Maria Perez-Ortiz, Omar Rivasplata, Emilio Parrado-Hernandez +2
A learning method is self-certified if it uses all available data to simultaneously learn a predictor and certify its quality with a tight statistical certificate that is valid on…
Learning PAC-Bayes Priors for Probabilistic Neural Networks
Maria Perez-Ortiz, Omar Rivasplata, Benjamin Guedj +5
Recent works have investigated deep learning models trained by optimising PAC-Bayes bounds, with priors that are learnt on subsets of the data. This combination has been shown to l…
On Margins and Derandomisation in PAC-Bayes
Felix Biggs, Benjamin Guedj
We give a general recipe for derandomising PAC-Bayesian bounds using margins, with the critical ingredient being that our randomised predictions concentrate around some value. The…
Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound
Valentina Zantedeschi, Paul Viallard, Emilie Morvant +4
We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary dist…
A PAC-Bayesian Perspective on Structured Prediction with Implicit Loss Embeddings
Théophile Cantelobre, Benjamin Guedj, María Pérez-Ortiz +1
Many practical machine learning tasks can be framed as Structured prediction problems, where several output variables are predicted and considered interdependent. Recent theoretica…
Upper and Lower Bounds on the Performance of Kernel PCA
Maxime Haddouche, Benjamin Guedj, John Shawe-Taylor
Principal Component Analysis (PCA) is a popular method for dimension reduction and has attracted an unfailing interest for decades. More recently, kernel PCA (KPCA) has emerged as…
Cluster-Specific Predictions with Multi-Task Gaussian Processes
Arthur Leroy, Pierre Latouche, Benjamin Guedj +1
A model involving Gaussian processes (GPs) is introduced to simultaneously handle multi-task learning, clustering, and prediction for multiple functional data. This procedure acts…
PAC-Bayesian Bound for the Conditional Value at Risk
Zakaria Mhammedi, Benjamin Guedj, Robert C. Williamson
Conditional Value at Risk (CVaR) is a family of "coherent risk measures" which generalize the traditional mathematical expectation. Widely used in mathematical finance, it is garne…
Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networks
Felix Biggs, Benjamin Guedj
We make three related contributions motivated by the challenge of training stochastic neural networks, particularly in a PAC-Bayesian setting: (1) we show how averaging over an ens…
Kernel-Based Ensemble Learning in Python
Benjamin Guedj, Bhargav Srinivasa Desikan
We propose a new supervised learning algorithm, for classification and regression problems where two or more preliminary predictors are available. We introduce \texttt{KernelCobra}…
Still no free lunches: the price to pay for tighter PAC-Bayes bounds
Benjamin Guedj, Louis Pujol
"No free lunch" results state the impossibility of obtaining meaningful bounds on the error of a learning algorithm without prior assumptions and modelling. Some models are expensi…
PAC-Bayesian Contrastive Unsupervised Representation Learning
Kento Nozawa, Pascal Germain, Benjamin Guedj
Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has colle…
Online k-means Clustering
Vincent Cohen-Addad, Benjamin Guedj, Varun Kanade +1
We study the problem of online clustering where a clustering algorithm has to assign a new point that arrives to one of clusters. The specific formulation we use is the -mea…
PAC-Bayes Un-Expected Bernstein Inequality
Zakaria Mhammedi, Peter D. Grunwald, Benjamin Guedj
We present a new PAC-Bayesian generalization bound. Standard bounds contain a $\sqrt{L_n \cdot \KL/n}$ complexity term which dominates unless , the empirical error of the lear…
Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks
Gaël Letarte, Pascal Germain, Benjamin Guedj +1
We present a comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. Our contributions are twofold: (i) we develop an end-to-e…
Model Validation Using Mutated Training Labels: An Exploratory Study
Jie M. Zhang, Mark Harman, Benjamin Guedj +2
We introduce an exploratory study on Mutation Validation (MV), a model validation method using mutated training labels for supervised learning. MV mutates training data labels, ret…
Revisiting clustering as matrix factorisation on the Stiefel manifold
Stéphane Chrétien, Benjamin Guedj
This paper studies clustering for possibly high dimensional data (e.g. images, time series, gene expression data, and many other settings), and rephrase it as low rank matrix estim…