183 citations · 706 across the 60 of their papers we have counts for
22 papers · 1 filter
Non-Parametric Learning of Stochastic Differential Equations with Non-asymptotic Fast Rates of Convergence
Riccardo Bonalli, Alessandro Rudi
We propose a novel non-parametric learning paradigm for the identification of drift and diffusion coefficients of multi-dimensional non-linear stochastic differential equations, wh…
Active Labeling: Streaming Stochastic Gradients
Vivien Cabannes, Francis Bach, Vianney Perchet +1
The workhorse of machine learning is stochastic gradient descent. To access stochastic gradients, it is common to consider iteratively input/output pairs of a training dataset. Int…
Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares
Blake Woodworth, Francis Bach, Alessandro Rudi
We consider potentially non-convex optimization problems, for which optimal rates of approximation depend on the dimension of the parameter space and the smoothness of the function…
Mixability made efficient: Fast online multiclass logistic regression
Rémi Jézéquel, Pierre Gaillard, Alessandro Rudi
Mixability has been shown to be a powerful tool to obtain algorithms with optimal regret. However, the resulting methods often suffer from high computational complexity which has r…
A Note on Optimizing Distributions using Kernel Mean Embeddings
Boris Muzellec, Francis Bach, Alessandro Rudi
Kernel mean embeddings are a popular tool that consists in representing probability measures by their infinite-dimensional mean embeddings in a reproducing kernel Hilbert space. Wh…
PSD Representations for Effective Probability Models
Alessandro Rudi, Carlo Ciliberto
Finding a good way to model probability densities is key to probabilistic inference. An ideal model should be able to concisely approximate any probability while being also compati…
Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularization
Gaspard Beugnot, Julien Mairal, Alessandro Rudi
The theory of spectral filtering is a remarkable tool to understand the statistical properties of learning with kernels. For least squares, it allows to derive various regularizati…
On the Consistency of Max-Margin Losses
Alex Nowak-Vila, Alessandro Rudi, Francis Bach
The foundational concept of Max-Margin in machine learning is ill-posed for output spaces with more than two labels such as in structured prediction. In this paper, we show that th…
Disambiguation of weak supervision with exponential convergence rates
Vivien Cabannes, Francis Bach, Alessandro Rudi
Machine learning approached through supervised learning requires expensive annotation of data. This motivates weakly supervised learning, where data are annotated with incomplete y…
Consistent Structured Prediction with Max-Min Margin Markov Networks
Alex Nowak-Vila, Francis Bach, Alessandro Rudi
Max-margin methods for binary classification such as the support vector machine (SVM) have been extended to the structured prediction setting under the name of max-margin Markov ne…
Non-parametric Models for Non-negative Functions
Ulysse Marteau-Ferey, Francis Bach, Alessandro Rudi
Linear models have shown great effectiveness and flexibility in many fields such as machine learning, signal processing and statistics. They can represent rich spaces of functions…
Structured and Localized Image Restoration
Thomas Eboli, Alex Nowak-Vila, Jian Sun +3
We present a novel approach to image restoration that leverages ideas from localized structured prediction and non-linear multi-task learning. We optimize a penalized energy functi…
Kernel methods through the roof: handling billions of points efficiently
Giacomo Meanti, Luigi Carratino, Lorenzo Rosasco +1
Kernel methods provide an elegant and principled approach to nonparametric learning, but so far could hardly be used in large scale problems, since naïve implementations scale poor…
Efficient improper learning for online logistic regression
Rémi Jézéquel, Pierre Gaillard, Alessandro Rudi
We consider the setting of online logistic regression and consider the regret with respect to the 2-ball of radius B. It is known (see [Hazan et al., 2014]) that any proper algorit…
Structured Prediction with Partial Labelling through the Infimum Loss
Vivien Cabannes, Alessandro Rudi, Francis Bach
Annotating datasets is one of the main costs in nowadays supervised learning. The goal of weak supervision is to enable models to learn using only forms of labelling which are chea…
A General Theory for Structured Prediction with Smooth Convex Surrogates
Alex Nowak-Vila, Francis Bach, Alessandro Rudi
In this work we provide a theoretical framework for structured prediction that generalizes the existing theory of surrogate methods for binary and multiclass classification based o…
Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordance
Ulysse Marteau-Ferey, Dmitrii Ostrovskii, Francis Bach +1
We consider learning methods based on the regularization of a convex empirical risk by a squared Hilbertian norm, a setting that includes linear predictors and non-linear predictor…
Sharp Analysis of Learning with Discrete Losses
Alex Nowak-Vila, Francis Bach, Alessandro Rudi
The problem of devising learning strategies for discrete losses (e.g., multilabeling, ranking) is currently addressed with methods and theoretical analyses ad-hoc for each loss. In…
Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
We consider stochastic gradient descent (SGD) for least-squares regression with potentially several passes over the data. While several passes have been widely reported to perform…
Exponential convergence of testing error for stochastic gradient methods
Loucas Pillaud-Vivien, Alessandro Rudi, Francis Bach
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the ex…
Consistent Multitask Learning with Nonlinear Output Relations
Carlo Ciliberto, Alessandro Rudi, Lorenzo Rosasco +1
Key to multitask learning is exploiting relationships between different tasks to improve prediction performance. If the relations are linear, regularization approaches can be used…
A Consistent Regularization Approach for Structured Prediction
Carlo Ciliberto, Alessandro Rudi, Lorenzo Rosasco
We propose and analyze a regularization approach for structured prediction problems. We characterize a large class of loss functions that allows to naturally embed structured outpu…