PAC-Bayesian bounds for the Gram matrix and least squares regression with a random design

Abstract

The topics dicussed in this paper take their origin inthe estimation of the Gram matrix of a random vector from a sample made of n independent copies. They comprise the estimation of the covariance matrix and the study of least squares regression with a random design. We propose four types of results, based on non-asymptotic PAC-Bayesian generalization bounds: a new robust estimator of the Gram matrix and of the covariance matrix, new results on the empirical Gram matrix, new robust least squares estimators and new results on the ordinary least squares estimator, including its exact rate of convergence under polynomial moment assumptions.

81 pages

References in corpus (1)

• PAC-Bayesian Bounds for Randomized Empirical Risk Minimizers

Cited by in corpus (9)

• A Primer on PAC-Bayesian Learning
• Simpler PAC-Bayesian Bounds for Hostile Data
• Estimation of the covariance structure of heavy-tailed distributions
• Distribution-Free Robust Linear Regression
• Robust Modifications of U-statistics and Applications to Covariance Estimation Problems
• Convergence rates of least squares regression estimators with heavy-tailed errors
• On the estimation of the mean of a random vector
• Robust covariance estimation for distributed principal component analysis
• II. High Dimensional Estimation under Weak Moment Assumptions: Structured Recovery and Matrix Estimation