A PAC-Bayesian Perspective on Structured Prediction with Implicit Loss Embeddings
arXiv:2012.03780
Abstract
Many practical machine learning tasks can be framed as Structured prediction problems, where several output variables are predicted and considered interdependent. Recent theoretical advances in structured prediction have focused on obtaining fast rates convergence guarantees, especially in the Implicit Loss Embedding (ILE) framework. PAC-Bayes has gained interest recently for its capacity of producing tight risk bounds for predictor distributions. This work proposes a novel PAC-Bayes perspective on the ILE Structured prediction framework. We present two generalization bounds, on the risk and excess risk, which yield insights into the behavior of ILE predictors. Two learning algorithms are derived from these bounds. The algorithms are implemented and their behavior analyzed, with source code available at \url{https://github.com/theophilec/PAC-Bayes-ILE-Structured-Prediction}.
38 pages
References in corpus (9)
- Scikit-learn: Machine Learning in Python
- A scikit-based Python environment for performing multi-label classification
- Variational Bayesian Inference with Stochastic Search
- Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
- Differentiable Dynamic Programming for Structured Prediction and Attention
- PAC-Bayesian Bound for the Conditional Value at Risk
- On Structured Prediction Theory with Calibrated Convex Surrogate Losses
- Sharp Analysis of Learning with Discrete Losses
- Consistent Structured Prediction with Max-Min Margin Markov Networks