Learning Using Privileged Information: SVM+ and Weighted SVM
arXiv:1306.3161 · doi:10.1016/j.neunet.2014.02.002
Abstract
Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training. The same goal is pursued within the learning using privileged information paradigm which was recently introduced by Vapnik et al. and is aimed at utilizing additional information available only at training time -- a framework implemented by SVM+. We relate the privileged information to importance weighting and show that the prior knowledge expressible with privileged features can also be encoded by weights associated with every training example. We show that a weighted SVM can always replicate an SVM+ solution, while the converse is not true and we construct a counterexample highlighting the limitations of SVM+. Finally, we touch on the problem of choosing weights for weighted SVMs when privileged features are not available.
arXiv comment18 pages, 8 figures; integrated reviewer comments, improved typesetting
Cited by in corpus (21)
- Few-Shot Adversarial Domain Adaptation
- Unifying distillation and privileged information
- A New Learning Paradigm for Random Vector Functional-Link Network: RVFL+
- SPIGAN: Privileged Adversarial Learning from Simulation
- Mind the Nuisance: Gaussian Process Classification using Privileged Noise
- Learning to Transfer Privileged Information
- Weakly Supervised Object Localization Using Size Estimates
- Simple and Efficient Learning using Privileged Information
- Improving Semantic Analysis on Point Clouds via Auxiliary Supervision of Local Geometric Priors
- Training Group Orthogonal Neural Networks with Privileged Information
- Transfer Hashing with Privileged Information
- Sequential Minimal Optimization Algorithm for One-Class Support Vector Machines With Privileged Information
- Graph Distillation for Action Detection with Privileged Modalities
- OCKELM+: Kernel Extreme Learning Machine based One-class Classification using Privileged Information (or KOC+: Kernel Ridge Regression or Least Square SVM with zero bias based One-class Classification using Privileged Information)
- VisualBackProp for learning using privileged information with CNNs
- A Selective Survey on Versatile Knowledge Distillation Paradigm for Neural Network Models
- Adaptively Denoising Proposal Collection for Weakly Supervised Object Localization
- Ordering as privileged information
- Sidekick Policy Learning for Active Visual Exploration
- Span error bound for weighted SVM with applications in hyperparameter selection
- Efficient Learning of Pinball TWSVM using Privileged Information and its applications