Accelerating Cross-Validation in Multinomial Logistic Regression with -Regularization
arXiv:1711.05420
Abstract
We develop an approximate formula for evaluating a cross-validation estimator of predictive likelihood for multinomial logistic regression regularized by an -norm. This allows us to avoid repeated optimizations required for literally conducting cross-validation; hence, the computational time can be significantly reduced. The formula is derived through a perturbative approach employing the largeness of the data size and the model dimensionality. An extension to the elastic net regularization is also addressed. The usefulness of the approximate formula is demonstrated on simulated data and the ISOLET dataset from the UCI machine learning repository.
arXiv comment30 pages, 9 figures. MATLAB and python codes implementing the formula derived in the manuscript are distributed in https://github.com/T-Obuchi/AcceleratedCVonMLR_matlab and https://github.com/T-Obuchi/AcceleratedCVonMLR_python
References in corpus (5)
- Message Passing Algorithms for Compressed Sensing
- Tractable approximations for probabilistic models: The adaptive TAP mean field approach
- A scalable estimate of the extra-sample prediction error via approximate leave-one-out
- Approximate Leave-One-Out for Fast Parameter Tuning in High Dimensions
- Accelerating cross-validation with total variation and its application to super-resolution imaging