Modern Subsampling Methods for Large-Scale Least Squares Regression
arXiv:2105.01552
Abstract
Subsampling methods aim to select a subsample as a surrogate for the observed sample. As a powerful technique for large-scale data analysis, various subsampling methods are developed for more effective coefficient estimation and model prediction. This review presents some cutting-edge subsampling methods based on the large-scale least squares estimation. Two major families of subsampling methods are introduced, respectively, the randomized subsampling approach and the optimal subsampling approach. The former aims to develop a more effective data-dependent sampling probability, while the latter aims to select a deterministic subsample in accordance with certain optimality criteria. Real data examples are provided to compare these methods empirically, respecting both the estimation accuracy and the computing time.
References in corpus (13)
- Prioritized Experience Replay
- GraphLab: A New Framework For Parallel Machine Learning
- Online Batch Selection for Faster Training of Neural Networks
- Improving CUR Matrix Decomposition and the Nyström Approximation via Adaptive Sampling
- Not All Samples Are Created Equal: Deep Learning with Importance Sampling
- Variance Reduction in SGD by Distributed Importance Sampling
- Scalable Kernel K-Means Clustering with Nystrom Approximation: Relative-Error Bounds
- Leveraged volume sampling for linear regression
- Asymptotic Analysis of Sampling Estimators for Randomized Numerical Linear Algebra Algorithms
- On Computationally Tractable Selection of Experiments in Measurement-Constrained Regression Models
- Unbiased estimates for linear regression via volume sampling
- Minimax experimental design: Bridging the gap between statistical and worst-case approaches to least squares regression
- An Asympirical Smoothing Parameters Selection Approach for Smoothing Spline ANOVA Models in Large Samples