Optimal Subsampling for Large Sample Logistic Regression

Abstract

For massive data, the family of subsampling algorithms is popular to downsize the data volume and reduce computational burden. Existing studies focus on approximating the ordinary least squares estimate in linear regression, where statistical leverage scores are often used to define subsampling probabilities. In this paper, we propose fast subsampling algorithms to efficiently approximate the maximum likelihood estimate in logistic regression. We first establish consistency and asymptotic normality of the estimator from a general subsampling algorithm, and then derive optimal subsampling probabilities that minimize the asymptotic mean squared error of the resultant estimator. An alternative minimization criterion is also proposed to further reduce the computational cost. The optimal subsampling probabilities depend on the full data estimate, so we develop a two-step algorithm to approximate the optimal subsampling procedure. This algorithm is computationally efficient and has a significant reduction in computing time compared to the full data approach. Consistency and asymptotic normality of the estimator from a two-step algorithm are also established. Synthetic and real data sets are used to evaluate the practical performance of the proposed method.

Cited by in corpus (13)

• Information-Based Optimal Subdata Selection for Big Data Linear Regression
• Least Squares Approximation for a Distributed System
• Learning nonlocal constitutive models with neural networks
• BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees
• Divide-and-Conquer Information-Based Optimal Subdata Selection Algorithm
• A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques
• Optimal Cox Regression Subsampling Procedure with Rare Events
• Bayesian Estimation Under Informative Sampling with Unattenuated Dependence
• Optimal Sampling Designs for Multi-dimensional Streaming Time Series with Application to Power Grid Sensor Data
• Active sampling: A machine-learning-assisted framework for finite population inference with optimal subsamples
• Surprise sampling: improving and extending the local case-control sampling
• On the asymptotic properties of a bagging estimator with a massive dataset
• Subsampled One-Step Estimation for Fast Statistical Inference