The existence of maximum likelihood estimate in high-dimensional binary response generalized linear models

Abstract

Motivated by recent works on the high-dimensional logistic regression, we establish that the existence of the maximum likelihood estimate exhibits a phase transition for a wide range of generalized linear models with binary outcome and elliptical covariates. This extends a previous result of Candès and Sur who proved the phase transition for the logistic regression with Gaussian covariates. Our result reveals a rich structure in the phase transition phenomenon, which is simply overlooked by Gaussianity. The main tools for deriving the result are data separation, convex geometry and stochastic approximation. We also conduct simulation studies to corroborate our theoretical findings, and explore other features of the problem.

arXiv comment23 pages, 7 figures, 1 table. This paper is published by https://projecteuclid.org/euclid.ejs/1604044928

References in corpus (4)

• A modern maximum-likelihood theory for high-dimensional logistic regression
• Maximum likelihood estimation in log-linear models
• The generalization error of max-margin linear classifiers: Benign overfitting and high dimensional asymptotics in the overparametrized regime
• Stochastic Tverberg theorems and their applications in multi-class logistic regression, data separability, and centerpoints of data