Strong log-concavity in probit regression

Abstract

We show that strong log-concavity emerges in probit regression likelihoods without ridge penalization (i.e. Gaussian priors), unlike for the logistic case. Specifically, we provide: (a) a non-asymptotic characterization of strong log-concavity for fixed designs, similar to that for the existence of the maximum likelihood estimator (MLE) and (b) an asymptotic analysis for random designs, in the proportional regime when the sample size $n$ and the number of covariates $d$ grow proportionally $d/n\to r\in [0,1)$. In the latter case we show that, provided $r$ is small enough, the resulting condition number is finite and independent of $n,d,r$ with high-probability. Numerically tractable estimates are given in case $r=0$. Thus, probit regression provides a non-trivial example of high-dimensional, well-conditioned log-concave objective.