Structure Learning in Inverse Ising Problems Using -Regularized Linear Estimator
arXiv:2008.08342 · doi:10.1088/1742-5468/abfa10
Abstract
The inference performance of the pseudolikelihood method is discussed in the framework of the inverse Ising problem when the -regularized (ridge) linear regression is adopted. This setup is introduced for theoretically investigating the situation where the data generation model is different from the inference one, namely the model mismatch situation. In the teacher-student scenario under the assumption that the teacher couplings are sparse, the analysis is conducted using the replica and cavity methods, with a special focus on whether the presence/absence of teacher couplings is correctly inferred or not. The result indicates that despite the model mismatch, one can perfectly identify the network structure using naive linear regression without regularization when the number of spins is smaller than the dataset size , in the thermodynamic limit . Further, to access the underdetermined region , we examine the effect of the regularization, and find that biases appear in all the coupling estimates, preventing the perfect identification of the network structure. We, however, find that the biases are shown to decay exponentially fast as the distance from the center spin chosen in the pseudolikelihood method grows. Based on this finding, we propose a two-stage estimator: In the first stage, the ridge regression is used and the estimates are pruned by a relatively small threshold; in the second stage the naive linear regression is conducted only on the remaining couplings, and the resultant estimates are again pruned by another relatively large threshold. This estimator with the appropriate regularization coefficient and thresholds is shown to achieve the perfect identification of the network structure even in . Results of extensive numerical experiments support these findings.
arXiv comment35 pages, 8 figures
References in corpus (14)
- High-dimensional Ising model selection using -regularized logistic regression
- Inverse statistical problems: from the inverse Ising problem to data science
- Ising models on locally tree-like graphs
- Inverse Ising inference using all the data
- Small-correlation expansions for the inverse Ising problem
- A single defect approximation for localized states on random lattices
- The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
- Pseudolikelihood Decimation Algorithm Improving the Inference of the Interaction Network in a General Class of Ising Models
- Bethe-Peierls approximation and the inverse Ising model
- Consistent inference of a general model using the pseudo-likelihood method
- A statistical physics approach to learning curves for the Inverse Ising problem
- Learning of couplings for random asymmetric kinetic Ising models revisited: random correlation matrices and learning curves
- Statistical mechanics of the inverse Ising problem and the optimal objective function
- Learning performance in inverse Ising problems with sparse teacher couplings