Network Cross-Validation for Determining the Number of Communities in Network Data
arXiv:1411.1715
Abstract
The stochastic block model and its variants have been a popular tool in analyzing large network data with community structures. In this paper we develop an efficient network cross-validation (NCV) approach to determine the number of communities, as well as to choose between the regular stochastic block model and the degree corrected block model. The proposed NCV method is based on a block-wise node-pair splitting technique, combined with an integrated step of community recovery using sub-blocks of the adjacency matrix. We prove that the probability of under selection vanishes as the number of node increases, under mild conditions satisfied by a wide range of popular community recovery algorithms. The solid performance of our method is also demonstrated in extensive simulations and a data example.
arXiv comment27 pages. Added further theoretical results and more references
References in corpus (13)
- Finding and evaluating community structure in networks
- Modularity and community structure in networks
- Stochastic blockmodels and community structure in networks
- Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications
- Spectral redemption: clustering sparse networks
- Fast community detection by SCORE
- Modeling homophily and stochastic equivalence in symmetric relational data
- Community extraction for social networks
- Model Selection for Degree-corrected Block Models
- Hypergeometric tail inequalities: ending the insanity
- A Generic Sample Splitting Approach for Refined Community Recovery in Stochastic Block Models
- Community detection thresholds and the weak Ramanujan property
- How Many Communities Are There?