L_1-regularized Boltzmann machine learning using majorizer minimization

Abstract

We propose an inference method to estimate sparse interactions and biases according to Boltzmann machine learning. The basis of this method is $L_1$ regularization, which is often used in compressed sensing, a technique for reconstructing sparse input signals from undersampled outputs. $L_1$ regularization impedes the simple application of the gradient method, which optimizes the cost function that leads to accurate estimations, owing to the cost function's lack of smoothness. In this study, we utilize the majorizer minimization method, which is a well-known technique implemented in optimization problems, to avoid the non-smoothness of the cost function. By using the majorizer minimization method, we elucidate essentially relevant biases and interactions from given data with seemingly strongly-correlated components.

16pages, 6 figures

References in corpus (10)

• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• Adaptive Cluster Expansion for Inferring Boltzmann Machines with Noisy Data
• Small-correlation expansions for the inverse Ising problem
• The Bethe approximation for solving the inverse Ising problem: a comparison with other inference methods
• Pseudolikelihood Decimation Algorithm Improving the Inference of the Interaction Network in a General Class of Ising Models
• Susceptibility Propagation by Using Diagonal Consistency
• A mean field method with correlations determined by linear response
• Detection of cheating by decimation algorithm
• Statistical-mechanical analysis of pre-training and fine tuning in deep learning
• Belief propagation with multipoint correlations and its application to Inverse problem

Cited by in corpus (5)

• Construction of Hamiltonians by supervised learning of energy and entanglement spectra
• Efficient Algorithm for Binary Quadratic Problem by Column Generation and Quantum Annealing
• Statistical-mechanical analysis of compressed sensing for Hamiltonian estimation of Ising spin glass
• Stochastic gradient method with accelerated stochastic dynamics
• Phase transition in binary compressed sensing based on $L_1$-norm minimization