Statistical Mechanical Development of a Sparse Bayesian Classifier

Abstract

The demand for extracting rules from high dimensional real world data is increasing in various fields. However, the possible redundancy of such data sometimes makes it difficult to obtain a good generalization ability for novel samples. To resolve this problem, we provide a scheme that reduces the effective dimensions of data by pruning redundant components for bicategorical classification based on the Bayesian framework. First, the potential of the proposed method is confirmed in ideal situations using the replica method. Unfortunately, performing the scheme exactly is computationally difficult. So, we next develop a tractable approximation algorithm, which turns out to offer nearly optimal performance in ideal cases when the system size is large. Finally, the efficacy of the developed classifier is experimentally examined for a real world problem of colon cancer classification, which shows that the developed method can be practically useful.

13 pages, 6 figures

References in corpus (2)

• The Nishimori line and Bayesian Statistics
• Propagating beliefs in spin glass models

Cited by in corpus (9)

• Perturbation Biology: inferring signaling networks in cellular systems
• Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
• Inference algorithms for gene networks: a statistical mechanics analysis
• Perceptron capacity revisited: classification ability for correlated patterns
• Learning of correlated patterns by simple perceptrons
• Gene-network inference by message passing
• Statistical mechanics of sparse generalization and model selection
• Classification and sparse-signature extraction from gene-expression data
• Expectation propagation on the diluted Bayesian classifier