Statistical mechanics of lossy compression using multilayer perceptrons
arXiv:cond-mat/0508598 · doi:10.1103/PhysRevE.74.026108
Abstract
Statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased Boolean messages. We utilize a tree-like committee machine (committee tree) and tree-like parity machine (parity tree) whose transfer functions are monotonic. For compression using committee tree, a lower bound of achievable distortion becomes small as the number of hidden units K increases. However, it cannot reach the Shannon bound even where K -> infty. For a compression using a parity tree with K >= 2 hidden units, the rate distortion function, which is known as the theoretical limit for compression, is derived where the code length becomes infinity.
12 pages, 5 figures
References in corpus (6)
- Statistical mechanics of image restoration and error-correcting codes
- Typical Performance of Gallager-type Error-Correcting Codes
- The statistical mechanics of turbo codes
- Thouless-Anderson-Palmer Approach for Lossy Compression
- Statistical mechanics of lossy data compression using a non-monotonic perceptron
- One step RSB scheme for the rate distortion function
Cited by in corpus (8)
- Perceptron capacity revisited: classification ability for correlated patterns
- Learning of correlated patterns by simple perceptrons
- Statistical mechanics of lossy compression for non-monotonic multilayer perceptrons
- Typical Performance of Irregular Low-Density Generator-Matrix Codes for Lossy Compression
- Linear Complexity Lossy Compressor for Binary Redundant Memoryless Sources
- Belief Propagation for Error Correcting Codes and Lossy Compression Using Multilayer Perceptrons
- Parallel dynamics of continuous Hopfield model revisited
- Error correcting code using tree-like multilayer perceptron