Statistical mechanics of error exponents for error-correcting codes

Abstract

Error exponents characterize the exponential decay, when increasing message length, of the probability of error of many error-correcting codes. To tackle the long standing problem of computing them exactly, we introduce a general, thermodynamic, formalism that we illustrate with maximum-likelihood decoding of low-density parity-check (LDPC) codes on the binary erasure channel (BEC) and the binary symmetric channel (BSC). In this formalism, we apply the cavity method for large deviations to derive expressions for both the average and typical error exponents, which differ by the procedure used to select the codes from specified ensembles. When decreasing the noise intensity, we find that two phase transitions take place, at two different levels: a glass to ferromagnetic transition in the space of codewords, and a paramagnetic to glass transition in the space of codes.

32 pages, 13 figures

References in corpus (2)

• Exactly solvable models of adaptive networks
• The polynomial error probability for LDPC codes

Cited by in corpus (4)

• Gallager error correcting codes for binary asymmetric channels
• The random energy model in a magnetic field and joint source-channel coding
• Relations between random coding exponents and the statistical physics of random codes
• An identity of Chernoff bounds with an interpretation in statistical physics and applications in information theory