Contextual Memory-Enhanced Source Coding for Low-SNR Communications

Abstract

Separate Source-Channel Coding (SSCC) remains vulnerable in noisy text transmission due to the fragility of autoregressive source decoding, especially when Arithmetic Coding (AC) relies on Large Language Model (LLM)-based probability estimation. This letter proposes a Memory-Augmented Source Coding (MASC) scheme that internalizes contextual patterns into a source model. Specifically, MASC employs a shared Parameterized Contextual Memory (PCM) for multi-order $n$-gram patterns, and a Mixture-of-Memory-Experts Router (MMER) for sparse, hidden-state-dependent routing over memory experts. This adaptive activation refines source probability estimation, shortens codelength, and mitigates decoding sensitivity to residual channel errors. Experiments over Rayleigh fading and AWGN channels demonstrate its effectiveness.