Memorization for Good: Encryption with Autoregressive Language Models

Samuel Stevens, Yu Su

Over-parameterized neural language models (LMs) can memorize and recite long sequences of training data. While such memorization is normally associated with undesired properties su…