Spontaneous Context Restoration: How Language Models Recover from Corrupted Inputs

Pranjal Garg, Jacob Beck

Language models sometimes produce correct outputs even when their inputs are corrupted by deletion, replacement, or misspelling. We study the internal processes accompanying this b…