Revocable Learned State via Process Sidecars

Abstract

Language models are often adapted in stages: a public skill phase, a private memory phase, and a later safety phase that learns to refuse outputs tied to the remembered entities. Revoking the memory after the safety phase is not the same problem as subtracting the memory update: the later safety optimizer has transported the memory direction. We introduce process sidecars, a two-coefficient edit family $\hat{\hat{I}}¸(\hat{I}»,{\hat{I}}^3)=\hat{I}{¸}_{\mathrm{AMS}}-\hat{I}»\hat{I}{”}_{\mathrm{M}}-{\hat{I}}^3{\hat{R}}_{\mathrm{S}\leftarrow \mathrm{M}}$, with ${\hat{R}}_{\mathrm{S}\leftarrow \mathrm{M}}={\hat{J}}_{\mathrm{S},\epsilon }(\hat{I}{”}_{\mathrm{M}})-\hat{I}{”}_{\mathrm{M}}$, where ${\hat{J}}_{\mathrm{S},\epsilon }$ is a centered secant through the realized future AdamW safety-training process. The implementation uses $\epsilon =1$ at the natural memory-edit scale; it reuses $\hat{I}{¸}_{\mathrm{AMS}}$ as the positive endpoint and computes one additional safety trace at $\hat{I}{¸}_{\mathrm{A}}-\hat{I}{”}_{\mathrm{M}}$. We prove two things. First, the exact sidecar, using the true transported direction $R_{\mathrm{S}\leftarrow \mathrm{M}}$ rather than the secant estimate, at $(\hat{I}»,{\hat{I}}^3)=(1,1)$ recovers the counterfactual safety-only oracle $\hat{I}{¸}_{\mathrm{AS}}$ up to second order; the proof treats AdamW as an augmented-state map over parameters, first moments, and second moments. Second, this process information is necessary: whenever future safety training bends the memory direction, every scalar task-arithmetic edit leaves first-order counterfactual error, while the process-sidecar edit is second-order accurate. Across three models, the validation-selected 2D edit improves held-out refusal closure over naive task arithmetic in all trials, and over the ${\hat{I}}^3=\hat{I}»$ process-JVP subfamily, the diagonal slice of the cached 2D grid, in all paired trials.

23 pages, 2 figures, 6 tables