6 papers
Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models
John Sweeney
Gradient updates on different data generally do not commute: training a language model on two data sources in opposite orders gives different weights, even with the same data and t…
Signed-Permutation Coordinate Transport for RMSNorm Transformers
John Sweeney
Modern LLM workflows move coordinate-indexed objects across checkpoints: steering vectors, sparse autoencoders, top- neuron sets, attribution lists, and merge alignments. This i…
Revocable Learned State via Process Sidecars
John Sweeney
Language models are often adapted in stages: a public skill phase, a private memory phase, and a later safety phase that learns to refuse outputs tied to the remembered entities. R…
Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise
John Sweeney
Shuffle order can be a larger source of fine-tuning noise than a memoryless analysis predicts: fixed-clock optimizer memory makes local equal-multiset contrasts first order in the…
The Geometry of Updates: Fisher Alignment at Vocabulary Scale
John Sweeney
Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora…
The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order
John Sweeney
Sequential learning is order-dependent: from Pile-style next-token domain adaptation to instruction-SFT and DPO, N candidate sources induce N! possible curricula. We show that the…