Showing 2026 · cs.LGShow all
3 papers · 2 filters
cs.LG2026
Relevant and Irrelevant: A Renormalization Group Analysis of Transformer Attention
Parviz Haggi-Mani, Irina Rish
Using the language of Wilsonian renormalization group theory (RG), we treat the Transformer's attention mechanism as a perturbation of the trained MLP residual-stack fixed point an…
cs.LG2026
Rank Collapse, Fixed Points, and the Renormalization Group Structure of MLP Residual Networks
Parviz Haggi-Mani, Irina Rish
The analogy between deep neural network forward passes and renormalization group (RG) flows has been repeatedly noted in the literature, but existing treatments remain qualitative:…
cs.LG2026
Unified Neural Scaling Laws
Ethan Caballero, Priyank Jaini, David Krueger +1
We present a functional form (that we refer to as a Unified Neural Scaling Law (UNSL)) that accurately models and extrapolates the scaling behaviors of deep neural networks as mult…