2 papers
cs.LG2026
Correction of Decoupled Weight Decay
Jason Chuan-Chih Chou
Decoupled weight decay, solely responsible for the performance advantage of AdamW over Adam, has long been set to proportional to learning rate without questioning. Some resea…
cs.CL2026
ViT Registers and Fractal ViT
Jason Chuan-Chih Chou, Abhinav Kumar, Shivank Garg
Drawing inspiration from recent findings including surprisingly decent performance of transformers without positional encoding (NoPE) in the domain of language models and how regis…