3 papers
cs.LG2026
Correction of Decoupled Weight Decay
Jason Chuan-Chih Chou
Decoupled weight decay, solely responsible for the performance advantage of AdamW over Adam, has long been set to proportional to learning rate without questioning. Some resea…
cs.CL2026
ViT Registers and Fractal ViT
Jason Chuan-Chih Chou, Abhinav Kumar, Shivank Garg
Drawing inspiration from recent findings including surprisingly decent performance of transformers without positional encoding (NoPE) in the domain of language models and how regis…
cs.LG2024
Embedding Geometries of Contrastive Language-Image Pre-Training
Jason Chuan-Chih Chou, Nahid Alam
Since the publication of CLIP, the approach of using InfoNCE loss for contrastive pre-training has become widely popular for bridging two or more modalities. Despite its wide adopt…