1 citations · 1 across the 7 of their papers we have counts for
4 papers · 1 filter
From Tables to Signals: Revealing Spectral Adaptivity in TabPFN
Jianqiao Zheng, Cameron Gordon, Yiping Ji +2
Task-agnostic tabular foundation models such as TabPFN have achieved impressive performance on tabular learning tasks, yet the origins of their inductive biases remain poorly under…
Cutting the Skip: Training Residual-Free Transformers
Yiping Ji, James Martens, Jianqiao Zheng +5
Transformers have achieved remarkable success across a wide range of applications, a feat often attributed to their scalability. Yet training them without skip (residual) connectio…
SineLoRA: Sine-Activated Delta Compression
Cameron Gordon, Yiping Ji, Hemanth Saratchandran +2
Resource-constrained weight deployment is a task of immense practical importance. Recently, there has been interest in the specific task of \textit{Delta Compression}, where partie…
Always Skip Attention
Yiping Ji, Hemanth Saratchandran, Peyman Moghadam +1
We highlight a curious empirical result within modern Vision Transformers (ViTs). Specifically, self-attention catastrophically fails to train unless it is used in conjunction with…