1 citations · 2 across the 14 of their papers we have counts for
5 papers · 1 filter
Grokking as a Falsifiable Finite-Size Transition
Yuda Bi, Chenyu Zhang, Qiheng Wang +1
Grokking -- the delayed onset of generalization after early memorization -- is often described with phase-transition language, but that claim has lacked falsifiable finite-size inp…
Reservoir Subspace Injection for Online ICA under Top-n Whitening
Wenjun Xiao, Yuda Bi, Vince D Calhoun
Reservoir expansion can improve online independent component analysis (ICA) under nonlinear mixing, yet top- whitening may discard injected features. We formalize this bottlenec…
A 1/R Law for Kurtosis Contrast in Balanced Mixtures
Yuda Bi, Wenjun Xiao, Linhao Bai +1
Kurtosis-based Independent Component Analysis (ICA) weakens in wide, balanced mixtures. We prove a sharp redundancy law: for a standardized projection with effective width $R_{\mat…
Redundancy as a Structural Information Principle for Learning and Generalization
Yuda Bi, Ying Zhu, Vince D Calhoun
We present a theoretical framework that extends classical information theory to finite and structured systems by redefining redundancy as a fundamental property of information orga…
Scaling Laws are Redundancy Laws
Yuda Bi, Vince D Calhoun
Scaling laws, a defining feature of deep learning, reveal a striking power-law improvement in model performance with increasing dataset and model size. Yet, their mathematical orig…