5 papers
Understanding and Correcting Low-Frequency Bias in EEG Foundation Model
Junjie Yu, Zihan Deng, Jianyu Zhang +8
Increasing EEG pretraining data scale or model capacity does not consistently improve downstream performance. We identify a persistent low-frequency bias in representations learned…
Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation
Junjie Yu, Yue Wang, Zihan Deng +3
Finetuning pretrained models occurs in a low-dimensional subspace of the full parameter space. Prior work has focused on characterizing this optimization subspace, but largely igno…
Understanding Generalization from Embedding Dimension and Distributional Convergence
Junjie Yu, Zhuoli Ouyang, Haotian Deng +5
Deep neural networks often generalize well despite heavy over-parameterization, challenging classical parameter-based analyses. We study generalization from a representation-centri…
Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Brain
Junjie Yu, Wenxiao Ma, Chen Wei +4
Recent work has found that neural networks with stronger generalization tend to exhibit higher representational alignment with one another across architectures and training paradig…
Scale-Invariance Drives Convergence in AI and Brain Representations
Junjie Yu, Wenxiao Ma, Jianyu Zhang +4
Despite variations in architecture and pretraining strategies, recent studies indicate that large-scale AI models often converge toward similar internal representations that also a…