3 papers
cs.LG2026
Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models
Mingze Wang, Shuchen Zhu, Yuxin Fang +3
Normalization layers in modern large language models (LLMs) consist of a deterministic normalization operation and a learnable scale vector. While the normalization operation has b…
cs.LG2026
Lightweight Gaussian Process Inference in C++ on Metal and CUDA
Yu-Hsueh Fang
Gaussian process (GP) inference in Python is dominated by libraries such as GPyTorch and GPflow, which are built on deep-learning frameworks and inherit their dispatch overhead and…
cs.CL2026
Mixture-of-Depths Attention
Lianghui Zhu, Yuxin Fang, Bencheng Liao +10
Scaling depth is a key driver for large language models (LLMs). Yet, as LLMs become deeper, they often suffer from signal degradation: informative features formed in shallow layers…