2 papers
cs.LG2026
Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models
Chen Liu, Xingzhi Sun, Xi Xiao +8
Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs. To better understand LLM scaling…
cs.LG2024
Exploring the Manifold of Neural Networks Using Diffusion Geometry
Elliott Abel, Andrew J. Steindl, Selma Mazioud +12
Drawing motivation from the manifold hypothesis, which posits that most high-dimensional data lies on or near low-dimensional manifolds, we apply manifold learning to the space of…