6 papers
Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
Buze Zhang, Jinkai Tao, Zilang Zeng +4
Large Language Models (LLMs) have achieved remarkable progress, with Parameter-Efficient Fine-Tuning (PEFT) emerging as a key technique for downstream task adaptation. However, exi…
Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries
Neil He, Jiahong Liu, Buze Zhang +6
In the era of foundation models and Large Language Models (LLMs), Euclidean space has been the de facto geometric setting for machine learning architectures. However, recent litera…
HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
Neil He, Rishabh Anand, Hiren Madhu +5
Large language models (LLMs) have shown great success in text modeling tasks across domains. However, natural language exhibits inherent semantic hierarchies and nuanced geometric…
Hyperbolic Deep Learning for Foundation Models: A Survey
Neil He, Hiren Madhu, Ngoc Bui +2
Foundation models pre-trained on massive datasets, including large language models (LLMs), vision-language models (VLMs), and large multimodal models, have demonstrated remarkable…
HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules
Neil He, Menglin Yang, Rex Ying
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data across diverse modalities. Recent studies show that token distributions in foundation mode…
Lorentzian Residual Neural Networks
Neil He, Menglin Yang, Rex Ying
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data structures prevalent in real-world datasets. Notably, residual connections, which facilita…