9 citations · 23 across the 21 of their papers we have counts for
14 papers · 1 filter
Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning
Yuwen Hao, Menglin Yang
Chain-of-thought (CoT) reasoning improves the reasoning ability of large language models by introducing intermediate computation, but explicit rationales increase decoding length,…
FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space
Jiahong Liu, Ram Samarth B B, Xinyu Fu +4
Federated learning enables privacy-preserving collaborative training, but highly heterogeneous client data remain challenging, especially in graph federated learning where clients…
Hyperbolic Multimodal Continual Learning
Jiahong Liu, Ming Shen, Xiaohao Liu +4
Hyperbolic geometry has recently emerged as a powerful representation space for multimodal learning, as it naturally captures hierarchical semantic structure across modalities. Des…
Hierarchical Abstract Tree for Cross-Document Retrieval-Augmented Generation
Ziwen Zhao, Menglin Yang
Retrieval-augmented generation (RAG) enhances large language models with external knowledge, and tree-based RAG organizes documents into hierarchical indexes to support queries at…
Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
Buze Zhang, Jinkai Tao, Zilang Zeng +4
Large Language Models (LLMs) have achieved remarkable progress, with Parameter-Efficient Fine-Tuning (PEFT) emerging as a key technique for downstream task adaptation. However, exi…
Hyperbolic Deep Learning for Foundation Models: A Survey
Neil He, Hiren Madhu, Ngoc Bui +2
Foundation models pre-trained on massive datasets, including large language models (LLMs), vision-language models (VLMs), and large multimodal models, have demonstrated remarkable…
Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning
Ngoc Bui, Menglin Yang, Runjin Chen +5
Backward compatible representation learning enables updated models to integrate seamlessly with existing ones, avoiding to reprocess stored data. Despite recent advances, existing…
HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
Neil He, Rishabh Anand, Hiren Madhu +5
Large language models (LLMs) have shown great success in text modeling tasks across domains. However, natural language exhibits inherent semantic hierarchies and nuanced geometric…
HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules
Neil He, Menglin Yang, Rex Ying
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data across diverse modalities. Recent studies show that token distributions in foundation mode…
Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries
Neil He, Jiahong Liu, Buze Zhang +6
In the era of foundation models and Large Language Models (LLMs), Euclidean space has been the de facto geometric setting for machine learning architectures. However, recent litera…
Lorentzian Residual Neural Networks
Neil He, Menglin Yang, Rex Ying
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data structures prevalent in real-world datasets. Notably, residual connections, which facilita…
Low-Rank Adaptation for Foundation Models: A Comprehensive Review
Menglin Yang, Jialin Chen, Jinkai Tao +9
The rapid advancement of foundation modelslarge-scale neural networks trained on diverse, extensive datasetshas revolutionized artificial intelligence, enabling unprecedented advan…
Hyperbolic Fine-Tuning for Large Language Models
Menglin Yang, Ram Samarth B B, Aosong Feng +4
Large language models (LLMs) have demonstrated remarkable performance across various tasks. However, it remains an open question whether the default Euclidean space is the most sui…
Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space
Menglin Yang, Harshit Verma, Delvin Ce Zhang +3
Hyperbolic geometry have shown significant potential in modeling complex structured data, particularly those with underlying tree-like and hierarchical structures. Despite the impr…