9 citations · 16 across the 12 of their papers we have counts for
11 papers · 1 filter
Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
Ngoc Bui, Shubham Sharma, Simran Lamba +2
Memory and computation remain core bottlenecks in long-horizon LLM inference due to the quadratic cost of self-attention and the ever-growing key-value (KV) cache. Existing strateg…
Implicit Reasoning in Large Language Models: A Comprehensive Survey
Jindong Li, Yali Fu, Li Fan +6
Large Language Models (LLMs) have demonstrated strong generalization across a wide range of tasks. Reasoning with LLMs is central to solving multi-step problems and complex decisio…
Hyperbolic Deep Learning for Foundation Models: A Survey
Neil He, Hiren Madhu, Ngoc Bui +2
Foundation models pre-trained on massive datasets, including large language models (LLMs), vision-language models (VLMs), and large multimodal models, have demonstrated remarkable…
Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning
Ngoc Bui, Menglin Yang, Runjin Chen +5
Backward compatible representation learning enables updated models to integrate seamlessly with existing ones, avoiding to reprocess stored data. Despite recent advances, existing…
Towards Non-Euclidean Foundation Models: Advancing AI Beyond Euclidean Frameworks
Menglin Yang, Yifei Zhang, Jialin Chen +2
In the era of foundation models and Large Language Models (LLMs), Euclidean space is the de facto geometric setting of our machine learning architectures. However, recent literatur…
DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learning
Borui Wang, Kathleen McKeown, Rex Ying
Reinforcement learning from expert demonstrations has long remained a challenging research problem, and existing state-of-the-art methods using behavioral cloning plus further RL t…
HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
Neil He, Rishabh Anand, Hiren Madhu +5
Large language models (LLMs) have shown great success in text modeling tasks across domains. However, natural language exhibits inherent semantic hierarchies and nuanced geometric…
HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules
Neil He, Menglin Yang, Rex Ying
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data across diverse modalities. Recent studies show that token distributions in foundation mode…
Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries
Neil He, Jiahong Liu, Buze Zhang +6
In the era of foundation models and Large Language Models (LLMs), Euclidean space has been the de facto geometric setting for machine learning architectures. However, recent litera…
Lorentzian Residual Neural Networks
Neil He, Menglin Yang, Rex Ying
Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data structures prevalent in real-world datasets. Notably, residual connections, which facilita…
Low-Rank Adaptation for Foundation Models: A Comprehensive Review
Menglin Yang, Jialin Chen, Jinkai Tao +9
The rapid advancement of foundation modelslarge-scale neural networks trained on diverse, extensive datasetshas revolutionized artificial intelligence, enabling unprecedented advan…