3 citations · 4 across the 4 of their papers we have counts for
5 papers
Kronecker Product Attention: Memory-Efficient KV Compression via Structured Factorization
Bahareh Nikpour, Hossein Rajabzadeh, Maryam Dialameh +2
Efficient autoregressive generation is increasingly limited by the memory footprint of the key-value (KV) cache, which grows linearly with sequence length, batch size, and model de…
Large Language Models with At Most One Spike per Neuron
Zhuoya Zhao, Parsa Omidi, Aref Jafari +1
Leveraging their inherent sparse event-driven computation, spiking neural networks (SNNs) offer a promising path toward energy-efficient large language models (LLMs). Time-to-first…
Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency
Xingshuai Huang, Derek Li, Bahareh Nikpour +1
Chain-of-Thought (CoT) prompting has significantly improved the reasoning capabilities of large language models (LLMs). However, conventional CoT often relies on unstructured, flat…
Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures
Parsa Omidi, Xingshuai Huang, Axel Laborieux +3
Memory is fundamental to intelligence, enabling learning, reasoning, and adaptability across biological and artificial systems. While Transformer architectures excel at sequence mo…
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
Xindi Wang, Mahsa Salmani, Parsa Omidi +3
Recently, large language models (LLMs) have shown remarkable capabilities including understanding context, engaging in logical reasoning, and generating responses. However, this is…