2 citations · 2 across the 10 of their papers we have counts for
5 papers · 1 filter
Mamba Modulation: On the Length Generalization of Mamba
Peng Lu, Jerry Huang, Qiuhao Zeng +4
The quadratic complexity of the attention mechanism in Transformer models has motivated the development of alternative architectures with sub-quadratic scaling, such as state-space…
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Maxime Heuillet, Yufei Cui, Boxing Chen +2
Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT). In standard ReFT framewor…
PoTPTQ: A Two-step Power-of-Two Post-training for LLMs
Xinyu Wang, Vahid Partovi Nia, Peng Lu +4
Large Language Models (LLMs) have demonstrated remarkable performance across various natural language processing (NLP) tasks. However, their deployment is challenging due to the su…
Resona: Improving Context Copying in Linear Recurrence Models with Retrieval
Xinyu Wang, Linrui Ma, Jerry Huang +5
Recent shifts in the space of large language model (LLM) research have shown an increasing focus on novel architectures to compete with prototypical Transformer-based models that h…
Rethinking Post-Training Quantization: Introducing a Statistical Pre-Calibration Approach
Alireza Ghaffari, Sharareh Younesian, Boxing Chen +2
As Large Language Models (LLMs) become increasingly computationally complex, developing efficient deployment strategies, such as quantization, becomes crucial. State-of-the-art Pos…