495 citations · 521 across the 8 of their papers we have counts for
4 papers · 1 filter
Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA
Wenzhi Fang, Nicholas Tzou, Lazar Valkov +1
Reinforcement learning (RL)-based post-training has become an effective approach for eliciting reasoning capabilities in large language models (LLMs). However, adapting post-RL mod…
Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks
Avinash Amballa, Yashas Malur Saidutta, Wenbo Li +2
Merging low-rank adapters (LoRAs) promises to eliminate the overhead of swapping task-specific weights at inference time. However, existing merging methods assume every layer needs…
Doc-to-Atom: Learning to Compile and Compose Memory Atoms
Xingjian Diao, Wenbo Li, Yashas Malur Saidutta +3
Long input sequences are central to document understanding and multi-step reasoning in Large Language Models, yet the quadratic cost of attention makes inference both memory-intens…
Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models
Juhong Min, Lazar Valkov, Vitali Petsiuk +2
Vision-language models benefit from high-resolution images, but the increase in visual-token count incurs high compute overhead. Humans resolve this tension via foveation: a coarse…