2 citations · 2 across the 7 of their papers we have counts for
3 papers · 1 filter
On-Policy Attention Linearization
Arian Raje, Anupam Nayak, Anthony Fei +3
Hybrid transformer architectures that replace most softmax attention layers with linear attention offer transformer-level quality at a fraction of the memory cost. Rather than pret…
PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
Anupam Nayak, Baris Askin, Muhammed Ustaomeroglu +2
Reasoning post-training with reinforcement learning from verifiable rewards (RLVR) is typically studied in centralized settings, yet many realistic applications involve decentraliz…
MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
Arian Raje, Anupam Nayak, Gauri Joshi
Mixture-of-Experts (MoE) model architectures can significantly reduce the number of activated parameters per token, enabling computationally efficient training and inference. Howev…