2 citations · 2 across the 7 of their papers we have counts for
6 papers · 1 filter
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
Ranajoy Sadhukhan, Euijin Hong, Sheng Cao +9
Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains expensive and data-h…
ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?
Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan +2
Large language model agents are increasingly deployed to perform complex tasks in real-world environments. However, the knowledge required for correct behavior in these environment…
Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
Yang Zhou, Ranajoy Sadhukhan, Zhaofeng Sun +7
Despite being powerful, reinforcement learning with verifiable rewards (RLVR) induces extremely long COT, making it computationally expensive. Since RLVR per-step cost is dominated…
STEM: Scaling Transformers with Embedding Modules
Ranajoy Sadhukhan, Sheng Cao, Harry Dong +5
Fine-grained sparsity promises higher parametric capacity without proportional per-token compute, but often suffers from training instability, load balancing, and communication ove…
Kinetics: Rethinking Test-Time Scaling Laws
Ranajoy Sadhukhan, Zhuoming Chen, Haizhong Zheng +3
We rethink test-time scaling laws from a practical efficiency perspective, revealing that the effectiveness of smaller models is significantly overestimated. Prior work, grounded i…
Memory Mosaics
Jianyu Zhang, Niklas Nolte, Ranajoy Sadhukhan +2
Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilitie…