1 citations · 2 across the 5 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
Ranajoy Sadhukhan, Euijin Hong, Sheng Cao +9
Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains expensive and data-h…
cs.LG2026
STEM: Scaling Transformers with Embedding Modules
Ranajoy Sadhukhan, Sheng Cao, Harry Dong +5
Fine-grained sparsity promises higher parametric capacity without proportional per-token compute, but often suffers from training instability, load balancing, and communication ove…