162 citations · 163 across the 5 of their papers we have counts for
5 papers
FlashBoot: Sub-Second Weight Loading for Large Models at Rack Scale
Issac Zhu, Hscos Zhang, Keith Jiang +3
Flagship Mixture-of-Experts (MoE) models are growing fast along two axes at once: total parameter count and the number of experts. In elastic deployment scenarios, many GPUs across…
SparseRL-Sync: Lossless Weight Synchronization with ~100x Less Communication
Lucas Hu, Ranchi Zhao, Isaac Zhu +4
In large-scale reinforcement learning (RL) systems with decoupled Trainer-Rollout execution, the Trainer must regularly synchronize policy weights to the Rollout side to limit poli…
Dissecting Lottery Ticket Transformers: Structural and Behavioral Study of Sparse Neural Machine Translation
Rajiv Movva, Jason Y. Zhao
Recent work on the lottery ticket hypothesis has produced highly sparse Transformers for NMT while maintaining BLEU. However, it is unclear how such pruning techniques affect a mod…
Recurrent Control Nets for Deep Reinforcement Learning
Vincent Liu, Ademi Adeniji, Nathaniel Lee +2
Central Pattern Generators (CPGs) are biological neural circuits capable of producing coordinated rhythmic outputs in the absence of rhythmic input. As a result, they are responsib…
SlowFuzz: Automated Domain-Independent Detection of Algorithmic Complexity Vulnerabilities
Theofilos Petsios, Jason Zhao, Angelos D. Keromytis +1
Algorithmic complexity vulnerabilities occur when the worst-case time/space complexity of an application is significantly higher than the respective average case for particular use…