Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection

Jianwei Li, Weizhi Gao, Qi Lei +1

It is widely acknowledged that large and sparse models have higher accuracy than small and dense models under the same model size constraints. This motivates us to train a large mo…