189 citations · 268 across the 8 of their papers we have counts for
4 papers · 1 filter
Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models
Daochen Zha, Louis Feng, Liang Luo +8
Sharding a large machine learning model across multiple devices to balance the costs is important in distributed training. This is challenging because partitioning is NP-hard, and…
Self-discipline on multiple channels
Jiutian Zhao, Liang Luo, Hao Wang
Self-distillation relies on its own information to improve the generalization ability of the model and has a bright future. Existing self-distillation methods either require additi…
Large-Scale Generative Data-Free Distillation
Liangchen Luo, Mark Sandler, Zi Lin +2
Knowledge distillation is one of the most popular and effective techniques for knowledge transfer, model compression and semi-supervised learning. Most existing distillation approa…
Adaptive Gradient Methods with Dynamic Bound of Learning Rate
Liangchen Luo, Yuanhao Xiong, Yan Liu +1
Adaptive optimization methods such as AdaGrad, RMSprop and Adam have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates. Though p…