443 citations · 544 across the 17 of their papers we have counts for
5 papers · 1 filter
Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application
Yulan Hu, Yong Liu
Benefiting from the strong ability of the pre-trained model, the research on Chinese Word Segmentation (CWS) has made great progress in recent years. However, due to massive comput…
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
Kaichao You, Yong Liu, Ziyang Zhang +3
Model hubs with many pre-trained models (PTMs) have become a cornerstone of deep learning. Although built at a high cost, they remain \emph{under-exploited} -- practitioners usuall…
Go Wider Instead of Deeper
Fuzhao Xue, Ziji Shi, Futao Wei +3
More transformer blocks with residual connections have recently achieved impressive results on various tasks. To achieve better performance with fewer trainable parameters, recent…
Concurrent Adversarial Learning for Large-Batch Training
Yong Liu, Xiangning Chen, Minhao Cheng +2
Large-batch training has become a commonly used technique when training neural networks with a large number of GPU/TPU processors. As batch size increases, stochastic optimizers te…
LogME: Practical Assessment of Pre-trained Models for Transfer Learning
Kaichao You, Yong Liu, Jianmin Wang +1
This paper studies task adaptive pre-trained model selection, an underexplored problem of assessing pre-trained models for the target task and select best ones from the model zoo \…