29 citations · 82 across the 29 of their papers we have counts for
4 papers · 1 filter
Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources
Ji Liu, Daxiang Dong, Xi Wang +5
Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which ex…
Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization
Hang Hua, Xingjian Li, Dejing Dou +2
The advent of large-scale pre-trained language models has contributed greatly to the recent progress in natural language processing. Many state-of-the-art language models are first…
Deep Active Learning with Noise Stability
Xingjian Li, Pengkun Yang, Yangcheng Gu +4
Uncertainty estimation for unlabeled data is crucial to active learning. With a deep neural network employed as the backbone model, the data selection process is highly challenging…
Towards Inadequately Pre-trained Models in Transfer Learning
Andong Deng, Xingjian Li, Di Hu +3
Pre-training has been a popular learning paradigm in deep learning era, especially in annotation-insufficient scenario. Better ImageNet pre-trained models have been demonstrated, f…