11 citations · 22 across the 3 of their papers we have counts for
5 papers
Variance-reduced Language Pretraining via a Mask Proposal Network
Liang Chen
Self-supervised learning, a.k.a., pretraining, is important in natural language processing. Most of the pretraining methods first randomly mask some positions in a sentence and the…
MC-BERT: Efficient Language Pre-Training via a Meta Controller
Zhenhui Xu, Linyuan Gong, Guolin Ke +5
Pre-trained contextual representations (e.g., BERT) have become the foundation to achieve state-of-the-art results on many NLP tasks. However, large-scale pre-training is computati…
Invertible Image Rescaling
Mingqing Xiao, Shuxin Zheng, Chang Liu +6
High-resolution digital images are usually downscaled to fit various display screens or save the cost of storage and bandwidth, meanwhile the post-upscaling is adpoted to recover t…
LightMC: A Dynamic and Efficient Multiclass Decomposition Algorithm
Ziyu Liu, Guolin Ke, Jiang Bian +1
Multiclass decomposition splits a multiclass classification problem into a series of independent binary learners and recomposes them by combining their outputs to reconstruct the m…
Light Multi-segment Activation for Model Compression
Zhenhui Xu, Guolin Ke, Jia Zhang +2
Model compression has become necessary when applying neural networks (NN) into many real application tasks that can accept slightly-reduced model accuracy with strict tolerance to…