20 citations · 86 across the 19 of their papers we have counts for
6 papers · 1 filter
A Study on Transformer Configuration and Training Objective
Fuzhao Xue, Jianghai Chen, Aixin Sun +6
Transformer-based models have delivered impressive results on many tasks, particularly vision and language tasks. In many model training situations, conventional configurations are…
Reliable Label Correction is a Good Booster When Learning with Extremely Noisy Labels
Kai Wang, Xiangyu Peng, Shuo Yang +4
Learning with noisy labels has aroused much research interest since data annotations, especially for large-scale datasets, may be inevitably imperfect. Recent approaches resort to…
CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU
Zangwei Zheng, Pengtai Xu, Xuan Zou +12
The click-through rate (CTR) prediction task is to predict whether a user will click on the recommended item. As mind-boggling amounts of data are produced online daily, accelerati…
Crafting Better Contrastive Views for Siamese Representation Learning
Xiangyu Peng, Kai Wang, Zheng Zhu +2
Recent self-supervised contrastive learning methods greatly benefit from the Siamese structure that aims at minimizing distances between positive pairs. For high performance Siames…
Towards Efficient and Scalable Sharpness-Aware Minimization
Yong Liu, Siqi Mai, Xiangning Chen +2
Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training larg…
One Student Knows All Experts Know: From Sparse to Dense
Fuzhao Xue, Xiaoxin He, Xiaozhe Ren +2
Human education system trains one student by multiple experts. Mixture-of-experts (MoE) is a powerful sparse architecture including multiple experts. However, sparse MoE model is e…