Why pre-training is beneficial for downstream classification tasks?

Xin Jiang, Xu Cheng, Zechao Li

Pre-training has exhibited notable benefits to downstream tasks by boosting accuracy and speeding up convergence, but the exact reasons for these benefits still remain unclear. To…