Enhancing Transformer Training Efficiency with Dynamic Dropout

Hanrui Yan, Dan Shao

We introduce Dynamic Dropout, a novel regularization technique designed to enhance the training efficiency of Transformer models by dynamically adjusting the dropout rate based on…