SpanDrop: Simple and Effective Counterfactual Learning for Long Sequences

Peng Qi, Guangtao Wang, Jing Huang

Distilling supervision signal from a long sequence to make predictions is a challenging task in machine learning, especially when not all elements in the input sequence contribute…