Towards preserving word order importance through Forced Invalidation

Hadeel Al-Negheimish, Pranava Madhyastha, Alessandra Russo

Large pre-trained language models such as BERT have been widely used as a framework for natural language understanding (NLU) tasks. However, recent findings have revealed that pre-…