2 citations · 2 across the 2 of their papers we have counts for
2 papers
cs.CL2021
Decoupled Transformer for Scalable Inference in Open-domain Question Answering
Haytham ElFadeel, Stan Peshterliev
Large transformer models, such as BERT, achieve state-of-the-art results in machine reading comprehension (MRC) for open-domain question answering (QA). However, transformers have…
cs.CL2021★ 2 cited
Robustly Optimized and Distilled Training for Natural Language Understanding
Haytham ElFadeel, Stan Peshterliev
In this paper, we explore multi-task learning (MTL) as a second pretraining step to learn enhanced universal language representation for transformer language models. We use the MTL…