Transformers and the representation of biomedical background knowledge

Oskar Wysocki, Zili Zhou, Paul O'Regan +4

Specialised transformers-based models (such as BioBERT and BioMegatron) are adapted for the biomedical domain based on publicly available biomedical corpora. As such, they have the…