GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning

Soumyadeep Roy, Jonas Wallat, Sowmya S Sundaram +2

Large-scale language models such as DNABert and LOGO aim to learn optimal gene representations and are trained on the entire Human Reference Genome. However, standard tokenization…