HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence Modeling

Hexiong Yang, Mingrui Chen, Huaibo Huang +4

Inspired by the great success of Masked Language Modeling (MLM) in the natural language domain, the paradigm of self-supervised pre-training and fine-tuning has also achieved remar…