35 citations · 50 across the 9 of their papers we have counts for
3 papers · 1 filter
GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning
Soumyadeep Roy, Jonas Wallat, Sowmya S Sundaram +2
Large-scale language models such as DNABert and LOGO aim to learn optimal gene representations and are trained on the entire Human Reference Genome. However, standard tokenization…
The Effect of Masking Strategies on Knowledge Retention by Language Models
Jonas Wallat, Tianyi Zhang, Avishek Anand
Language models retain a significant amount of world knowledge from their pre-training stage. This allows knowledgeable models to be applied to knowledge-intensive tasks prevalent…
A Review of the Role of Causality in Developing Trustworthy AI Systems
Niloy Ganguly, Dren Fazlija, Maryam Badar +9
State-of-the-art AI models largely lack an understanding of the cause-effect relationship that governs human understanding of the real world. Consequently, these models do not gene…