1 citations · 3 across the 4 of their papers we have counts for
4 papers
Gene42: Long-Range Genomic Foundation Model With Dense Attention
Kirill Vishniakov, Boulbaba Ben Amor, Engin Tekin +12
We introduce Gene42, a novel family of Genomic Foundation Models (GFMs) designed to manage context lengths of up to 192,000 base pairs (bp) at a single-nucleotide resolution. Gene4…
Initializing Models with Larger Ones
Zhiqiu Xu, Yanjie Chen, Kirill Vishniakov +5
Weight initialization plays an important role in neural network training. Widely used initialization methods are proposed and evaluated for networks that are trained from scratch.…
ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy
Kirill Vishniakov, Zhiqiang Shen, Zhuang Liu
Modern computer vision offers a great variety of models to practitioners, and selecting a model from multiple options for specific applications can be challenging. Conventionally,…
MixMask: Revisiting Masking Strategy for Siamese ConvNets
Kirill Vishniakov, Eric Xing, Zhiqiang Shen
The recent progress in self-supervised learning has successfully combined Masked Image Modeling (MIM) with Siamese Networks, harnessing the strengths of both methodologies. Nonethe…