5 papers
VANDAM: Viewing a nucleotide sequence with DNA molecular priors
Jeremy Levy, Ariel Larey, Yury Nahshan +18
Contemporary Genomic Foundation Models (GFMs) rely on a DNA-as-a-string paradigm that employs masked token prediction objectives for pretraining. However, this abstraction does not…
Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models
Yury Nahshan, Nati Daniel, Jacob Goldberger +1
Sparse mixture-of-experts (MoE) large language models scale model capacity by routing each token to a small subset of experts. Their routers are regularized with load balancing ter…
JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
Ariel Larey, Elay Dahan, Amit Bleiweiss +15
Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local s…
CSG: A Context-Semantic Guided Diffusion Approach in De Novo Musculoskeletal Ultrasound Image Generation
Elay Dahan, Hedda Cohen Indelman, Angeles M. Perez-Agosto +4
The use of synthetic images in medical imaging Artificial Intelligence (AI) solutions has been shown to be beneficial in addressing the limited availability of diverse, unbiased, a…
Semantic Segmentation Refiner for Ultrasound Applications with Zero-Shot Foundation Models
Hedda Cohen Indelman, Elay Dahan, Angeles M. Perez-Agosto +3
Despite the remarkable success of deep learning in medical imaging analysis, medical image segmentation remains challenging due to the scarcity of high-quality labeled images for s…